| Author |
Message
|
| SRIkanth1234 |
 Posted: Sat Oct 27, 2012 2:47 pm Post subject: Deployment takes too long time and at times fails |
 |
|
Novice
Joined: 31 Aug 2011
Posts: 16
|
I have a message flow and don't think it's too complex. It's have few long schemas to include and uses WTX maps but are accessed externally. The deployment takes too long on dev and test servers and at times fails with the below error message. I'm using WMB V8.0.0.1 and WTX V8.4.0.3 ..
I did increased the JVM heap size but the deployment time seems to be the same.
Do you advice i need to raise a PMR for this...have you guys encountered long deployments with good size artifacts?
( TCUSBK01.TEST_EG ) The XLXP-C compiler ran out of memory during the preprocessing of XML schemas for the XMLNSC domain.
During the preprocessing of XML schemas for the XMLNSC domain, the XLXP-C compiler ran out of memory.
This has caused the deployment of XML schemas to stop.
Increase the memory available to the XLXP-C compiler by increasing the JVM heap size for the execution group. This can be done using mqsichangeproperties, or the Message Broker Explorer.
After the JVM heap size has been increased, restart the execution group and try the deployment operation again. You may have to increase the JVM heap size multiple times until deployment succeeds.
If you are unable to resolve the problem, contact your IBM support center. |
|
| Back to top |
|
 |
| broker_new |
| Posted: Sat Oct 27, 2012 6:23 pm Post subject: |
|
|
Yatiri
Joined: 30 Nov 2006
Posts: 614
Location: Washington DC
|
1) This is the common problem that we noticed when started using version 8.0.0.1. We increased the JVM heap size to 1 gig then the deployment was successful...the flows were very very simple and had a DFDL associated with it.
2) After overcoming this issue we also noticed that the deployments were taking long time and gets time out, unusual thing is the flows get deployed this is completely unacceptable and opened the PMR..they asked us to collect the trace whenever it happens next time...this problem is completely sporadic.
I will be lucky if i get it captured and sent to IBM for investigation.
3) Sometimes the deployments or any request for the broker simply sit in the queue and gets reflected only on the broker bounce.
It would be nice to have cancel deployments to delete these requests from the broker queue to process...looking forward to open a RFE.
_________________
IBM ->Let's build a smarter planet |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sun Oct 28, 2012 2:36 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Do you observe the same behavior with mqsideploy? |
|
| Back to top |
|
|
| SRIkanth1234 |
| Posted: Mon Oct 29, 2012 7:00 am Post subject: |
|
|
Novice
Joined: 31 Aug 2011
Posts: 16
|
| yes, mqsideploy gives me the same behavior. |
|
| Back to top |
|
|
| mayheminMQ |
| Posted: Mon Nov 05, 2012 9:44 am Post subject: |
|
|
Voyager
Joined: 04 Sep 2012
Posts: 77
Location: UK beyond the meadows of RocknRoll
|
@Broker_new: I dont understand on why is it unusual for response to timeout but the event takes place. It would be unusual if the deployment provided a response as failed and then went on successfully.
We have a few bar files that are about 2MB in size with loads of XSDs(50+). They generally timeout after 204 seconds but do deploy as required.
Also increasing the heap size might not necessarily help with improved deployment times. Might just help with memory usage during processing of the message flows within that EG ( I might be wrong here!!) |
|
| Back to top |
|
|
| Dhiren |
| Posted: Thu Nov 29, 2012 3:58 pm Post subject: |
|
|
Novice
Joined: 27 Jan 2005
Posts: 17
|
@SRIkanth1234 : Did you manage to resolve the issue or identify the root or raise a PMR ?
We have a similar problem, except that it complains about the DFDL compiler running OOM during deployment. We (progressively) increased the JVM Max Heap size to nearly 2GB, but to no avail.
Our Application has a pair of Req-Resp transformation flows:
Request flow SOAPInput node --> Route based on 'Service Operation' --> Transform from to COBOL format (DFDL Domain) --> MQOutput node.
Response Flow MQInput --> retrieve Context -->Transform to SOAP Reply (XMLNC) --> SOAPReplyNode.
There is a common Message Models Library that the app references to and it contains WSDL definition and base schema files + 20 DFDLs (2 per operation).
Broker Log excerpt below :
**********************************************************************
Message: BIP5836E
Source: Runtime Response
Creation time: Tue Nov 27 15:50:35 IST 2012
BIP5836E: The DFDL compiler ran out of memory during the preprocessing of DFDL schemas for the DFDL domain.
During the preprocessing of DFDL schemas for the DFDL domain, the DFDL compiler ran out of memory.
This has caused the deployment of DFDL schemas to stop.
Increase the memory available to the DFDL compiler by increasing the JVM heap size for the execution group. This can be done using mqsichangeproperties, or the Message Broker Explorer.
After the JVM heap size has been increased, restart the execution group and try the deployment operation again. You may have to increase the JVM heap size multiple times until deployment succeeds.
If you are unable to resolve the problem, contact your IBM support center.
**********************************************************************
Any info/guidance will be much appreciated. If we manage to find the root cause / resolution, I will post it here. Thank You in advance. |
|
| Back to top |
|
|
| SnapJudgement |
| Posted: Fri Dec 07, 2012 1:32 am Post subject: |
|
|
Newbie
Joined: 07 Dec 2012
Posts: 1
|
I assume that you have installed WMB 8.0.0.0 originally and then installed the Fixpack 1 ; there by you have WMB 8.0.0.1. Correct?
Could be a dump question but did you uninstall MB 8.0.0.0 version post FP1 installation? If not, you may want to try doing that and have only 8.0.0.1. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Dec 07, 2012 5:50 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| hotshot |
| Posted: Mon Jan 04, 2016 2:27 am Post subject: |
|
|
Novice
Joined: 04 Jan 2013
Posts: 20
|
Happy new year everyone.
Appologies for reviving a more than three year old thread but it seems that I am getting the exact same behaviour while at version 8.0.0.6.
Does anyone have details about the APAR mentioned in the last post? I tried searching through the fix list for versions 8.0.0.1 to 8.0.0.6 and could not find it.
I am trying to deploy an app which references a library with about a dozen XSD schemas. One of them is quite large (148 KB) and has choices of up to 1400 elements.
I was getting the out of memory error mentioned in this thread and managed to deploy the library by increasing the jvm max heap size to 8GB. It took about 20 minutes to get it deployed.
I would like to get some idea on what exactly the broker does whilst preparing the EG for the deployment of the schemas and try and workout what changes I can do to the schemas to influence its behaviour.
Any details or suggestions around this would be highly appreciated.
Thanks,
B |
|
| Back to top |
|
|
| timber |
| Posted: Mon Jan 04, 2016 8:37 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
First, decide whether you actually need to deploy these schemas. You need to deploy them if either
a) you need to perform schema validation using XMLNSC or SOAP domain
or
b) they are DFDL schemas
Assuming that you do need to deploy them, you may find that your options are limited. I know a little about the internals of this part of the product, and I can see how a choice with 1400 branches might cause some problems.
I recommend that you open a PMR - the XSD compiler has been improved quite a lot in recent versions, and you may get lucky. At minimum, you will get some well-informed advice from IBM service. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jan 04, 2016 9:01 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| timber wrote: |
| I can see how a choice with 1400 branches might cause some problems. |
I can see how that would cause some problems outside the pure software world!
Can you give some insights into what design decision have led you to such a chunky schema (148 KB is a lot of XML) with all those choices, and how you manage it?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Jan 04, 2016 9:28 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I'd hope it was possible to create smaller schemas that describe particular messages, that only follow one of the choices... or otherwise only contain a smaller set of choices.
For example if 5 of the choices are all really different types of the same message, and another 20 of another message, and etc.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| hotshot |
| Posted: Mon Jan 04, 2016 12:30 pm Post subject: |
|
|
Novice
Joined: 04 Jan 2013
Posts: 20
|
Thanks for your quick replies.
| Quote: |
First, decide whether you actually need to deploy these schemas. You need to deploy them if either
a) you need to perform schema validation using XMLNSC or SOAP domain |
Yes, I need to deploy them as I am under case a). I am well aware that validating against suge a huge schema will not have the best of performance, but luckily it doesn't need to have a very good throughput, as we only expect a couple of hundred such messages per day.
| Quote: |
| I know a little about the internals of this part of the product. |
Anything that you could share in this forum and that might be of any help?
| Quote: |
I recommend that you open a PMR - the XSD compiler has been improved quite a lot in recent versions, and you may get lucky. At minimum, you will get some well-informed advice from IBM service.
|
That's definitely what it will come to, but since this is not happening for a production system, my hopes for a quick response time are not the best.
| Quote: |
Can you give some insights into what design decision have led you to such a chunky schema (148 KB is a lot of XML) with all those choices, and how you manage it?
|
In theory, all 1400 fields can exist. In practice - or more accurately in the current implementation phase - we only use around 200 of them in different cases. An average XML built against it is under 10 KB. The reason we modeled the other 1200 is because in future phases of the implementation, many others can be added and we want to make try and minimise the schema changes required.
As for its current design, I agree that it seems cumbersome to maintain it (to say the least), but since the field naming convention is quite simple (F1 to F1400), it's not overly complex to track down what you're after.
| Quote: |
I'd hope it was possible to create smaller schemas that describe particular messages, that only follow one of the choices... or otherwise only contain a smaller set of choices.
|
No, that's not the case. There are a couple of different message types modeled, but most of them use the datatype which includes the huge repeatable choice.
I will try and tinker with the schema and see what might get it behaving reasonably. Will update here if I get something useful or if I receive a response from the PMR. |
|
| Back to top |
|
|
| timber |
| Posted: Tue Jan 05, 2016 12:18 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| I would like to get some idea on what exactly the broker does whilst preparing the EG for the deployment of the schemas...Anything that you could share in this forum and that might be of any help? |
I am not going to reveal internal design details. As you have discovered, very large choices ( or wildcards which can match a very large number of global elements ) can lead to very long compilation times. Some aspects of this have been mitigated in previous fix packs which is why I recommend a PMR. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jan 05, 2016 5:49 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I'm surprised that every single message type needs to make a choice between all possible elements.
It strikes me as a terribly poor design of messages... but I get the sense that somehow it's an industry standard format rather than an internally developed format.
If it is an internally developed format, raise a critical defect with the message architecture team. Leave a dead trout on their desk, as well... 
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
|
|
