| Author |
Message
|
| mca |
 Posted: Mon Apr 01, 2013 9:27 am Post subject: Issue Deployment on v8 Broker |
 |
|
Disciple
Joined: 09 Mar 2005
Posts: 196
|
We are currently using MQ v7.0.1.9 and MB v8.0.0.1 on our z/Linux platform with 8 GB memory assigned to this server. There are 5 Execution groups defined on the Broker and the message flows use Java (fault handling) and ESQL (SQL server connection) nodes in addition to basic nodes in the message flow.
When the developers try to deploy the BAR file to the execution group it is taking 5+ minutes to deploy via MB toolkit/Explorer (drag/drop) and 15 seconds to deploy to their local Broker same way. We altered the Execution group JVM Max and Min heap size to 512 and 256 MB and it does not make any difference. Except the deployment, everything else is exceptional on the Broker like message processing speed and other performance. There are 3 developers working on the project and they do app. 10 deployments per day to different EGs and they each have to spend one hour waiting for the deployment to be done.
Is there a know issue with this version of Broker. If not, can i get any suggestions on how to rectify this issue. We can also open a PMR with IBM on this. Any advice is much appreciated. |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Mon Apr 01, 2013 9:43 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
z/OS systems, including z/Linux, are much slower than other platforms. Many companies have migrated to AIX. If you look at the z/OS chargeback statistics, even an EG with no resources deployed consumes a boat-load of CPU ticks. Much more than AIX or RHEL.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Apr 01, 2013 10:08 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| z/OS systems, including z/Linux, are much slower than other platforms. |
Care to reconcil this comment with:
| mca wrote: |
| Except the deployment, everything else is exceptional on the Broker like message processing speed and other performance. |
Or do you just feel it's been too long since you were on one of your favourite soapboxes? 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Apr 01, 2013 10:30 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Also, generally it is a good idea to pretend that "zLinux" doesn't include the zed. That is, one should read it as "linux".
It's linux running in an LPAR, it's not akin in any real way to things like USS.
Secondarily, there are a lot of things that could be slowing down deployment. An easy thing to try is to ftp or otherwise move the BAR file to the zlinux image and then issue a local mqsideploy. If that goes very quickly, then the problem is something with the client channel between the toolkit instance and the Broker qmgr. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Apr 01, 2013 10:34 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| Vitor wrote: |
| lancelotlinc wrote: |
| z/OS systems, including z/Linux, are much slower than other platforms. |
Care to reconcil this comment with:
| mca wrote: |
| Except the deployment, everything else is exceptional on the Broker like message processing speed and other performance. |
Or do you just feel it's been too long since you were on one of your favourite soapboxes? |
I've not had the opportunity to measure first-hand the performance of the OP's flows, so who is to say that the CPU consumption is equivocal?. The best way to find out is to measure and post stats.
A no-resource EG running on z/OS has, in the past, and as personally measured, consumed a chunk of CPU time during idle when I compared same to RHEL and AIX. YMMV. It really depends on what the functions are in the OP's flows.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Apr 01, 2013 11:03 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| I've not had the opportunity to measure first-hand the performance of the OP's flows, so who is to say that the CPU consumption is equivocal?. The best way to find out is to measure and post stats. |
And again and with the tired resignation of someone who's been this way before:
CPU consumpition is not speed;
z System CPU cannot be directly compaired meaningfully to distributed usage;
It's a miracle all these people who disagree with you and use z/OS or (z)Linux (point well taken @mqjeff) manage to stay in business. I imagine you'll have a good laugh when they all go broke. Assuming one of them isn't the bank where you keep your money.
And this isn't the OP's problem. He's not talking about general speed, he's talking about deploy speed. Again @mqjeff appears to be on the right track.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mca |
| Posted: Wed Apr 03, 2013 9:36 am Post subject: |
|
|
Disciple
Joined: 09 Mar 2005
Posts: 196
|
We reported the problem of slow deployment to IBM and they came back with this answer...
"Your performance issue could be related to settings on the channel definitions that connect to the Broker queue manager from your workstation. I would make sure the MAXMSGLN attribute for the queue manager, SVRCONN channel and Transmission Queue are all set to 104857600 first off. The reason I would suspect this at first is because deploying locally to a Broker does not require a channel connection to be used.
Another thing I have seen in the past is what the JVM Heap size is set to on your workstation. What usually affects this is a large MRM or SAP IDOC MRM that is being deployed. Sometimes the heap size isn't big enough to efficiently deploy the BAR files containing these"
We already have that MAXMSGLN setting on SVRCONN Channel and Broker. Is there any other Broker setting we are missing here for reducing deployment time? |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Wed Apr 03, 2013 9:50 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| Vitor wrote: |
| lancelotlinc wrote: |
| I've not had the opportunity to measure first-hand the performance of the OP's flows, so who is to say that the CPU consumption is equivocal?. The best way to find out is to measure and post stats. |
And again and with the tired resignation of someone who's been this way before:
CPU consumpition is not speed;
z System CPU cannot be directly compaired meaningfully to distributed usage;
It's a miracle all these people who disagree with you and use z/OS or (z)Linux (point well taken @mqjeff) manage to stay in business. I imagine you'll have a good laugh when they all go broke. Assuming one of them isn't the bank where you keep your money.
And this isn't the OP's problem. He's not talking about general speed, he's talking about deploy speed. Again @mqjeff appears to be on the right track. |
All true statements, my esteemed colleague. My opine relates to efficiency. z Systems are exceptional systems at what they do. I merely point out that other systems are more performant- and cost-efficient.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Apr 03, 2013 10:38 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| mca wrote: |
"Your performance issue could be related to settings on the channel definitions that connect to the Broker queue manager from your workstation. I would make sure the MAXMSGLN attribute for the queue manager, SVRCONN channel and Transmission Queue are all set to 104857600 first off. The reason I would suspect this at first is because deploying locally to a Broker does not require a channel connection to be used.
|
So if the Max Message Length is set to 1 MB and you need to deploy a 1.1 MB bar file, they are saying it will eventually work, just very slow? Really? Or will it immediately fail with a very specific reason code saying the message was to big, and its likely Max Message Size settings have nothing to do with slow performance.
| mca wrote: |
Another thing I have seen in the past is what the JVM Heap size is set to on your workstation. What usually affects this is a large MRM or SAP IDOC MRM that is being deployed. Sometimes the heap size isn't big enough to efficiently deploy the BAR files containing these"
|
Ask them if the same toolkit using the same JVM deploys the same bar file to a local broker and it is fast, how do the JVM settings apply differently to a deploy to a remote Broker?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Apr 03, 2013 10:52 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| lancelotlinc wrote: |
| z/OS systems, including z/Linux, are much slower than other platforms. Many companies have migrated to AIX. If you look at the z/OS chargeback statistics, even an EG with no resources deployed consumes a boat-load of CPU ticks. Much more than AIX or RHEL. |
Poor Soul looking for help:
My 2011 Toyota Corolla seems to go slower at a certain RPM than it did before, what can I check?
lancelotlinc:
A 2013 Ferrari F12berlinetta can go 200+ MPH.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Apr 03, 2013 11:10 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Channel performance and etc. is why I said "copy the bar file to the Broker machine and deploy it there".
You have said it works fine when deploying to a *local* broker, which I assume means "the one on the machine running Toolkit". I am also guessing that is not the same machine as the one that is running the broker that deploys slow.
If you move the bar file to the machine that hosts the broker that deploys slowly, and then you use mqsideploy to deploy to the broker, and it still runs slow, then it's not a channel problem! |
|
| Back to top |
|
|
| NealM |
| Posted: Thu Apr 04, 2013 1:31 pm Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
A couple things:
1. Regarding CPUs, normally when one is running zLinux on a mainframe, one has also bought one or more rather inexpensive IFLs, which processor(s) are utilized by the Broker. See http://www-03.ibm.com/systems/z/os/linux/solutions/ifl.html
(for a capacity test comparison, IFL vs z CP, see http://www.redbooks.ibm.com/Redbooks.nsf/RedbookAbstracts/tips0479.html)
2. We saw the same slowness mentioned above on our zLinux guests, v8 seeming even slower than v6.1 for deploys from a Windows based Toolkit. But one has to consider that with the earlier version, all that was accounted for on the Toolkit was the time to get the objects to the Config Mgr. Watching what was happening on the Broker by putting a tail -f on the user.log, there really wasn't much difference overall between v6.1 and v8
3. I just made the suggested change to maxmsgln on the zLinux broker qm's SYSTEM.BKR.CONFIG server-connection channel, and it does appear to make a difference. Unfortunately I didn't measure the deploy time prior to making the change. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Apr 04, 2013 1:59 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| NealM wrote: |
3. I just made the suggested change to maxmsgln on the zLinux broker qm's SYSTEM.BKR.CONFIG server-connection channel, and it does appear to make a difference. Unfortunately I didn't measure the deploy time prior to making the change. |
Change it back to confirm deploy times go back up, then back again to the max size to see if the times drop again.
If this really does make a difference I would love to know how/why, and be referred to the TechNote or InfoCenter entry that says you need to do this if you expect reasonable deploy times.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| NealM |
| Posted: Thu Apr 04, 2013 3:34 pm Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
OK Peter, I have a bar file that actually fails on the deploy (have an open PMR on that), but it was all I could grab for a quick experiment before the admins took all the test systems down. It is 188KB in size, has a few flows, a bunch of maps and a few common libraries.
There appear to be 3 factors that affect the (attempted) deploy time, in descending order of their effect:
1. Time since last deploy to an EG.
2. Emptiness of the EG
3. Maxmsglen on the svrconn channel
For #1, my first deploy on a non-empty EG (some other flows plus the same library already deployed on it) took 18 seconds (to fail). Tried again a few seconds later, time down to 12 seconds.
For #2, the same bar, same channel size, to an empty EG, 8 secs, then 1 sec,
For #3, to the non-empty EG, chl maxmsglen = 4194304, took 26 secs, then 14 secs.
There does seem to be a lot of cross-checking on the deploy, to make sure nothing is missing, etc. It might be less if independent Applications are deployed instead. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Apr 04, 2013 11:44 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| lancelotlinc wrote: |
| z/OS systems, including z/Linux, are much slower than other platforms. |
Good one, April 1st.  |
|
| Back to top |
|
|
|
|
