| Author |
Message
|
| lancelotlinc |
 Posted: Fri Jul 01, 2011 6:58 am Post subject: z/OS WMB Performance Observations |
 |
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
There is a glitch in z/OS that inhibits good performance scalability, because of the amount of time spent by the CPU processing each message.
This can be seen in this document, page 49:
ftp://public.dhe.ibm.com/software/integration/support/supportpacs/individual/ip14.pdf
"This drop in throughput for 8 instances when compared with 4 instances is being investigated and a fix will be shipped in due course. Contact IBM service for the latest status on this problem."
The fix may already be in GA. I don't know about its status.
In any event, as late as last year, I observed that z/OS apple-to-apple comparison of a standard use case with other platforms is dramatically different. I suspect another contributing factor is the way z/OS manages its memory. Other more modern architectures have vastly larger memory pools, especially for the JVM.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
 |
| Vitor |
| Posted: Fri Jul 01, 2011 7:06 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
For new readers, this is the new thread talked about here
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 01, 2011 7:13 am Post subject: Re: z/OS WMB Performance Observations |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Given that the document refers to WMBv6.0 dated 2006 you'd hope it would be out by now. It's also (to be clear) an issue with WMB rather than z/OS per se yes?
| lancelotlinc wrote: |
| In any event, as late as last year, I observed that z/OS apple-to-apple comparison of a standard use case with other platforms is dramatically different. |
At which time we can assume you were not using WMBv6.0.
So what was the use case? How were the apples configured on z/OS and the other platforms? What were the other platforms? The devil is in the detail, and there's a lot of detail in z/OS. If (for example) it was a Dev WMB on an LPAR throttled right back that has an impact. If one of the other platforms was one of your Power7 machines the size of a house with "Mom" tattooed down a processor board that too has an impact.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Jul 01, 2011 7:35 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
IDK about the root cause of the issue, but I have heard rumors that it is related to how z/OS starts and manages additional instances compared to how the other OSs do it. This leads me to believe it is related to OS issues more than WMB binary issues. Hursely (Dunn, et. al.) may shed some light about it.
It really doesn't matter what the use cases are as long as they are consistent between the platforms.
A favorite use case of mine is a 528 byte payload of copybook data converting to XML, also copy Msg Id to Correl Id via MQ transport. I find AIX and Linux outperform z/OS consistently, especially with additional instances.
On AIX Power7, achievement of close to 10k TPS with average latencies between 130 and 450 ms on 8 EGs with multiple instances in each EG for several hours (ie. not a spike; consistent performance). z/OS no where near this, best I could coax out of z/OS was less than 100 TPS. Sometimes latencies of indivdual transactions on z/OS extended to 7 seconds. RHEL beefy system (8 CPUs Intel Xeon X5550 4 cores @ 2.67 Ghz) could approach 3,000 TPS on this use case.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 01, 2011 7:44 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| IDK about the root cause of the issue |
Nor do I unless I'm required to fix it in the OS, or work round it in a WMB installation. But I do try to be interested in such things.
| lancelotlinc wrote: |
| I have heard rumors that it is related to how z/OS starts and manages additional instances compared to how the other OSs do it. |
Spill - who, where, when? I know it's bad to spread gossip but what else can you do with it??? 
| lancelotlinc wrote: |
| This leads me to believe it is related to OS issues more than WMB binary issues. Hursely (Dunn, et. al.) may shed some light about it. |
We can hope.
| lancelotlinc wrote: |
| It really doesn't matter what the use cases are as long as they are consistent between the platforms. |
It does if other readers fancy having a go at their own comparisions.
| lancelotlinc wrote: |
| A favorite use case of mine is a 528 byte payload of copybook data converting to XML, also copy Msg Id to Correl Id via MQ transport. |
Now that wasn't difficult, was it?
| lancelotlinc wrote: |
| I find AIX and Linux outperform z/OS consistently, especially with additional instances. |
And it will be interesting to see if people can get equivalent results.
| lancelotlinc wrote: |
| On AIX Power7, achievement of close to 10k TPS with average latencies between 130 and 450 ms on 8 EGs with multiple instances in each EG for several hours (ie. not a spike; consistent performance). z/OS no where near this, best I could coax out of z/OS was less than 100 TPS. Sometimes latencies of indivdual transactions on z/OS extended to 7 seconds. RHEL beefy system (8 CPUs Intel Xeon X5550 4 cores @ 2.67 Ghz) could approach 3,000 TPS on this use case. |
You have access to such a rich mix of technologies. Interesting to see what results the lurking classes can get.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Jul 01, 2011 7:56 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| Vitor wrote: |
| lancelotlinc wrote: |
| IDK about the root cause of the issue |
Nor do I unless I'm required to fix it in the OS, or work round it in a WMB installation. But I do try to be interested in such things.
| lancelotlinc wrote: |
| I have heard rumors that it is related to how z/OS starts and manages additional instances compared to how the other OSs do it. |
Spill - who, where, when? I know it's bad to spread gossip but what else can you do with it???
| lancelotlinc wrote: |
| This leads me to believe it is related to OS issues more than WMB binary issues. Hursely (Dunn, et. al.) may shed some light about it. |
We can hope.
| lancelotlinc wrote: |
| It really doesn't matter what the use cases are as long as they are consistent between the platforms. |
It does if other readers fancy having a go at their own comparisions.
| lancelotlinc wrote: |
| A favorite use case of mine is a 528 byte payload of copybook data converting to XML, also copy Msg Id to Correl Id via MQ transport. |
Now that wasn't difficult, was it?
| lancelotlinc wrote: |
| I find AIX and Linux outperform z/OS consistently, especially with additional instances. |
And it will be interesting to see if people can get equivalent results.
| lancelotlinc wrote: |
| On AIX Power7, achievement of close to 10k TPS with average latencies between 130 and 450 ms on 8 EGs with multiple instances in each EG for several hours (ie. not a spike; consistent performance). z/OS no where near this, best I could coax out of z/OS was less than 100 TPS. Sometimes latencies of indivdual transactions on z/OS extended to 7 seconds. RHEL beefy system (8 CPUs Intel Xeon X5550 4 cores @ 2.67 Ghz) could approach 3,000 TPS on this use case. |
You have access to such a rich mix of technologies. Interesting to see what results the lurking classes can get. |
I hope others perform an apple-to-apple comparison of similar tests with the technologies they have access to. I would like to see what they can achieve performance wise.
When I went to WMB v5 training in Raleigh, NC in 2004, the instructor said z/OS has a problem with WMB because it cannot manage the same granularity of processes that Unix can. By the way, the hallmark seafood resturant downtown was the best. Absolutely stunning.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 01, 2011 8:12 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| When I went to WMB v5 training in Raleigh, NC in 2004, the instructor said z/OS has a problem with WMB because it cannot manage the same granularity of processes that Unix can. By the way, the hallmark seafood resturant downtown was the best. Absolutely stunning. |
It'll be interesting to know to what extent this is true 7 years and 2 versions on (given that v5 had more issues than performance).
Also interesting to know if the resturant is still open 7 years on. Measure of the downturn and so forth.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Jul 01, 2011 9:01 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
@lancelotlinc
Did you ever measure performance on zlinux? If yes care to share? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Jul 05, 2011 4:30 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| rbicheno |
| Posted: Tue Jul 05, 2011 4:50 am Post subject: |
|
|
Apprentice
Joined: 07 Jul 2009
Posts: 43
|
Surprises me in this performance discussion that no one has mentioned the Updated (v6.1 and v7) Message Broker Performance Reports?
http://www-01.ibm.com/support/docview.wss?uid=swg27007150
They were produced on all platforms for v6.1 and available for several on v7. Generally the same use cases were run across all platforms. Whist i hate people comparing platforms against each other with these due to hardware differences they do show broker is performance tested and scales/performs well on ALL platforms.  |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Jul 05, 2011 4:55 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| rbicheno wrote: |
Surprises me in this performance discussion that no one has mentioned the Updated (v6.1 and v7) Message Broker Performance Reports?
http://www-01.ibm.com/support/docview.wss?uid=swg27007150
They were produced on all platforms for v6.1 and available for several on v7. Generally the same use cases were run across all platforms. Whist i hate people comparing platforms against each other with these due to hardware differences they do show broker is performance tested and scales/performs well on ALL platforms. |
This is not true for classic z, when compared with the scalability of the others.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jul 05, 2011 5:02 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| rbicheno wrote: |
Surprises me in this performance discussion that no one has mentioned the Updated (v6.1 and v7) Message Broker Performance Reports?
http://www-01.ibm.com/support/docview.wss?uid=swg27007150
They were produced on all platforms for v6.1 and available for several on v7. Generally the same use cases were run across all platforms. Whist i hate people comparing platforms against each other with these due to hardware differences they do show broker is performance tested and scales/performs well on ALL platforms. |
This is not true for classic z, when compared with the scalability of the others. |
I must have missed where it says that..... 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| rbicheno |
| Posted: Tue Jul 05, 2011 5:25 am Post subject: |
|
|
Apprentice
Joined: 07 Jul 2009
Posts: 43
|
| The comparisons of z/OS vs AIX/RHEL above are flawed you maybe running the same use case but you are comparing hardware which has difference speeds and numbers of CPU. It is this and not WMB or the OS which is the overriding reasons for the performance differences you see. Performing comparisons like this is not meaningful/useful. A more common/interesting comparison is TCO per platform i.e. $ per message. Which takes into account hardware costs and the performance achievable on that hardware. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Jul 05, 2011 5:26 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Jul 05, 2011 5:29 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| rbicheno wrote: |
| The comparisons of z/OS vs AIX/RHEL above are flawed you maybe running the same use case but you are comparing hardware which has difference speeds and numbers of CPU. It is this and not WMB or the OS which is the overriding reasons for the performance differences you see. Performing comparisons like this is not meaningful/useful. A more common/interesting comparison is TCO per platform i.e. $ per message. Which takes into account hardware costs and the performance achievable on that hardware. |
I would agree with you if my audience were business owners. As for my experience, I find that business owners stick with z/OS out of allegiance to things other than technical merit.
TCO for WMB on Power7 is four transactions per US$0.01 (penny). z/OS cannot come even close to that number.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
|
|
