| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| How threads work in Broker? |
« View previous topic :: View next topic » |
| Author |
Message
|
| mqjeff |
 Posted: Sat Apr 21, 2012 10:13 am Post subject: |
 |
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Have you considered examining the available Accounting And Statistics that are provided to allow you to monitor the performance of your Broker environment to see if it allows you to monitor the performance of your SOAP nodes? |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Sat Apr 21, 2012 3:48 pm Post subject: Re: Swamping the number of threads on SOAP input (cont...) |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| MikeH wrote: |
I made a test jig and worked it to get the answer.
The SOAP input node manages the requests and the ones that time out may never get to the message flow instance. This is great for efficiency but you simply never know if the work came and went before you started processing.
So the real answer is ...
You can't find out if the broker is falling behind on a SOAP input node.
Well, you can but then you end up staging work on a queue and handling the timeouts yourself. Which isn't really the "as fast as possible" approach that I need.
Mike |
I'll have to disagree with you here Mike. As I stated above, using a Singleton, you can measure latency to five decimal places. No queues involved.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Apr 21, 2012 8:12 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| lancelotlinc wrote: |
With real data that needs to be parsed, you may find that SOAPInput node is the more efficient method than SOAP over HTTPInput. With dummy data that is not parsed, the opposite is true.
To measure internal performance, use a Singleton through a Java Compute node to collect statistics in JMX MBeans. Track latency through the MBeans. When Broker starts to fall behind, the latency will expand. |
Can you expand a little? What statistics exactly are you capturing in the java compute node? and what change in the statistics shows increased lag time? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Sun Apr 22, 2012 4:28 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
SOAPInput -> JCN or ESQL Compute [measurement only "A"]-> Trace node -> SOAPRequest or ESQL Compute or any other transformation/routing activity -> Trace node -> SOAPReply -> JCN or ESQL Compute [measurement only "B"].
You are measuring many things, but primarily in relation to the OP's question, the distance in time from "A" to "B". Under nominal load, you will achieve a certain latency "AB-prime". Because the "A" and "B" nodes call a Singleton, which marks various things into one or more JMX MBeans, using jconsole, you can view the statistics in real-time as you tweak various parameters, like number of threads, etc. For a given set of parameters (threads or other variables), you will achieve a reliable maximum transaction-per-second throughput without increase in latency "AB-prime-one". As you increase the TPS demand past that point, you will notice the latency between A and B increase. If you define an acceptable deviation, say ten percent - "AB-prime-two", you can say that at so-and-so TPS, the latency becomes unacceptable. Perform several cycles with different parameters. Config 1, Config 2, Config 3. Each config will have a different AB-prime, AB-prime-one, and AB-prime-two. Some scenarios have limitations caused by external resources (ie. downstream). Some do not.
Mike's assertion that you cannot measure latency or WMB work backlog without queues is incorrect. You can use JMX MBeans and jconsole to measure many things that are internal to Broker. I chiefly use these tools to measure performance and help identify areas that can be improved. For example, in many of my flows, a main flow might have several subflows, each subflow making a downstream call. Each subflow will report through the Singleton and JMX framework, transaction start time, transaction end time and distance between the two. Using real-time jconsole view, I can see which of the many downstream calls are not within acceptable standards in real-time and tweak some internal variables that may influence how the performance is achieved.
Another useful trick with JMX is to capture and display the system environment, which may help diagnose issues. As you promote your flows from sandbox to dev to integration to perf test to pen-vul to prod to hot-fix, your environment is at the mercy of system admins, not necessarily under your direct control. The use of some insight through JMX to see these differences can shorten the time to make an operational recommendation to change some environment parameter.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sun Apr 22, 2012 6:36 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| lancelotlinc wrote: |
SOAPInput -> JCN or ESQL Compute [measurement only "A"]-> Trace node -> SOAPRequest or ESQL Compute or any other transformation/routing activity -> Trace node -> SOAPReply -> JCN or ESQL Compute [measurement only "B"].
You are measuring many things, but primarily in relation to the OP's question, the distance in time from "A" to "B". Under nominal load, you will achieve a certain latency "AB-prime". |
SO, the same numbers you could get by enabling monitoring and a minimum monitoring profile on the SOAPInput and SOAPReply nodes.
But, you know, JMXy so it's cool? |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sun Apr 22, 2012 4:13 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| lancelotlinc wrote: |
SOAPInput -> JCN or ESQL Compute [measurement only "A"]-> Trace node -> SOAPRequest or ESQL Compute or any other transformation/routing activity -> Trace node -> SOAPReply -> JCN or ESQL Compute [measurement only "B"].
You are measuring many things, but primarily in relation to the OP's question, the distance in time from "A" to "B". |
No, and no. The "OP" refers to "w", who asked a completely different question. "MikeH" seemed more interested in measuring the latency from client -> SOAPInput, before it gets to "A". The proposed singleton doesn't seem to measure that at all. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Apr 23, 2012 5:00 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| rekarm01 wrote: |
| lancelotlinc wrote: |
SOAPInput -> JCN or ESQL Compute [measurement only "A"]-> Trace node -> SOAPRequest or ESQL Compute or any other transformation/routing activity -> Trace node -> SOAPReply -> JCN or ESQL Compute [measurement only "B"].
You are measuring many things, but primarily in relation to the OP's question, the distance in time from "A" to "B". |
No, and no. The "OP" refers to "w", who asked a completely different question. "MikeH" seemed more interested in measuring the latency from client -> SOAPInput, before it gets to "A". The proposed singleton doesn't seem to measure that at all. |
Granted, with the OP reference. Mike asked about WMB backlog and knowing how much is too much. My hypothesis is, if the latency starts to increase, then you can make an assumption that the backlog is also increasing (aside from external factors).
In any event, the solution I expanded on is one way to do that. There may be others.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Apr 23, 2012 5:19 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| mqjeff wrote: |
| lancelotlinc wrote: |
SOAPInput -> JCN or ESQL Compute [measurement only "A"]-> Trace node -> SOAPRequest or ESQL Compute or any other transformation/routing activity -> Trace node -> SOAPReply -> JCN or ESQL Compute [measurement only "B"].
You are measuring many things, but primarily in relation to the OP's question, the distance in time from "A" to "B". Under nominal load, you will achieve a certain latency "AB-prime". |
SO, the same numbers you could get by enabling monitoring and a minimum monitoring profile on the SOAPInput and SOAPReply nodes.
But, you know, JMXy so it's cool? |
This is true. I would really like to see more people use the approach you advocate. One hindrance to wider acceptance of that approach is the licensing issue for WebSphere Business Monitor (WBM). One PVU is $1,020. A minimally configured RedHat Linux system needs 100 of those PVUs to successfully license WBM.
With JMX, there is no license fee due.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Apr 23, 2012 5:27 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| This is true. I would really like to see more people use the approach you advocate. One hindrance to wider acceptance of that approach is the licensing issue for WebSphere Business Monitor (WBM). One PVU is $1,020. A minimally configured RedHat Linux system needs 100 of those PVUs to successfully license WBM. |
Though you don't need WBM to implement the advocated approach. You get a lot of mileage out of it, some cool tricks and all sorts of good stuff but you don't need it.
Which you know, but clarifying for future readers who could get the impression that you can't use monitoring profiles without WBM.
The debate on PVU cost for WBM is a separate one where I'm firmly on the fence. It's expensive for a monitor, but it's so much more than a monitor, so how do you price it? 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Apr 23, 2012 5:31 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| Vitor wrote: |
| lancelotlinc wrote: |
| This is true. I would really like to see more people use the approach you advocate. One hindrance to wider acceptance of that approach is the licensing issue for WebSphere Business Monitor (WBM). One PVU is $1,020. A minimally configured RedHat Linux system needs 100 of those PVUs to successfully license WBM. |
Though you don't need WBM to implement the advocated approach. You get a lot of mileage out of it, some cool tricks and all sorts of good stuff but you don't need it.
Which you know, but clarifying for future readers who could get the impression that you can't use monitoring profiles without WBM.
The debate on PVU cost for WBM is a separate one where I'm firmly on the fence. It's expensive for a monitor, but it's so much more than a monitor, so how do you price it? |
Personally, I think it's priced right. I agree with you that the title of the product does not reflect all the functionality in the product. Because the only exposure a manager has is the title, and the fact that most managers don't do due diligence on what the product really does, you get a doe-in-the-headlights stare when you ask for WBM. Some way to better sell WBM capabilities is needed. Its a challenge, even for the most experienced IBM sales rep.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Tue Apr 24, 2012 1:31 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| lancelotlinc wrote: |
| My hypothesis is, if the latency starts to increase, then you can make an assumption that the backlog is also increasing (aside from external factors). |
Even if the proposed JCNs can calculate an optimal throughput, they still can't detect or measure an input backlog. It's entirely possible that the input backlog increases without affecting the message flow latency, or that the message flow latency increases without affecting the input backlog, (even aside from external factors). Without any evidence at all to establish a correlation between the two, it's not reasonable to assume that they're correlated.
| lancelotlinc wrote: |
| In any event, the solution I expanded on is one way to do that. |
"Solution" isn't quite the right word, if it doesn't really solve the problem. |
|
| Back to top |
|
|
| iamfat |
| Posted: Tue Apr 24, 2012 4:47 pm Post subject: |
|
|
Apprentice
Joined: 08 Mar 2012
Posts: 32
Location: China
|
This topic is very helpful for me.
I tested in my pc. i modify the 'additional instances to 2.
And use broker toolkit and soap ui to sent 2 soap request.
Then in the debug view, i saw two thread stop at the break point.
So it's multi-thread now, |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Wed Apr 25, 2012 4:59 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| rekarm01 wrote: |
| lancelotlinc wrote: |
| My hypothesis is, if the latency starts to increase, then you can make an assumption that the backlog is also increasing (aside from external factors). |
Even if the proposed JCNs can calculate an optimal throughput, they still can't detect or measure an input backlog. It's entirely possible that the input backlog increases without affecting the message flow latency, or that the message flow latency increases without affecting the input backlog, (even aside from external factors). Without any evidence at all to establish a correlation between the two, it's not reasonable to assume that they're correlated.
| lancelotlinc wrote: |
| In any event, the solution I expanded on is one way to do that. |
"Solution" isn't quite the right word, if it doesn't really solve the problem. |
Other people may find other solutions more effective for them. I use soapReplyId to correlate activities. You are correct that many factors combine to form the total miles per gallon, your mileage may vary, even if the car models are the same model year. An effective performance improvement program requires consistent effort and an open mind to try new things.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
