| Author |
Message
|
| mqjeff |
 Posted: Tue May 11, 2010 4:37 am Post subject: |
 |
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| fatherjack wrote: |
| mqjeff wrote: |
| You can get this information from ..... the Toolkit |
How? |
From the property of the deployed message flow named "Additional Instances" in the Broker Domain view.
 |
|
| Back to top |
|
 |
| fatherjack |
| Posted: Tue May 11, 2010 4:45 am Post subject: |
|
|
Knight
Joined: 14 Apr 2010
Posts: 522
Location: Craggy Island
|
| mqjeff wrote: |
| From the property of the deployed message flow named "Additional Instances" in the Broker Domain view. |
That tells you how many instances there are. The poster specifcally clarified his requirements when I asked
| fatherjack wrote: |
| Do you mean how many instances are running because that will be whatever you specified when you deployed won't it? Or do you mean how many are actively processing a message at any given time? |
And he replied
| lcl3977 wrote: |
| I want to know how many instances are actively processing message at any given time. |
So not simply instances that are sat there waiting for a message on the input node.
_________________
Never let the facts get in the way of a good theory. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue May 11, 2010 5:08 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| fatherjack wrote: |
| That tells you how many instances there are. |
No, it tells you how many instances can possibly be running simultaneously.
| fatherjack wrote: |
| lcl3977 wrote: |
| I want to know how many instances are actively processing message at any given time. |
|
Ah. This is a much harder problem. Solving it requires monitoring events or accounting&statistics.
There's still no valid business case for using this information inside the flow itself. |
|
| Back to top |
|
|
| Esa |
| Posted: Tue May 11, 2010 6:21 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| mqjeff wrote: |
There's still no valid business case for using this information inside the flow itself. |
Yes there is. It is obvious that lcl3977 wants to implement service level monitoring and protect the backend service from overload by sending "service is busy" replies to http clients after the number of current threads reaches a certain treshold. |
|
| Back to top |
|
|
| Esa |
| Posted: Tue May 11, 2010 6:36 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
I think the solution is maintaining an instance counter the way I described in the beginning of this thread is still a valid solution. It can be implemented with ESQL shared variables as well by putting a flow order node in the beginning of the flow and incrementing the counter after terminal 1 and decrementing it after terminal 2.
I would add a couple of Environment.Variables telling if the counter has already been incremented or decremented in the current instance. The variables should be updated within the same ATOMIC or java synchronized block where the counter is incremented or decremented. That way the flow will know if the counter has to be decremented in the input nodes catch path. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue May 11, 2010 6:38 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Esa wrote: |
| mqjeff wrote: |
There's still no valid business case for using this information inside the flow itself. |
Yes there is. It is obvious that lcl3977 wants to implement service level monitoring and protect the backend service from overload by sending "service is busy" replies to http clients after the number of current threads reaches a certain treshold. |
The problem is much larger here. Say you have configured 9 additional instances of a flow. Say that all 10 instances of the flow are running. This still does not mean that all 10 instances are running at full capacity. So at which time can you send the "service busy" signal?
For that you would need to have
- The flow statistics with min / max / and avg elapsed time
- the number of active instances
- the number of total instances
- if number of total instances * input count * avg elapsed flow time = number of max input count for that time period you can say flow busy...
- Quite difficult to asses...
- It might be easier to say that as long as inputqdepth = 0 the flow is fine... but then how do you count an input queue backing up when the input is SOAP over http?
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue May 11, 2010 7:13 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
The difficulty with your singleton is that it is an instantaneous measure. It's only valid the nanosecond that you inquire against it, and invalid in the next one.
it might even be invalid between the time that you invoke the method to check the value and the time it takes that method to actually return the value.
This is true of all measures of how many instances are active in a given second. They might all be in process of quiescing, and so aren't in use by the time you try to do real work.
If the requirement is really to provide a "service is busy" response INSTEAD of a timeout, then a service proxy needs to be put in place that will capture the timeout of the actual service and return a "service is busy" response.
There is no valid business case for measuring the active number of instances in a given message flow, and even less validity for using that number to make a decision inside that message flow. |
|
| Back to top |
|
|
| Esa |
| Posted: Tue May 11, 2010 9:43 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| lcl3977 wrote: |
my message flow has two ways input,the http input and mq input node.
when a lot of request come in,and the message flow process the message is too slow(because other system give response to my message flow is too slow).
and then the http request will time out.
|
Jeff, yes, maintaining an instance counter could be quite unreliable with a normal message flow that runs an instance in milliseconds. But in this specific case the backend system seems to be considerably slow to response, seconds or even tens of seconds depending on the load. Perhaps an application that is running in one thread only? So most flow instances will be in wait state. I think my approach gives a good enough approximation and can be used to control backend workload, don't you agree? |
|
| Back to top |
|
|
| Esa |
| Posted: Tue May 11, 2010 10:48 pm Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| mqjeff wrote: |
If the requirement is really to provide a "service is busy" response INSTEAD of a timeout, then a service proxy needs to be put in place that will capture the timeout of the actual service and return a "service is busy" response.
There is no valid business case for measuring the active number of instances in a given message flow, and even less validity for using that number to make a decision inside that message flow. |
You don't want to let the instances just time out, because that would cause a lot of unnesecary processing in the backend. So if you can detect that there is a lot of workload, you can stop sending request to the backend service from your flow.
Of course, as always with Message Broker, ANY other coding than what you need for message transformation should be considered only as a last resort if you cannot CONFIGURE the required functionality with the wealth of built-in nodes. So there should be a way to configure some sort of service level management instead of coding it. With MQInput nodes you can adjust the number of additional instances, but that option is not available with HTTPInput nodes. Now let's ask why. Perhaps because this is something that should be configured OUTSIDE Message Broker?
Message Brokers own http listener has a built-in limit ow 50 concurrent threads, so 150 http clients should not be possible. Unless you have overridden the limit. But the limit is in place to make you consider starting to use an external application server instead as the traffic increases. And if you do, you can configure maximum number of concurrent threads per service either with WAS or the http server.
In other words - restricting http traffic should be done in the proper place, which is not Message Broker.
After all this resolution I must finally agree with mqjeff - there is no valid business case. |
|
| Back to top |
|
|
| ashwgupt |
| Posted: Fri Jul 25, 2014 4:46 am Post subject: |
|
|
Novice
Joined: 31 Oct 2011
Posts: 12
|
I am going to revive this question and re-ask what was originally asked.
Now, when there is feature in IIB of seeing Active Threads (Instances) in the web admin console, does anyone know if there is a way to record these and other flow stats values? If yes, we can configure the interval for these flow stats and record as per one's business requirement.
The question then comes is business case. In our case IIB is behaving as a pass-through ESB and is the face to our Web application for all the underline legacy services. Many a times we are caught in net, when business users come with claims of ESB running slow. Where we know that IIB is such a powerful product and can't really go slow in such simple scenarios it becomes difficult to prove to business (as you would understand). Esp for transient issues we lack any proofs to show that it was not running even at the fullest at the time of issue, and the issue was actually from the legacy systems.
For such cases, if we can record the active instances for the flows for all the times, those can be used a proof. We can club those with our Event Monitoring setup to monitoring the ESB/IIB a better way.
Any thoughts are welcome. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Jul 25, 2014 4:55 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Now the purpose becomes clear. For this you do need monitoring and if the backend is too slow it will show up in the average time spent in the flow. As well if you have node statistics info you can then pinpoint to the real bottleneck.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 25, 2014 4:57 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ashwgupt wrote: |
| does anyone know if there is a way to record these and other flow stats values? |
Yes. There's a reference in this thread to the flow statisitics; all the new IIB web interface is doing is subscribing to this information and presenting it in the same way the MB explorer does. So you can't "record" this information through the web interface, but you can record it. This is a different use case to the original poster on this 4 year old thread, as they wanted "real time" count (which is hard & useless) and you want a historical record (which is easy and valuable).
| ashwgupt wrote: |
| If yes, we can configure the interval for these flow stats and record as per one's business requirement. |
It's configurable as described in the documentation.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| ashwgupt |
| Posted: Mon Jul 28, 2014 3:49 am Post subject: |
|
|
Novice
Joined: 31 Oct 2011
Posts: 12
|
Yeah, even I believed so. Thanks for hint Vitor.
It's presented to Web in JSON format, I can try to record it in XML format using mqsichangeflowstats command and see if usable. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jul 28, 2014 6:24 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ashwgupt wrote: |
| I can try to record it in XML format using mqsichangeflowstats command and see if usable. |
The XML format is going to be much easier to work with in your use case.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
