| Author |
Message
|
| arunkumar1989 |
 Posted: Thu Mar 21, 2013 12:04 am Post subject: How mq input node get messages from queue |
 |
|
Voyager
Joined: 21 Nov 2012
Posts: 98
Location: Chennai
|
Hi All,
How mq input node get messages from queue?
Detail of my doubt :
whether we are using mq input / file input or any other input node.....
while putting message into queue or file it automatically triggers mq input or corresponding node..... how it is happning......
any one can explain this....
_________________
Being in a crowd when you are alone is ignorance. Enlightenment is being alone in a crowd; a feeling of oneness in a crowd. |
|
| Back to top |
|
 |
| Esa |
| Posted: Thu Mar 21, 2013 12:09 am Post subject: Re: How mq input node get messages from queue |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| arunkumar1989 wrote: |
while putting message into queue or file it automatically triggers mq input or corresponding node..... how it is happning......
|
There are no triggers involved. The node polls the queue (loops with MQGET). FileInput does the similar thing with directories. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Mar 21, 2013 12:35 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Almost, the MQInput node uses MQGET with MQGMO_WAIT. This means that the node will get the message immediately it arrives on the queue with no latency. Another way to code it is the MQI call back method.
Never, ever timer poll a queue (without a MQGMO_WAIT) as it wastes QM resources and causes latency because the message might arrive when you weren't looking! It also causes MQI overhead, which if it is on a MQ client connection would actually be over the network.
The file node has no choice but to timer poll the file system or file server - but then files (and FTP) belongs in a museum of IT. So once again MQ demonstrates superior behaviour compared to use of other protocols. Providing you code the MQI calls properly (which of course WMB does). |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Mar 21, 2013 1:01 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| zpat wrote: |
| Almost, the MQInput node uses MQGET with MQGMO_WAIT. |
Almost,
MQGET with MQGMO_WAIT = 0
This means sit there with a read on the queue until a message is written and importantly COMMITTED. Then the read will complete and bingo! you have your message.
There is no doubt here. That is how MQ apps have been developed for years.(14 years in my case).
MQ Programming 101.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Mar 21, 2013 1:06 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
It may not use an indefinite wait, looking at some MQ statistics would suggest that it issues MQGET with a wait interval, but not indefinite, then repeats the request. Possibly to allow the flow to stop if commanded.
Of course this is standard good MQ programming. But these MQI options are NOT the default in the MQI and a whole new generation of untrained (how often do we hear that!) developers have now assumed that the correct way is to "poll" the queue every second or thereabouts.
Thus causing massive overhead when using MQ client based applications and will cause latency on average of 1/2 the delay between polls. |
|
| Back to top |
|
|
| nukalas2010 |
| Posted: Thu Mar 21, 2013 1:13 am Post subject: |
|
|
Master
Joined: 04 Oct 2010
Posts: 220
Location: Somewhere in the World....
|
| smdavies99 wrote: |
There is no doubt here. That is how MQ apps have been developed for years.(14 years in my case).
MQ Programming 101. |
 |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Mar 21, 2013 1:28 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| zpat wrote: |
It may not use an indefinite wait, looking at some MQ statistics would suggest that it issues MQGET with a wait interval, but not indefinite, then repeats the request. Possibly to allow the flow to stop if commanded.
|
Yes, that's what i meant when I wrote loops with MQGET
MQGET with unlimited wait is AFAIK characteristic of badly behaving MQ applications... |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Mar 21, 2013 1:34 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
It depends how the application queue "listener" is terminated. If it specifies FAIL_IF_QUIESCING then the QMgr can tell it to close down.
Alternatively a closedown message can be sent to the application (assuming it has the logic to act on it).
However I generally code a 30 second wait and when control is returned back with MQRC 2033 (no message available), I check for closedown request in the application and if none, go back for more messages with another MQGET with a 30 second MQGMO_WAIT.
Why do you think an indefinite wait is bad? You're lucky that your developers have even heard of the MQGMO_WAIT option (or in JMS it's called receivetimeout). |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Mar 21, 2013 2:00 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| zpat wrote: |
It depends how the application queue "listener" is terminated. If it specifies FAIL_IF_QUIESCING then the QMgr can tell it to close down.
|
In the case of MQInput node it's very seldom the QMgr that wants to terminate the MQGET call. More likely it's the broker stopping a message flow. I don't think FAIL_IF_QUIESCING helps there (although I'm sure it's specified with the call). Looping with a reasonable wait interval does. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Mar 21, 2013 3:37 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Yes, but we are not writing the WMB code, so surely your comment relates to applications other than IBM written ones? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Mar 21, 2013 3:37 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Esa wrote: |
| zpat wrote: |
It depends how the application queue "listener" is terminated. If it specifies FAIL_IF_QUIESCING then the QMgr can tell it to close down.
|
In the case of MQInput node it's very seldom the QMgr that wants to terminate the MQGET call. More likely it's the broker stopping a message flow. I don't think FAIL_IF_QUIESCING helps there (although I'm sure it's specified with the call). Looping with a reasonable wait interval does. |
I'm not sure I agree with your police work.
If I write a flow MQInput->Compute->MQGet, then I can create a loop that will cause the EG to hang and fail to shut down regardless of whether I specify a timeout on the MQGet or not.
I can also create a loop that will terminate as soon as possible when the queue manager is shutting down, regardless of whether I specify a timeout on the MQGet or not. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Mar 21, 2013 3:41 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
I am referring to the MQI call MQGET performed by the MQinput node.
Using the MQGET node requires proper understanding of how MQI works and is worthy of another thread entirely. |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Mar 21, 2013 5:20 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| zpat wrote: |
| Yes, but we are not writing the WMB code, so surely your comment relates to applications other than IBM written ones? |
No, I'm referring to WMB code. I mean if MQInput nodes issue MQGET with 30 second wait interval, stopping a message flow would take up to 30 seconds. If the broker lets the flow shut itself gracefully.
But you are also claiming that the flows are not shut down gracefully but the broker just kills the thread when it wants to stop a flow.
That's an interesting thought. And probably even the best performing solution. I have no arguments. I'm sure you have reliable sources of information. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Mar 21, 2013 5:38 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Esa wrote: |
| zpat wrote: |
| Yes, but we are not writing the WMB code, so surely your comment relates to applications other than IBM written ones? |
No, I'm referring to WMB code. I mean if MQInput nodes issue MQGET with 30 second wait interval, stopping a message flow would take up to 30 seconds. If the broker lets the flow shut itself gracefully.
But you are also claiming that the flows are not shut down gracefully but the broker just kills the thread when it wants to stop a flow.
That's an interesting thought. And probably even the best performing solution. I have no arguments. I'm sure you have reliable sources of information. |
I'm reasonably positive that MQInput node specifies FAIL-IF-QUIESCING, and does not need to kill any threads.
I'm reasonably sure an MQ API trace would allow you to answer this for sure.
I'm also reasonably positive that in modern versions of Broker, the message flow thread itself is not the thread that is polling any given queue. But, again, a broker service trace would answer this for you. |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Mar 21, 2013 5:53 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| mqjeff wrote: |
I'm reasonably positive that MQInput node specifies FAIL-IF-QUIESCING, and does not need to kill any threads.
|
I'm reasonably positive the broker does not shut down the queue manager when it wants to stop a message flow.
| mqjeff wrote: |
I'm also reasonably positive that in modern versions of Broker, the message flow thread itself is not the thread that is polling any given queue. But, again, a broker service trace would answer this for you. |
Yes, killing the flow thread that is processing a message would be like a kick in the air. I meant of course the thread that is listening to the queue.
| mqjeff wrote: |
I'm reasonably sure an MQ API trace would allow you to answer this for sure. |
Sure, but I'm more interested in arguing than running traces  |
|
| Back to top |
|
|
|
|
