| Author |
Message
|
| koen.jans |
 Posted: Tue Oct 16, 2007 11:47 pm Post subject: Check whether queues are empty? |
 |
|
Novice
Joined: 16 Oct 2007
Posts: 12
|
Hello,
i'm facing this problem:
i'm running an application which receives messages from an external application on a queue and processes these messages, hereby using some internal queues.
Now, at some point, i have to shut the listener ports to stop new messages from getting in from the external application. So far, so good. But at this point, i need a check to wait until all messages on the internal queues have been processed.
Any idea how to implement this?
I read queue depths in business logic are not a good idea.
+ i do not know how to get the queue depth, starting from a websphere listener port name / JNDI name in websphere (as i don't want to use 'low level' code that needs connection, port names, .. to do some MQGET or whatever).
Thanks,
Koen |
|
| Back to top |
|
 |
| Vitor |
| Posted: Wed Oct 17, 2007 12:02 am Post subject: Re: Check whether queues are empty? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| koen.jans wrote: |
Now, at some point, i have to shut the listener ports to stop new messages from getting in from the external application. So far, so good. But at this point, i need a check to wait until all messages on the internal queues have been processed.
|
Can you give a bit more detail on your requirement - at what "point" do the inbound messages need to be stopped? A given time of day? After a particular number of messages have been processed? If all the remaining delivered messages must be processed after the flow is stopped, is there some kind of grouping at work?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| koen.jans |
| Posted: Wed Oct 17, 2007 12:07 am Post subject: |
|
|
Novice
Joined: 16 Oct 2007
Posts: 12
|
Well,
The application keeps receiving messages from the external app on queue A. the processing of these messages uses internal queues 1, 2, 3.
At some point, the external app sends a message like 'stop processing' on queue A. At this point the listener port on queue A is stopped, so no new messages from the external application arrive.
But, because it's quite a heavy app, the processing of messages on internal queues 1, 2, 3 might still be busy. When all the messages from these queues are processed (so queues 1, 2 and 3 are empty), a reply must be send to the external application that all processing has stopped. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Oct 17, 2007 12:12 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
So it would seem that all you need to do is either get each of the applications processing queues 1, 2 and 3 to signal that they're out of messages, or employ some kind of monitoring layer to determine that they've stopped.
This is so simple there's clearly a good reason why you're not doing it already & have posted here...... 
What am I missing? 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Wed Oct 17, 2007 12:14 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
is the external app connected through a client connection or MQ Server-Server?
Do you actually stop the listener itself or the channel or does your application stop reading from Queue A?
When processing on Queue 1,2,3 is done you basically want to send a message back saying I am done send me the next batch?
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| koen.jans |
| Posted: Wed Oct 17, 2007 12:19 am Post subject: |
|
|
Novice
Joined: 16 Oct 2007
Posts: 12
|
| Vitor wrote: |
| So it would seem that all you need to do is either get each of the applications processing queues 1, 2 and 3 to signal that they're out of messages, or employ some kind of monitoring layer to determine that they've stopped. |
Well, the processing of the messages on the queues is done using J2EE message driven beans. i don't think an MDB ever "knows" the queue is empty, right? it just processes messages one after another.
My first idea was this: just check that 1, 2 and 3 are empty. But this is clearly not that simple using just jndi names, and not MQ internals.
Oh yeah, another complexity: processing of messages from 1, 2, 3 may put new messages on 1, 2 or 3. So sending an "this is the end" messages to queues 1, 2, 3 is not an option. |
|
| Back to top |
|
|
| koen.jans |
| Posted: Wed Oct 17, 2007 12:22 am Post subject: |
|
|
Novice
Joined: 16 Oct 2007
Posts: 12
|
| Michael Dag wrote: |
| is the external app connected through a client connection or MQ Server-Server? |
It's server-server.
| Michael Dag wrote: |
| Do you actually stop the listener itself or the channel or does your application stop reading from Queue A? |
I stop the websphere listener port for queue A. (an extra software layer was already written to easily manage websphere listener port states)
| Michael Dag wrote: |
| When processing on Queue 1,2,3 is done you basically want to send a message back saying I am done send me the next batch? |
Not really, the external application can perform some master data changes then, that can only be performed when all processing in my application is stopped and fully done. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Oct 17, 2007 12:27 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| koen.jans wrote: |
Well, the processing of the messages on the queues is done using J2EE message driven beans. i don't think an MDB ever "knows" the queue is empty, right? it just processes messages one after another.
|
Ok, so that's what I was missing. 
If you were using or had access to the MQ internals, you'd be able to detect the out-of-message condition via the MQGET. My Java is not first class but I suspect you're right and this is not possible with an MDB - I await some better Java person to correct this suspicion.
Sounds like you need the "stop processing" message to trigger some monitoring as well as stopping the port.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Wed Oct 17, 2007 12:35 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| koen.jans wrote: |
I stop the websphere listener port for queue A. (an extra software layer was already written to easily manage websphere listener port states)
|
still not sure wether you stop the WebSphere MQ Listener or the WebSphere MDB listener? (just want to understand the full picture).
| koen.jans wrote: |
| Not really, the external application can perform some master data changes then, that can only be performed when all processing in my application is stopped and fully done. |
you just build yourself a heavy message affinity solution (use the search facility for affinity), this will come to bite and haunt you... need I say more this is a bad design
Better find a way to get rid of the affinity and store the updates for reprocesing while the original data has not arrived (yet)...
just my 2cents...
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| koen.jans |
| Posted: Wed Oct 17, 2007 12:41 am Post subject: |
|
|
Novice
Joined: 16 Oct 2007
Posts: 12
|
| Michael Dag wrote: |
| need I say more this is a bad design |
I see no other solution here. If the external application can only make changes to their master data when all processing in my application is stopped/done, there has to be some point where a check is done that no processing is going on..
|
|
| Back to top |
|
|
| Michael Dag |
| Posted: Wed Oct 17, 2007 12:44 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| koen.jans wrote: |
| Michael Dag wrote: |
| need I say more this is a bad design |
I see no other solution here. If the external application can only make changes to their master data when all processing in my application is stopped/done, there has to be some point where a check is done that no processing is going on..
|
there is always a 'solution' 
first describe what your application does when an update is received for data that has not been processed yet or is in 'process' and take it from there...
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Oct 17, 2007 2:27 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
In case of a stop processing message you can always use a pcf message to enquire about qdepth and wait until the internal queues all report 0.
As they are primarily fed by the processing of q A this should come about in a short interval of time. The real question here is what do you need to do then that needs the MDB's on the internal queues stopped?
Does that mean you are ready for a new deploy and change to the MDB's??
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| koen.jans |
| Posted: Wed Oct 17, 2007 2:49 am Post subject: |
|
|
Novice
Joined: 16 Oct 2007
Posts: 12
|
| fjb_saper wrote: |
| In case of a stop processing message you can always use a pcf message to enquire about qdepth and wait until the internal queues all report 0. |
This was my first idea too. But I if have no idea how to do this. As i do _not_ want to do something like:
| Code: |
MQEnvironment.hostname = hostName;
MQEnvironment.channel = channel;
MQEnvironment.port = port;
MQQueueManager mqQueueManager = new QQueueManager(qmgrName);
|
This is a websphere deployed application. Queue manager name, hostname, ports, .. are all configured in websphere. You definitely do not want to use hostnames, .. in your application  In the application, i just want to use listenerport names and jndi names.
Ideally, i'd like to do something like:
| Code: |
| listenerPortQueue1.getDestinationQueue().getDepth(); |
Maybe the websphere AdminService could help out in some way.
This is already used in some part of the application now:
| Code: |
ObjectName o = new ObjectName("WebSphere:cell=" +
currentApplicationServerVO.getCellName() + ",node=" +
currentApplicationServerVO.getNodeName() + ",process=" +
currentApplicationServerVO.getServerName() + ",type=[???],*");
Set objectNames = AdminServiceFactory.getAdminService().queryNames(o, null); |
|
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Oct 17, 2007 3:11 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Obviously you looked at support pack MS0B. Well you don't need the connection part of the support pack as you can use the qcf for that. All you need is to be able to interpret the message content and create the message content...
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 17, 2007 3:11 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
This is what I would do.
Propagate a version of the "stop processing" message to each of the internal queues.
As each MDB receives that "stop processing" message, it tells the listener port to stop.
Once the listener port for the MDB is stopped, it will send a reply back to the front end receiver, which when all replies are ready will then send response to external system.
Because the stop processing message will be the last message sent by the front end, it will be the last message on the queue...
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
