| Author |
Message
|
| Zyklark |
 Posted: Fri Oct 17, 2014 10:54 am Post subject: Exit event for expired message clean up task |
 |
|
Newbie
Joined: 15 Oct 2014
Posts: 4
|
Hi,
I have set ExpiryInterval property in qm.ini to 5 minutes. Can i configure queue manager to trigger a message every time this expiry message clean up task kicks in? Or is there any api exit that gets called when this task kicks in?
I am programming websphere mq api exits using C. I need to do some function periodically and i dont want to rely on outside systems to send a message to mq for this purpose.
Any suggestions? Thanks! |
|
| Back to top |
|
 |
| Vitor |
| Posted: Fri Oct 17, 2014 11:10 am Post subject: Re: Exit event for expired message clean up task |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Zyklark wrote: |
I have set ExpiryInterval property in qm.ini to 5 minutes. Can i configure queue manager to trigger a message every time this expiry message clean up task kicks in? Or is there any api exit that gets called when this task kicks in?
I am programming websphere mq api exits using C. I need to do some function periodically and i dont want to rely on outside systems to send a message to mq for this purpose. |
Ok, so firstly that's a very casual attitude to a very advanced task. MQ exits are an advanced topic and should not be attempted lightly. A badly written exit can do anything from slow the response time of the queue manager to bring the queue manager crashing down. Whatever the problem is, an exit is the last resort not the first one.
Secondly, what function are you trying to perform and why does a C application triggered by an expiry report message arriving as described in your previous post here not fit the bill? It's not an "outside system", is it?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Oct 17, 2014 11:15 am Post subject: Re: Exit event for expired message clean up task |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Zyklark wrote: |
| I have set ExpiryInterval property in qm.ini to 5 minutes. |
You might also want to think carefully about this. Remember that the scavenger will run on all of the local objects so if you have a large number of queues with any depth it may take 5 minutes to run the process, leaving the queue manager doing nothing but checking for expired messages.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Oct 17, 2014 1:10 pm Post subject: Re: Exit event for expired message clean up task |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Vitor wrote: |
| Zyklark wrote: |
| I have set ExpiryInterval property in qm.ini to 5 minutes. |
You might also want to think carefully about this. Remember that the scavenger will run on all of the local objects so if you have a large number of queues with any depth it may take 5 minutes to run the process, leaving the queue manager doing nothing but checking for expired messages. |
It might be a more healthy alternative to have a long running process performing an MQGet with wait on the queue to make sure the messages expire in a timely manner. This way you have the timely expiry for your queue and do not impose the expiry reaper interval on the queue manager where there could be legitimate deep queues....
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Fri Oct 17, 2014 1:41 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3264
Location: London, ON Canada
|
To take it one step further, you could create a daemon that does an MQGET based on a bogus CorrelID. The MQGET will always return 2033 and just put it in a loop that sleeps for 300 seconds (5 minutes). Since the MQGET will touch every message in the queue searching for the matching CorrelID, expired messages will be discarded.
You could do something like:
| Code: |
MQGetMessageOptions gmo = new MQGetMessageOptions();
gmo.options = MQC.MQGMO_FAIL_IF_QUIESCING + MQC.MQGMO_NO_WAIT + MQC.MQGMO_NO_SYNCPOINT;
gmo.matchOptions = MQC.MQMO_MATCH_CORREL_ID
MQMessage receiveMsg = new MQMessage();
while (loopAgain)
{
receiveMsg.messageId = MQC.MQMI_NONE;
receiveMsg.correlationId = "Some bogus CorrelID!!!!".getBytes(); // make sure it is 24 bytes long
try
{
inQ.get(receiveMsg, gmo);
receiveMsg.clearMessage();
}
catch (MQException e)
{
if ( (e.completionCode == MQException.MQCC_FAILED) &&
(e.reasonCode == MQException.MQRC_NO_MSG_AVAILABLE) )
{
// Sleep for 300 seconds
try { Thread.sleep(300*1000); } catch(Exception ee) {}
}
else if ( (e.completionCode == MQException.MQCC_FAILED) &&
( (e.reasonCode == MQException.MQRC_Q_MGR_QUIESCING) ||
(e.reasonCode == MQException.MQRC_Q_MGR_STOPPING) ||
(e.reasonCode == MQException.MQRC_Q_MGR_NOT_ACTIVE) ||
(e.reasonCode == MQException.MQRC_Q_MGR_NOT_AVAILABLE) ) )
{
loopAgain = false; // queue manager is stopping
}
else
{
System.err.println("MQException: " + e.getMessage());
System.err.println("MQCC=" + e.completionCode + " : MQRC="+e.reasonCode);
loopAgain = false;
}
}
} |
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| Andyh |
| Posted: Sat Oct 18, 2014 4:45 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
The distributed platforms assume all queues are indexed by CorrelId and hence a get by CorrelId will not examine each message.
What is the perceived advantage of a RYO expiry scan, as opposed to the expiry task in the queue manager that provides this service ? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Oct 18, 2014 6:06 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Andyh wrote: |
The distributed platforms assume all queues are indexed by CorrelId and hence a get by CorrelId will not examine each message.
What is the perceived advantage of a RYO expiry scan, as opposed to the expiry task in the queue manager that provides this service ? |
Andyh, Does that mean that a get with a non existing correlId will not expire any messages past their expiry time?  
I believe where the OP is trying to go, is to have a quasi immediate expiry instead of waiting for the reaper process...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| tczielke |
| Posted: Sat Oct 18, 2014 6:29 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
| When I read the original post, it sounds like the user wants to execute some type of user-written function on a scheduled basis, and was hoping to daisy chain off the already scheduled internal ExpiryInterval process to kick off that new function. If so, on Unix/Linux distributed platforms, I have used crontab for something like this, and just have a script run my program on a scheduled basis. Not sure what the platform here is, and maybe I am misunderstanding the post. |
|
| Back to top |
|
|
| Andyh |
| Posted: Sat Oct 18, 2014 8:16 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
| The CorrelId index is implemented as a hash table, so it's unpredictable what messages might collide with a non-existent CorrelId. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sat Oct 18, 2014 8:55 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Andyh wrote: |
| The CorrelId index is implemented as a hash table, so it's unpredictable what messages might collide with a non-existent CorrelId. |
If you are asking for a non existent CorrelID, doesn't that means you have to check every message? Otherwise, how can you know one of the ones you didn't check wouldn't match?
I agree with tczielke understanding of what the OP is after. But its not really clear hence the reason this thread is all over the place.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| tczielke |
| Posted: Sat Oct 18, 2014 10:41 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
| Hash tables work where you provide a value (in this case a non-existent CorrellId), and then that is used in a hash function to resolve to a location in the hash table. Sometimes multiple inputs can hash to the same location in the table. That is why there could be multiple messages that the non-existent CorrellId could map to. But a multiple hit would be uncommon (I would think), and would more than likely not account for all the messages on the queue. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sat Oct 18, 2014 2:55 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
I'm still missing the point of what kind of process would need to be instantiated by a message expiring in a queue - any queue. What is there to clean up? How can a non-consumable message be correlated to something in need of cleaning up?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Oct 18, 2014 5:54 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| bruce2359 wrote: |
| I'm still missing the point of what kind of process would need to be instantiated by a message expiring in a queue - any queue. What is there to clean up? How can a non-consumable message be correlated to something in need of cleaning up? |
The point of the get with a non existent correlId is to "effectively" expire messages beyond their expiry time and not waiting for the reaper process.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| tczielke |
| Posted: Sat Oct 18, 2014 6:41 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
It is possible that the OP wanted to start some type of functionality completed unrelated to expiring messages, and was just hoping to leverage the already scheduled internal expiry process to launch this other functionality. The OP needs to elaborate more on what they are doing to clear up this ambiguity.
Regarding doing a GET with a non-existent correlid to to effectively expire messages, it sounded like from andyh that this would not be very successful since a hash table is being used to implement the correlid index.
For example, if you PUT 10 messages to a queue and each with it a different correlid, more than likely each correlid would hash to a different slot in the correlid index hash table. Next if you do a GET with a non-existent correlid, all they have to do is hash that non-existent correlid and see if it maps to a slot on the hash table with an entry. More than likely, it will hash to an empty slot, and then they know there are no messages on the queue that could match this non-existent correlid, and therefore no need to check any other messages for a correlid match. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sat Oct 18, 2014 7:08 pm Post subject: Re: Exit event for expired message clean up task |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| Zyklark wrote: |
| I need to do some function periodically and i dont want to rely on outside systems to send a message to mq for this purpose. |
Is this function related to the handling of expired messages, or just something that needs to run every 5 minutes? Even if a queue manager's expiry task triggered some message, or called some api exit, that's probably not something that an unrelated function should rely on. Aside from the reasons already given, a queue manager doesn't always scan queues for expired messages, if, for example, it's part of a queue-sharing group, or if there aren't any expired messages to discard. When it does scan queues, the ExpiryInterval is approximate.
| fjb_saper wrote: |
| Andyh, Does that mean that a get with a non existing correlId will not expire any messages past their expiry time? |
AndyH may have understated that. A get with a non-matching correlId never discards an expired message, not even with a hashtable collision, at least according to the Knowledge Center:
| Quote: |
| The message is discarded when a browse or nonbrowse MQGET call occurs that would have returned the message had it not already expired. |
MQ for z/OS provides two ways for the qmgr to scan queues for expired messages, either by periodic scan, or by explicit request. MQ for distributed platforms doesn't seem to provide either; if that's the case, and if some sort of expiry scan is necessary, then a RYO solution is the only other option. An application that simply browses the desired queues as needed, (with no match options), should be sufficient; optionally providing a zero-length buffer and accept truncated message option should improve the performance. |
|
| Back to top |
|
|
|
|
