ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » User Exits » Exit event for expired message clean up task

Post new topic  Reply to topic Goto page 1, 2, 3  Next
 Exit event for expired message clean up task « View previous topic :: View next topic » 
Author Message
Zyklark
PostPosted: Fri Oct 17, 2014 10:54 am    Post subject: Exit event for expired message clean up task Reply with quote

Newbie

Joined: 15 Oct 2014
Posts: 4

Hi,

I have set ExpiryInterval property in qm.ini to 5 minutes. Can i configure queue manager to trigger a message every time this expiry message clean up task kicks in? Or is there any api exit that gets called when this task kicks in?

I am programming websphere mq api exits using C. I need to do some function periodically and i dont want to rely on outside systems to send a message to mq for this purpose.

Any suggestions? Thanks!
Back to top
View user's profile Send private message
Vitor
PostPosted: Fri Oct 17, 2014 11:10 am    Post subject: Re: Exit event for expired message clean up task Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

Zyklark wrote:
I have set ExpiryInterval property in qm.ini to 5 minutes. Can i configure queue manager to trigger a message every time this expiry message clean up task kicks in? Or is there any api exit that gets called when this task kicks in?

I am programming websphere mq api exits using C. I need to do some function periodically and i dont want to rely on outside systems to send a message to mq for this purpose.


Ok, so firstly that's a very casual attitude to a very advanced task. MQ exits are an advanced topic and should not be attempted lightly. A badly written exit can do anything from slow the response time of the queue manager to bring the queue manager crashing down. Whatever the problem is, an exit is the last resort not the first one.

Secondly, what function are you trying to perform and why does a C application triggered by an expiry report message arriving as described in your previous post here not fit the bill? It's not an "outside system", is it?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Vitor
PostPosted: Fri Oct 17, 2014 11:15 am    Post subject: Re: Exit event for expired message clean up task Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

Zyklark wrote:
I have set ExpiryInterval property in qm.ini to 5 minutes.


You might also want to think carefully about this. Remember that the scavenger will run on all of the local objects so if you have a large number of queues with any depth it may take 5 minutes to run the process, leaving the queue manager doing nothing but checking for expired messages.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Fri Oct 17, 2014 1:10 pm    Post subject: Re: Exit event for expired message clean up task Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Vitor wrote:
Zyklark wrote:
I have set ExpiryInterval property in qm.ini to 5 minutes.


You might also want to think carefully about this. Remember that the scavenger will run on all of the local objects so if you have a large number of queues with any depth it may take 5 minutes to run the process, leaving the queue manager doing nothing but checking for expired messages.


It might be a more healthy alternative to have a long running process performing an MQGet with wait on the queue to make sure the messages expire in a timely manner. This way you have the timely expiry for your queue and do not impose the expiry reaper interval on the queue manager where there could be legitimate deep queues....

Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
RogerLacroix
PostPosted: Fri Oct 17, 2014 1:41 pm    Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3252
Location: London, ON Canada

To take it one step further, you could create a daemon that does an MQGET based on a bogus CorrelID. The MQGET will always return 2033 and just put it in a loop that sleeps for 300 seconds (5 minutes). Since the MQGET will touch every message in the queue searching for the matching CorrelID, expired messages will be discarded.

You could do something like:
Code:
MQGetMessageOptions gmo = new MQGetMessageOptions();
gmo.options = MQC.MQGMO_FAIL_IF_QUIESCING + MQC.MQGMO_NO_WAIT + MQC.MQGMO_NO_SYNCPOINT;
gmo.matchOptions = MQC.MQMO_MATCH_CORREL_ID

MQMessage receiveMsg = new MQMessage();

while (loopAgain)
{
   receiveMsg.messageId     = MQC.MQMI_NONE;
   receiveMsg.correlationId = "Some bogus CorrelID!!!!".getBytes();  // make sure it is 24 bytes long

   try
   {
      inQ.get(receiveMsg, gmo);
      receiveMsg.clearMessage();
   }
   catch (MQException e)
   {
      if ( (e.completionCode == MQException.MQCC_FAILED)     &&
           (e.reasonCode == MQException.MQRC_NO_MSG_AVAILABLE) )
      {
         // Sleep for 300 seconds
         try { Thread.sleep(300*1000); } catch(Exception ee) {}
      }
      else if ( (e.completionCode == MQException.MQCC_FAILED)     &&
                ( (e.reasonCode == MQException.MQRC_Q_MGR_QUIESCING) ||
                  (e.reasonCode == MQException.MQRC_Q_MGR_STOPPING) ||
                  (e.reasonCode == MQException.MQRC_Q_MGR_NOT_ACTIVE) ||
                  (e.reasonCode == MQException.MQRC_Q_MGR_NOT_AVAILABLE) ) )
      {
         loopAgain = false;  // queue manager is stopping
      }
      else
      {
         System.err.println("MQException: " + e.getMessage());
         System.err.println("MQCC=" + e.completionCode + " : MQRC="+e.reasonCode);

         loopAgain = false;
      }
   }
}

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
Andyh
PostPosted: Sat Oct 18, 2014 4:45 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

The distributed platforms assume all queues are indexed by CorrelId and hence a get by CorrelId will not examine each message.

What is the perceived advantage of a RYO expiry scan, as opposed to the expiry task in the queue manager that provides this service ?
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Sat Oct 18, 2014 6:06 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Andyh wrote:
The distributed platforms assume all queues are indexed by CorrelId and hence a get by CorrelId will not examine each message.

What is the perceived advantage of a RYO expiry scan, as opposed to the expiry task in the queue manager that provides this service ?


Andyh, Does that mean that a get with a non existing correlId will not expire any messages past their expiry time?

I believe where the OP is trying to go, is to have a quasi immediate expiry instead of waiting for the reaper process...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
tczielke
PostPosted: Sat Oct 18, 2014 6:29 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

When I read the original post, it sounds like the user wants to execute some type of user-written function on a scheduled basis, and was hoping to daisy chain off the already scheduled internal ExpiryInterval process to kick off that new function. If so, on Unix/Linux distributed platforms, I have used crontab for something like this, and just have a script run my program on a scheduled basis. Not sure what the platform here is, and maybe I am misunderstanding the post.
Back to top
View user's profile Send private message
Andyh
PostPosted: Sat Oct 18, 2014 8:16 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

The CorrelId index is implemented as a hash table, so it's unpredictable what messages might collide with a non-existent CorrelId.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Sat Oct 18, 2014 8:55 am    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

Andyh wrote:
The CorrelId index is implemented as a hash table, so it's unpredictable what messages might collide with a non-existent CorrelId.


If you are asking for a non existent CorrelID, doesn't that means you have to check every message? Otherwise, how can you know one of the ones you didn't check wouldn't match?

I agree with tczielke understanding of what the OP is after. But its not really clear hence the reason this thread is all over the place.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
tczielke
PostPosted: Sat Oct 18, 2014 10:41 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

Hash tables work where you provide a value (in this case a non-existent CorrellId), and then that is used in a hash function to resolve to a location in the hash table. Sometimes multiple inputs can hash to the same location in the table. That is why there could be multiple messages that the non-existent CorrellId could map to. But a multiple hit would be uncommon (I would think), and would more than likely not account for all the messages on the queue.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Sat Oct 18, 2014 2:55 pm    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

I'm still missing the point of what kind of process would need to be instantiated by a message expiring in a queue - any queue. What is there to clean up? How can a non-consumable message be correlated to something in need of cleaning up?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Sat Oct 18, 2014 5:54 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

bruce2359 wrote:
I'm still missing the point of what kind of process would need to be instantiated by a message expiring in a queue - any queue. What is there to clean up? How can a non-consumable message be correlated to something in need of cleaning up?

The point of the get with a non existent correlId is to "effectively" expire messages beyond their expiry time and not waiting for the reaper process.
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
tczielke
PostPosted: Sat Oct 18, 2014 6:41 pm    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

It is possible that the OP wanted to start some type of functionality completed unrelated to expiring messages, and was just hoping to leverage the already scheduled internal expiry process to launch this other functionality. The OP needs to elaborate more on what they are doing to clear up this ambiguity.

Regarding doing a GET with a non-existent correlid to to effectively expire messages, it sounded like from andyh that this would not be very successful since a hash table is being used to implement the correlid index.

For example, if you PUT 10 messages to a queue and each with it a different correlid, more than likely each correlid would hash to a different slot in the correlid index hash table. Next if you do a GET with a non-existent correlid, all they have to do is hash that non-existent correlid and see if it maps to a slot on the hash table with an entry. More than likely, it will hash to an empty slot, and then they know there are no messages on the queue that could match this non-existent correlid, and therefore no need to check any other messages for a correlid match.
Back to top
View user's profile Send private message
rekarm01
PostPosted: Sat Oct 18, 2014 7:08 pm    Post subject: Re: Exit event for expired message clean up task Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

Zyklark wrote:
I need to do some function periodically and i dont want to rely on outside systems to send a message to mq for this purpose.

Is this function related to the handling of expired messages, or just something that needs to run every 5 minutes? Even if a queue manager's expiry task triggered some message, or called some api exit, that's probably not something that an unrelated function should rely on. Aside from the reasons already given, a queue manager doesn't always scan queues for expired messages, if, for example, it's part of a queue-sharing group, or if there aren't any expired messages to discard. When it does scan queues, the ExpiryInterval is approximate.

fjb_saper wrote:
Andyh, Does that mean that a get with a non existing correlId will not expire any messages past their expiry time?

AndyH may have understated that. A get with a non-matching correlId never discards an expired message, not even with a hashtable collision, at least according to the Knowledge Center:

Quote:
The message is discarded when a browse or nonbrowse MQGET call occurs that would have returned the message had it not already expired.

MQ for z/OS provides two ways for the qmgr to scan queues for expired messages, either by periodic scan, or by explicit request. MQ for distributed platforms doesn't seem to provide either; if that's the case, and if some sort of expiry scan is necessary, then a RYO solution is the only other option. An application that simply browses the desired queues as needed, (with no match options), should be sufficient; optionally providing a zero-length buffer and accept truncated message option should improve the performance.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2, 3  Next Page 1 of 3

MQSeries.net Forum Index » User Exits » Exit event for expired message clean up task
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.