| Author |
Message
|
| Adreno |
 Posted: Mon Jan 28, 2008 1:04 am Post subject: Processing actual messages after long downtimes . How to? |
 |
|
Acolyte
Joined: 05 Jul 2007
Posts: 71
|
Hey Dudes,
how can I realize the following scenario:
I got 2 QMgrs on 2 seperate machines, not running in a cluster. On the first machine I also got C# applications, which are communicating with a SQL Server (also local) and putting different status messages to a remote queue that points to a local queue on the second qmgr.
While the tcp/ip connection ist online everything workes fine, but in normal times due to customers network problems there is a downtime of 2 - 3 days a week. During that downtimes the messages stack on the transmission queue, which has been sized big enough for a worst case downtime of 7 days.
Now the requirements:
After that downtimes the actual messages that are just coming into the queue from application side should be transmitted first, because they have actual process relevance. The messages that stuck on the transmit queue during the downtime aren´t process relevant any longer, but they also need to be transfered for reviewing, but after all the actual messages.
How would you solve this?? - - - Working with priority isn`t possible.
_________________
There are two kinds of people in this world: people who s***, and Chuck Norris... |
|
| Back to top |
|
 |
| Vitor |
| Posted: Mon Jan 28, 2008 1:36 am Post subject: Re: Processing actual messages after long downtimes . How to |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Adreno wrote: |
The messages that stuck on the transmit queue during the downtime aren´t process relevant any longer, but they also need to be transfered for reviewing, but after all the actual messages.
|
This seems to be a contradictory requirement. If the "old" status messages are not relevent because they've been overtaken by event and more recent messages have been produced then these old messages should be discarded. If they need to be reviewed because they need to form part of a chain then how can the "new" messages be processed first if the old ones will then be delivered with a back status?
If the requirement is actually needed (and not some auditor claiming you need to review all the status messages because they're status messages) then you might want to consider using retained subscriptions to hold the status for given processes. Bit sledgehammer to crack a nut perhaps, but it would work. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Adreno |
| Posted: Mon Jan 28, 2008 1:53 am Post subject: |
|
|
Acolyte
Joined: 05 Jul 2007
Posts: 71
|
Well . . . u ´re absolutly right . . and these kind of contradictory requirements are really drivin ´ me mad.
But our costumer wants the things in that way and so i´ll have to find a solution.
Is there any kind of trigger you could use for that, may be an expiry trigger that transferes messages to another local queue if the aren´t tranfered?
_________________
There are two kinds of people in this world: people who s***, and Chuck Norris... |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Jan 28, 2008 2:19 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Why not make the messages have an expiry time in the MQMD? |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jan 28, 2008 2:21 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Adreno wrote: |
Well . . . u ´re absolutly right . . and these kind of contradictory requirements are really drivin ´ me mad.
But our costumer wants the things in that way and so i´ll have to find a solution. |
Do a search in the forum on "trout". I find the method very helpful with customers that generate stupid requirements. 
| Adreno wrote: |
Is there any kind of trigger you could use for that, may be an expiry trigger that transferes messages to another local queue if the aren´t tranfered? |
Oddly, IBM haven't included a trigger that fires "after a period of time", most business events being concerned with moving messages as fast as possible and as soon as they're available. It also messes with WMQ's advertised model of assured delivery; once the message has been sent it should be transmitted not hived off and fiddeled with.
You could probably write a piece of custom code and build it into whatever monitors the channel that moves messages off the xmitq queue once the channel's been stopped for a while. Not sure what you'd do about the xmit headers though. Also how you'd determine which messages were "replaced" by more recent ones would be a interesting piece of logic I wish you luck with.
Basically what you've got here is an extreme and bizzare kind of message affinity, where messages are related to earlier or later ones. There are any number of posts on why this is a bad idea.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jan 28, 2008 2:25 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zpat wrote: |
| Why not make the messages have an expiry time in the MQMD? |
Well I would have suggested that, but the poster said he wanted all the messages sent eventually; he's just trying to mess with the delivery order. Expired messages will never be recieved.
Also there's no apparent guarantee that a given message will be superceeded within the expiry period by a new one. That's why I suggested retained publications.
But if he can persuade his customer that a status report not delivered after a given period of time is irrelevent and can be discarded without ever being reviewed then expiry will be the ideal option.
For persuasion I refer again to the trout.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Mon Jan 28, 2008 3:23 am Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
| Quote: |
| Is there any kind of trigger you could use for that, may be an expiry trigger that transferes messages to another local queue if the aren´t tranfered? |
yes, some kind of. although i do not like that solution very much, its still better then fiddling around with exits or something similiar.
set expiry time on messages
set "report with full data" on expiry
use a qremote on the local (sending) queuemanager as replytoq
use the local (sending) queuemanager as replytoqmgr
now when the channel starts after the outage, your expired messages will be discarded, and a report (with full data) is written to the queue you specified. in your case a remote queue that can be transfered (either by the same of a different channel) to the target queuemanager (eihter to the same or a different target queue), depending on how you set up the remotequeue on the sending queuemanager.
but the receiving end must be able to receive these report messages instead of what it expects (e.g. datagram or whatever)
it may be worth reviewing the number of messages that can pile up, if all this effort is for 3 minutes of execution time (till all old and new messages are processed) then it may not be worth all the discussion and the (complicated) setup.
if something can be done for days, why has it to be back up (and actual) in seconds?
_________________
Regards, Butcher |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jan 28, 2008 3:38 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Quote: |
| Well I would have suggested that, but the poster said he wanted all the messages sent eventually; he's just trying to mess with the delivery order. Expired messages will never be recieved. |
True but consider this. Add a report option on expiry with full message and replyto q replytoqmgr that points to a 2nd queue on the same remote qmgr.
Now after expiration all messages go to this queue for processing of irrelevant stati. All you need is a second instance of your application that would process the irrelevant stati for reviewing....
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Jan 28, 2008 10:25 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
This is one of those cases where you want the q processed backwards - newer messages first, older messages last. This Expiry Report trick is the best way I can think of to accomplish what's required since FILO queues are not an option.
Neat idea.
| Mr Butcher wrote: |
| now when the channel starts after the outage, your expired messages will be discarded, and a report (with full data) is written to the queue you specified. |
One minor correction. The expiry reports will be created within minutes of the messages expiring as long as the q is at least open for puts of new messages. In MQ 6.0 the built in expiry scavenger which runs every few minutes will do this for active queues.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jan 28, 2008 10:39 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
This is one of those cases where you need to review your definition of an 'event', and make sure it includes information about how current it is - perhaps a timestamp.
Then it doesn't matter what order the event messages are read in, the program consuming them can make decisions based on whether the event just read is 'current' or not... At a simple level, deciding if it's more than X minutes old means it's "not current".
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
