| Author |
Message
|
| CoreySanders |
 Posted: Wed Feb 20, 2008 4:14 pm Post subject: Sequencing of messages within a unit of work |
 |
|
Novice
Joined: 20 Feb 2008
Posts: 11
|
I hope someone can help me with the following problem.
We have an application that sends messages to two different queues using the following sequence of events.
Open session
Open queue w/ SYNCPOINT (data)
Open queue w/ SYNCPOINT (trigger)
Put message to data queue
Put message to trigger queue
Commit
Close queue (data)
Close queue (trigger)
Close Session
What we hope will happen is that the data message will always commit and be available before the trigger message becomes available. What is actually happening, however, is that sometimes the trigger message will become available before the data message. This causes errors in our application.
Is there anyway to guarantee that the data message will always be available before the trigger message?
Java application using MQ Base APIs against MQ Version 5.3 Fixpack 4 on Linux.
Thanks! |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Wed Feb 20, 2008 4:53 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You appear to be trying to get the behavior of two transactions, but only using one transaction.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| CoreySanders |
| Posted: Wed Feb 20, 2008 8:26 pm Post subject: |
|
|
Novice
Joined: 20 Feb 2008
Posts: 11
|
Do you have any input as to why that might be? Is there something special I need to do to make operations on both queues participate in a single transaction? Are there any debugging methods I can use to tell if the messages are being put in separate transactions?
Thanks for your help. |
|
| Back to top |
|
|
| jsware |
| Posted: Thu Feb 21, 2008 12:12 am Post subject: Re: Sequencing of messages within a unit of work |
|
|
Chevalier
Joined: 17 May 2001
Posts: 455
|
| CoreySanders wrote: |
| What we hope will happen is that the data message will always commit and be available before the trigger message becomes available. What is actually happening, however, is that sometimes the trigger message will become available before the data message. This causes errors in our application. |
Both messages become available at the same time (when you commit) and not in the order in which you put them. I am assuming that you have an MQGET waiting on both queues, and its a scheduling thing with the operating system which means that the trigger message is being handled first.
What I think your are trying to do might be possible with a single queue if you follow a long list of constraints to keep them from switching. These constraints affect the scaleability of your application however (e.g. like only ever having a single MQGET).
Alternatively you can use grouping and sequencing to assure an order of delivery of a group of related messages.
_________________
Regards
John
The pain of low quaility far outlasts the joy of low price. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Feb 21, 2008 12:54 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Why not use proper MQ triggering, so that when the data message arrives MQ generates a trigger message (which can be processed by your trigger monitor application)? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Feb 21, 2008 2:34 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Let me try again.
You have written code that uses one transaction.
You seem to want that code to act like two transactions.
It won't.
Either create two transactions, and commit them in the order you want them to appear (and accept the fact that your reading app may see a delay between when the trigger message is available and when the data messages are available).
Or just use regular MQ triggering!
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 21, 2008 2:40 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zpat wrote: |
| Why not use proper MQ triggering, so that when the data message arrives MQ generates a trigger message (which can be processed by your trigger monitor application)? |
| jefflowrey wrote: |
| just use regular MQ triggering! |
It's a very nice wheel you're trying to invent there, but is it worth the effort?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| CoreySanders |
| Posted: Thu Feb 21, 2008 6:31 am Post subject: |
|
|
Novice
Joined: 20 Feb 2008
Posts: 11
|
| I don't know anything about MQ builtin triggering. I will look into it. I am guessing that it won't work for my purposes though. The content of my trigger message contains application metadata describing the data message. If the queue manager is generating the trigger message, I don't know how I am going to get my metadata into the trigger message. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 21, 2008 6:48 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| CoreySanders wrote: |
| The content of my trigger message contains application metadata describing the data message. |
You have message affinity (one message dependant on another) and should remove it. There are endless discussions on the board about why this is a bad idea, you've just invented another reason.
If the data message and the trigger message (in your original scenario) are being put by the same application in the same UOW, why are there 2 messages?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| CoreySanders |
| Posted: Thu Feb 21, 2008 6:51 am Post subject: |
|
|
Novice
Joined: 20 Feb 2008
Posts: 11
|
| The application needs to support large messaging. The application communicates with a mainframe version of MQ that does not support automatic message segmentation. So, we do application level data segmentation when the message is larger than our configured maximum message size. The trigger message tells the receiving application the group id of the group of data messages and also provides other application-level meta data. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Feb 21, 2008 6:51 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Even if they really *do* need to be two messages, why are they two queues?
If I had this set of unusual requirements, I'd use a message group and put "the trigger message" as the "the first message in the group".
Then I know I'd always get it first, and could use one transaction.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| CoreySanders |
| Posted: Thu Feb 21, 2008 7:00 am Post subject: |
|
|
Novice
Joined: 20 Feb 2008
Posts: 11
|
| jefflowrey wrote: |
Even if they really *do* need to be two messages, why are they two queues?
If I had this set of unusual requirements, I'd use a message group and put "the trigger message" as the "the first message in the group".
Then I know I'd always get it first, and could use one transaction. |
This is a really good idea that never came up in the application design. I'll definitely add this to our list of desired changes for the next release. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Feb 21, 2008 3:48 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| CoreySanders wrote: |
| The application needs to support large messaging. The application communicates with a mainframe version of MQ that does not support automatic message segmentation. So, we do application level data segmentation when the message is larger than our configured maximum message size. The trigger message tells the receiving application the group id of the group of data messages and also provides other application-level meta data. |
This is a common application design I have seen for mainframes.
The messages are usually correlated by correlation id: the message on the first queue - correl id carries the correlation id of the messages in the Data queue. All logical messages in the DATA queue carry the same correlation id.
One of the usual ways this works is to trigger on the triggered queue.
CICS is getting the message from the triggered queue and the DATA queue is passed with the trigger parameters...
Make sure to do a GET with WAIT for a reasonable period of time. This should allow you for the 'tiny' interval during which you seem to be at risk... You remove the message from the triggered queue and get the messages by correlid from the data queue.
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Feb 21, 2008 11:28 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Upgrade the mainframe MQ, version 531 is out of support very soon anyway.
The upgrade is very easy and much better than trying to program around the message size limitation with your own style of grouping. |
|
| Back to top |
|
|
|
|
