| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| How can I ensure receiver has all messages before triggering |
« View previous topic :: View next topic » |
| Author |
Message
|
| solberj |
 Posted: Mon May 13, 2002 1:35 pm Post subject: How can I ensure receiver has all messages before triggering |
 |
|
Novice
Joined: 18 Mar 2002
Posts: 18
Location: Jim Solberg
|
To all:
I seem to remember a discussion thread a month or two ago that talked about simulating message grouping, and I wonder if my recollection is valid. What I am trying to accomplish is this: I want to send a group of messages to a remote queue manager, but I want a way of ensuring that the receiver has all of the messages before the receiving application begins taking the messages off of the queue.
The jist of the conversation was that if your receiving application does not support message grouping, you can accomplish the same by having the receiving queue trigger on message priority. The sending application would send all messages except the last one with a low priority, then bump the priority on the last one. This seems logical to me but I have a few questions:
1) Should I set the BATCHSZ attribute on the channel to 1??? It seems to me that if the batch size is greater than 1, that (depending on how big the BATCHSZ is and how fast the application is) the receiving application could potentially do an MQGET, and find nothing on the queue while messages are still in transit. Stated another way, if we have 2 batches of 100, and the BATCHSZ is set to 100, the application would be triggered when the 1st message of the second batch hits the queue. If the sender sends the last message (message 200) with a higher priority, it will be the 101st message to hit the receiver queue. TRUE???
2) Will this concept work? Are there other ways to accomplish what I am trying to do? Are there better ways to accomplish this???
Thanks in advance,
Jim.
_________________
Jim Solberg
Senior System Architect - Technology Services
Conseco Finance Corporation |
|
| Back to top |
|
 |
| bduncan |
| Posted: Mon May 13, 2002 3:43 pm Post subject: |
|
|
Padawan
Joined: 11 Apr 2001
Posts: 1554
Location: Silicon Valley
|
Jim,
Rather than using a message priority scheme, is it possible for you to use message groups? This solves all the problems because you can specify MQGMO_ALL_MSGS_AVAILABLE which means none of the messages in the group will even be MQGET'able by the receiving application until all messages in that group are on the destination queue.
If you can't use message grouping, I would recommend not using priorities for the same reasons you mentioned. You could get around the problem by doing a MQGET as a browse, and then checking the priority of the browsed message. If it is the higher priority, start processing, if it is the lower priority, just do another MQGET with browse.
Otherwise you can do what one of my former employers did, in the days before message grouping was available. They had two queues, one where the first message in a group would go, and the other queue would hold the n-1 other messages in the same group. The first message would contain the Message Ids of all the other messages in his group. So you'd do an MQGET on the first queue, and get the first message in a group. After parsing this message, you'd have the Message Ids of all the other messages in the group, and then you'd do repeated MQGETs on the second queue, matching against each Message Id in sequence. One trick I pointed out was that they could use this same system, but rather than packaging all the Message Ids in the content of the first message in the group, they could use the CorrelId field. Basically, you'd get some unique value (queue manager name + timestamp would usually do the trick) and then tack on another four digits, 0000 through 9999 for each message in the group. This way, after you got the first message in the group, you could calculate the CorrelId of the next message in the group by adding 1 to the CorrelId of the current message. Repeat this, and you could get all messages in the group in sequential order.
_________________
Brandon Duncan
IBM Certified MQSeries Specialist
MQSeries.net forum moderator |
|
| Back to top |
|
|
| solberj |
| Posted: Tue May 14, 2002 6:09 am Post subject: |
|
|
Novice
Joined: 18 Mar 2002
Posts: 18
Location: Jim Solberg
|
Brandon,
Thanks for the input...my receiving application is an off-the-shelf EAI application that has an 'MQSeries connector'. One of the main 'features' of this connector is that you don't have to do any MQSeries adapter coding.
I'm not going to pick on them too badly, since similar off-the-shelf applications handle MQ Version 5 functionality pretty poorly. In this case, the app can detect end of transmission in 1 of 3 ways: based on a message count, based on the queue being idle for n number of seconds, or based on a wall clock time.
Of the 3, number 2 is closest to my needs, but the source system is an AS/400 and sometimes the AS/400 journals roll over during processing of the batch (introducing latency into the process).
You would think that they would support end of transmission based on MsgFlags in the message header, BUT NO!!!
If I can accomplish this without indroducing additional logic into the adapter on the source system, I would prefer to do so...on the other hand, if I use a priority scheme and set the BATCHSZ to 1 my performance is going to be pretty ugly.
Ideas???
Thanks, Brandon. Jim.
_________________
Jim Solberg
Senior System Architect - Technology Services
Conseco Finance Corporation |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue May 14, 2002 8:15 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7716
|
If we can assure that all the message have indeed made it to the queue, does it matter at that point what order they may have arrived in? In other words you only care that all the messages have made it, and not neccesarily in what order?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| solberj |
| Posted: Tue May 14, 2002 10:08 am Post subject: |
|
|
Novice
Joined: 18 Mar 2002
Posts: 18
Location: Jim Solberg
|
Peter,
I am checking with the people that understand the details...but I am pretty sure that the order does NOT matter...
Regards, Jim.
_________________
Jim Solberg
Senior System Architect - Technology Services
Conseco Finance Corporation |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue May 14, 2002 11:33 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7716
|
If order is not important, how 'bout this? (Assume 100 messages is the total)
Sending app: Send message# 1 with unique CorrelID and priority 0. Use this CorrelID for messages 2 thru 100 (all priority 0). Send message 100 with the COA option. When you have recieved the COA report, you "know" that all 100 messages have arrived on the remote queue. Send a "Kicker" message. This Kicker message has a Priority of 9, a MessageID that matches this batche's CorrelID, and a CorrelID of MQCI-NEW_SESSION. Put all these message under syncpoint and commit once the Kicker has been put succesfully.
Recieving App: Is triggered on Priority 9 messages. It's first get when triggered is qualified to only return a message that has MQCI_NEW_SESSION in the CorrelID field. This is to insure it only gets the Kicker first. (an uncommitted Kicker could start the process and cause the app to get Message #! otherwise) From this Kicker's MSG-ID, it now knows what CorrelID to do the rest of it's gets on. The app reads until 2033 while specifying this CorrelID.
The nice thing about this model is that even if the first batch is being processed, a second or third batch can be in the process of being dumped into this queue without affecting the first.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| solberj |
| Posted: Tue May 14, 2002 12:25 pm Post subject: |
|
|
Novice
Joined: 18 Mar 2002
Posts: 18
Location: Jim Solberg
|
Peter:
Read/researched your post. I think that this will probably work just fine, plus it satisfies my requirement for keeping the adapter relatively simple!
If we find a different/better solution in the mean time, I'll pass it on...
Thanks!
Jim.
_________________
Jim Solberg
Senior System Architect - Technology Services
Conseco Finance Corporation |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
