| Author |
Message
|
| mdfaraz |
 Posted: Wed Mar 31, 2010 6:46 am Post subject: Not getting messages in order when reconnecting |
 |
|
Novice
Joined: 31 Mar 2010
Posts: 19
Location: Houston, TX
|
Hi All,
I have a .NET program which reads from a queue and puts that message to 2 other queues. For every message I do 4 operations,
1. Read the message from Queue1 using options MQC.MQGMO_WAIT Or MQC.MQGMO_BROWSE_NEXT
2. Put the message on Queue2
3.Put the message on Queue3
4. A destructive read to destroy this message from Queue1 using options MQC.MQGMO_WAIT Or MQC.MQGMO_MSG_UNDER_CURSOR
I am trying to make this program reconnect if a network related error occurs. so whenever my 1,2,3 or 4 operations fail due to loss in connection, I wait until I get the connection back and then retry the same operation. But in my testing, I am seeing that I am getting few messages twice, especially after my reconnection. Still worse, I am also skipping few messages which is unacceptable for my business.
Any thoughts on how I can make sure I only read the message once and then destroy the message I just read and not skip or duplicate them? I thought MQ would make sure of that but probably there is something wrong in what I am doing. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Wed Mar 31, 2010 6:58 am Post subject: Re: Not getting messages in order when reconnecting |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mdfaraz wrote: |
| Any thoughts on how I can make sure I only read the message once and then destroy the message I just read and not skip or duplicate them? I thought MQ would make sure of that but probably there is something wrong in what I am doing. |
It will but you have to code for it. If all operations have to succeed or fail then make sure all of them are in the same Unit Of Work - look up SYNCPOINT options in the documentation.
The skipping messages is caused by your use of the browse cursor. This is never a good idea, and becomes redundant if you use a UOW. Don't browse for a message, get destructively and then put it to the 2 other queues. If the puts fail WMQ will roll the message back onto the original queue.
You should also incorporate code to ensure this doesn't happen multiple times (it might not be a simple connection error). Search for the term "poison message" in this forum and ensure your application gives up under appropriate circumstances.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sumit |
| Posted: Wed Mar 31, 2010 7:00 am Post subject: |
|
|
Partisan
Joined: 19 Jan 2006
Posts: 398
|
As you are using .NET program to get/put messages, it's the responsibility of the application to handle the problem that you are facing. You may use something like this-
1. Let all 4 steps work in sync.
2. Let the application roll back to last sync point status if it encounters problem in executing any of the steps.
3. Commit put and get only when all 4 steps execute successfuly.
_________________
Regards
Sumit |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Mar 31, 2010 7:35 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Don't use browse to read a queue in an application. Use a normal MQGET which will destructively read the message once only.
Browsing is for UTILITIES only (in general) and runs the risk of the same message being read more than once by the same or different programs.
Use MQGET with SYNCPOINT (and then MQCMIT or MQBACK) if you wish to defer the destruction of the message.
Last edited by zpat on Wed Mar 31, 2010 7:54 am; edited 1 time in total |
|
| Back to top |
|
|
| mvic |
| Posted: Wed Mar 31, 2010 7:43 am Post subject: Re: Not getting messages in order when reconnecting |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| mdfaraz wrote: |
| Any thoughts on how I can make sure I only read the message once and then destroy the message I just read and not skip or duplicate them? I thought MQ would make sure of that but probably there is something wrong in what I am doing. |
Is the application opening the queue with MQOO_INPUT_EXCLUSIVE ? This would be necessary to ensure ordering of messages.
Are you using MQGMO_SYNCPOINT, MQPMO_SYNCPOINT and MQCMIT ? This would be necessary to ensure transaction consistency.
"Duplicate" messages do not exist in MQ, so can you clarify what is meant there please? Do you mean you got a message inside a syncpoint, then failed before committing, MQ backed it out onto the queue, then when you run the app again you get the same message? If so, then this is not a duplicate message, it is the same message that the application simply failed to process, and therefore was not able to commit. MQ rolls it back to the queue when the application can be determined to have failed. |
|
| Back to top |
|
|
| mdfaraz |
| Posted: Wed Mar 31, 2010 7:52 am Post subject: |
|
|
Novice
Joined: 31 Mar 2010
Posts: 19
Location: Houston, TX
|
The application opens 'Queue1'(as I referred to) with options MQC.MQOO_FAIL_IF_QUIESCING Or MQC.MQOO_INPUT_SHARED Or MQC.MQOO_BROWSE Or MQC.MQOO_INQUIRE
and the other Queues to which I put are accessed with options MQC.MQOO_OUTPUT + MQC.MQOO_FAIL_IF_QUIESCING.
When I say duplicate messages, I mean that I browse a message, and when either of my two puts fail, I retry the puts and go ahead with destructive read, sometimes I destroy a different message(which was not I got with Browse) and then this 'undestroyed' message somehow comes back again in my future browses. I haven't looked at SYNCPOINTS yet. I am trying to read through the documentation and see if I can make appropriate code changes.
Thanks all for your interest and I would certainly need more suggestions. |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Mar 31, 2010 7:56 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
PLEASE DON'T BROWSE MESSAGES IN APPLICATIONS.
Don't try to fix the browse problem - use MQGET properly. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Mar 31, 2010 9:32 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mdfaraz wrote: |
| When I say duplicate messages, I mean that I browse a message, and when either of my two puts fail, I retry the puts and go ahead with destructive read, sometimes I destroy a different message(which was not I got with Browse) and then this 'undestroyed' message somehow comes back again in my future browses. |
This is because you're not using the browse cursor properly. Your code is making assumptions about it's state (especially when an intermediate put call has failed) & also how the queue manager handles it in various circumstances. I suspect this is because you've experience with database cursors and are trying to use it like one.
It's not like a database cursor, which is not a best practice technique anyway (result sets are far more efficient).
To echo my earlier comments and the comments of others - don't browse messages! It increases the complexity of your code to make it work, it's an order of magnitude more inefficient than reading a message and is utterly unnecessary.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
