| Author |
Message
|
| atheek |
 Posted: Wed Aug 13, 2008 9:26 pm Post subject: Message Loss after QMGR bounce |
 |
|
Partisan
Joined: 01 Jun 2006
Posts: 327
Location: Sydney
|
MQ Version : 6.0.2.3
Platform: Windows 2k3
We have a queue which receives persistence messages send from a remote system across a channel. We made change to qmgr to increase the number of primary log files and when we bounced the queue manager for the changes to take effect we could see that inflight messages (around 40) got lost.
Logging Type Used : Circular
Any idea on what could have went wrong here
TIA
-Atheek |
|
| Back to top |
|
 |
| exerk |
| Posted: Wed Aug 13, 2008 10:50 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
'inflight' in what way? How do you know it was 'around 40'? And are you absolutely certain that the 'lost' messages were persistent?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Wed Aug 13, 2008 11:05 pm Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
sure, you might not have properly set the properties...
verify and ensure all those first
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| atheek |
| Posted: Wed Aug 13, 2008 11:22 pm Post subject: |
|
|
Partisan
Joined: 01 Jun 2006
Posts: 327
Location: Sydney
|
app in production for last 1 year...of severity 1 ...business cant tolerate message loss.. have extensive monitoring in place to detect message loss......and come on every one knows the relevance of Persistence.
What I meant by "Inflight" are messages that the channel were putting to the queue prior to committing...
I am just wondering what caused this message loss...people here are questioning the reliablility of MQ against message loss. |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Aug 13, 2008 11:27 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Gaya3 wrote: |
| sure, you might not have properly set the properties... |
| atheek wrote: |
| We have a queue which receives persistence messages send from a remote system across a channel... |
What properties? Default persistence on the queue? Which will have no efect whatsoever.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Aug 13, 2008 11:29 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
MQ does not lose messages like this or the world economy would collapse (hang on....maybe..)
No, seriously - the messages must be set to persistent when put on the queue, the queue persistent attribute is just the default used if the application sets MQPMO_PERSISTENCE_AS_QDEF, it does not ensure the messages are persistent if they have arrived from another queue manager and were originally put as non-persistent.
I can't think of an occasion when MQ was being accused of "losing messages" that it was not in fact due to incorrect MQI programming and transactionally unsafe applications.
Developers need to understand ACID and use the MQI carefully.
Last edited by zpat on Wed Aug 13, 2008 11:33 pm; edited 1 time in total |
|
| Back to top |
|
|
| atheek |
| Posted: Wed Aug 13, 2008 11:31 pm Post subject: |
|
|
Partisan
Joined: 01 Jun 2006
Posts: 327
Location: Sydney
|
| Sender app sets the message to be persistent at the source side... |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Aug 13, 2008 11:34 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Do have a DLQ, is it big enough (both depth and maxmessagesize)? Does it contain the missing messages? |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Aug 13, 2008 11:35 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| atheek wrote: |
| ...and come on every one knows the relevance of Persistence. |
Yes, but just because the remote end says they are persistent does not mean they are; how many times have developers told me 'of course we're making the messages persistent' and then found that no, actually they weren't?
| atheek wrote: |
| What I meant by "Inflight" are messages that the channel were putting to the queue prior to committing... |
Then they weren't lost because they were never delivered to the queue, and would have remained on the remote end's XMITQ.
| atheek wrote: |
| I am just wondering what caused this message loss...people here are questioning the reliablility of MQ against message loss. |
If the bounce of the queue manager put the channel in-doubt, and the remote end resolved it by committing, then yes, it is possible that the messages were 'lost'.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| David.Partridge |
| Posted: Wed Aug 13, 2008 11:41 pm Post subject: |
|
|
Master
Joined: 28 Jun 2001
Posts: 249
|
Is the channel to this QM now running again at the sender end, or is it in retry? If the latter you probably have uncommitted messages pending in the xmit queue at the sending end.
If it is running did you have problems when you initially started the channel and exactly what steps were taken to recover (did you commit or rollback at the sender end)? If you committed and they weren't committed at the receiver end then you TOLD MQ to throw these messages away.
If in retry why? Are your sequence numbers out of sync?
MQ doesn't loose messages in my experience, though they often may be "hiding" somewhere (dead letter queues, transmit queues).
_________________
Cheers,
David C. Partridge |
|
| Back to top |
|
|
| kevinf2349 |
| Posted: Thu Aug 14, 2008 5:04 am Post subject: |
|
|
Grand Master
Joined: 28 Feb 2003
Posts: 1311
Location: USA
|
| Just a thought......do you have any sneaky automation that resets channel sequence numbers in an effort to make channels more 'recoverable'? |
|
| Back to top |
|
|
| sumit |
| Posted: Thu Aug 14, 2008 11:35 am Post subject: |
|
|
Partisan
Joined: 19 Jan 2006
Posts: 398
|
And what about the message expiry interval? Was that set to some value?
_________________
Regards
Sumit |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Aug 14, 2008 12:59 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| atheek wrote: |
What I meant by "Inflight" are messages that the channel were putting to the queue prior to committing...
|
SVRCONN channels? QM-to-QM channels?
I still don't understand how you came up with the number 40. How do you know there were 40 messages on the channel? It takes a channel less then a second to move 40 messages of average size.
IF the messages were persistent and IF the messages were committed by the application and IF the messages didn't expire and IF the receiving app didn't consume and IF someone didn't manually clear them, guess what - MQ did not lose them. I suspect one of the above is not true or the messages are in fact sitting in an XMITQ, in a DLQ or in an app's queue.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
