| Author |
Message
|
| sayleep |
 Posted: Fri Sep 21, 2012 12:55 am Post subject: Interpreting the output of dmpmqlog |
 |
|
Newbie
Joined: 21 Sep 2012
Posts: 8
|
Our client says that there has been a message loss apparently. An application say, application A, puts a message on a local queue through MQ client and the other application application B fetches the message.
Now the application team alleges that the message has been sent by appn A and the other appn B did not receive it.
We used dmpmqlog command, since we thought it might help in knowing the details of the message that were put in the queue. Can this approach help since we have the contents of the message?
In the output of the dmpmqlog, the message data is in hexadecimal form. Also there is no timestamp information in the logs. Can anyone tell us how to interpret the output of dmpmqlog?
LOG RECORD - LSN <0:13:17409:16581>
**********
HLG Header: lrecsize 940, version 1, rmid 0, eyecatcher HLRH
LogRecdType . . : AQM Put Message (257)
Eyecatcher . . : ALRH Version . . . . : 1
LogRecdLen . . : 920 LogRecdOwnr . . : 256 (AQM)
XTranid . . . . : TranType: NULL
QueueName . . . : HOSTCONV.INCOMING
Qid . . . . . . : {Hash 3872196839, Counter: 1}
ThisLSN . . . . : <0:0:0:0>
PrevLSN . . . . : <0:0:0:0>
Version . . . . : 4
MapIndex . . . : 3
PrevLink.Locn . : 2056 PrevLink.Length : 8
PrevDataLink . : {High 0, Low 3072}
Data.Locn . . . : 3072 Data.Length . . : 680
Data . . . . . :
00000: 41 51 52 48 04 00 00 00 FF FF FF FF FF FF FF FF AQRH....ÿÿÿÿÿÿÿÿ
00016: 00 00 00 00 00 00 00 00 03 00 00 00 02 00 C0 01 ................
00032: 00 00 00 00 00 00 01 00 E8 00 00 00 00 00 00 00 ........è.......
00048: 53 F1 02 00 41 4D 51 20 51 50 48 51 4D 47 52 20 Sñ..AMQ QPHQMGR
00064: 20 20 20 20 E9 E9 51 50 21 37 FD AF 00 00 00 00 ééQP!7ý¯....
00080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00096: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00112: 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 ................
00128: 00 00 00 00 FF FF FF FF 00 00 00 00 04 00 00 01 ....ÿÿÿÿ........
00144: 00 00 00 00 22 CA 04 80 D8 E4 1F B4 FF FF FF FF ...."..€.ä..ÿÿÿÿ
00160: 4D 44 20 20 01 00 00 00 00 00 00 00 08 00 00 00 MD ............
00176: 00 00 00 00 11 01 00 00 33 03 00 00 4D 51 53 54 ........3...MQST
00192: 52 20 20 20 00 00 00 00 01 00 00 00 20 20 20 20 R ........
00208: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
00224: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
00240: 20 20 20 20 20 20 20 20 20 20 20 20 51 50 48 51 QPHQ
00256: 4D 47 52 20 20 20 20 20 20 20 20 20 20 20 20 20 MGR
00272: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
00288: 20 20 20 20 20 20 20 20 20 20 20 20 72 74 67 73 rtgs
00304: 75 73 65 72 20 20 20 20 16 01 05 15 00 00 00 51 user .......Q
00320: DB 24 BD 77 86 F0 99 2E EF 15 D9 FC 03 00 00 00 .$.w†.™....ü....
00336: 00 00 00 00 00 00 00 0B 20 20 20 20 20 20 20 20 ........
00352: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
00368: 20 20 20 20 20 20 20 20 06 00 00 00 6C 69 73 72 ....lisr
00384: 76 72 00 00 00 00 00 00 00 00 00 00 00 00 00 00 vr..............
00400: 00 00 00 00 00 00 00 00 32 30 31 32 30 39 32 30 ........20120920
00416: 31 34 31 31 34 36 35 31 20 20 20 20 00 00 00 00 14114651 ....
00432: 00 00 00 00 00 00 00 00 00 00 00 00 FF FF 00 00 ............ÿÿ..
00448: 7B 41 3A 45 46 54 46 30 31 4F 32 39 38 4E 31 30 {A:EFTF01O298N10
00464: 55 42 49 4E 30 35 34 37 33 38 37 55 42 49 4E 30 UBIN0547387UBIN0
00480: 35 35 30 34 35 31 31 32 32 30 30 33 58 58 58 58 550451122003XXXX
00496: 58 58 58 58 58 58 58 58 58 58 58 58 32 45 46 54 XXXXXXXXXXXX2EFT
00512: 32 30 31 32 30 39 32 30 30 30 30 30 32 30 32 30 2012092000002020
00528: 37 37 34 31 33 30 58 58 58 58 58 58 58 58 58 55 774130XXXXXXXXXU
00544: 42 49 4E 4D 55 52 31 30 31 36 36 32 36 37 33 39 BINMUR1016626739
00560: 39 7D 7B 34 3A 0D 0A 3A 32 30 32 30 3A 55 42 49 9}{4:..:2020:UBI
00576: 4E 4D 55 52 31 30 31 36 36 32 36 37 33 0D 0A 3A NMUR101662673..:
00592: 32 30 32 30 3A 53 41 41 33 34 34 30 34 37 34 35 2020:SAA34404745
00608: 0D 0A 3A 35 35 31 38 3A 53 42 49 4E 30 30 30 30 ..:5518:SBIN0000
00624: 36 39 31 0D 0A 3A 32 30 30 36 3A 53 42 49 4E 48 691..:2006:SBINH
00640: 31 32 32 36 34 36 36 33 38 39 36 0D 0A 3A 33 35 12264663896..:35
00656: 30 31 3A 32 30 31 32 30 39 32 30 0D 0A 31 34 33 01:20120920..143
00672: 34 33 37 0D 0A 2D 7D 0D 437..-}.
This is one entry from the output. Please help us interpret this since, we are unable to do so.  |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Fri Sep 21, 2012 4:20 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
If the message your app team is looking for was persistent, AND your qmgr uses linear logs, then an image of the message will be found in one of the log segments - if you keep them.
How long ago was this message allegedly put to a queue? Do you retain log files that go back that far?
Ask the app team to tell you exactly what the data in the alleged message contained (a name, an address, something like that); then use a search/find utility to look digitally through the log segments.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Sep 21, 2012 5:17 am Post subject: Re: Interpreting the output of dmpmqlog |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sayleep wrote: |
| Now the application team alleges that the message has been sent by appn A and the other appn B did not receive it. |
How is the application team supporting this allegation? Can they prove the message was sent? This cry of "MQ's lost my message" is depressingly familiar to those of us who've been working with the software for many years (and are so very, very tired) & my experience is that in 80% of such cases the lost message turns up in the poor error handling of the putting application, i.e. the put failed and the application failed to detect it.
Another 5% of cases are catered for by the getting application being in error, but I'm assuming you've already looked for the message in the target queue. Remember to try the dead letter queues and backout queues as well.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Sep 21, 2012 5:31 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
 wholeheartedly.
If your search through the logs does NOT result in finding the 'lost' message, the app developers will use this as proof that WMQ did, in fact, lose the message.
Conversely, if your search DOES find the message, the app developers will use this as proof that WMQ lost the message.
Don't take this personally. We've all been through this. WMQ doesn't lose messages, but applications do - through bad coding.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Sep 21, 2012 6:10 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
This argument is not limited to MQ.
I've been around long enough to have heard developers complain about DB2 and Oracle losing rows of table data, Windows (and UNIX) losing files, CICS losing transactions. Back when dinosaurs roamed the earth, developers complained that we were losing their (Hollerith) cards.
To quote Bart Simpson: "You're damned if you do, and damned if you don't."
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Sep 21, 2012 6:24 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| Don't take this personally. |
even more wholeheartedly.
| bruce2359 wrote: |
| We've all been through this. WMQ doesn't lose messages, but applications do - through bad coding. |
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Sep 21, 2012 6:35 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
In any case, lead the search for the allegedly missing message.
You will be seen as a leader and cooperative - two attributes not usually associated with propeller-heads.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| sayleep |
| Posted: Sat Sep 22, 2012 10:18 pm Post subject: |
|
|
Newbie
Joined: 21 Sep 2012
Posts: 8
|
| bruce2359 wrote: |
If the message your app team is looking for was persistent, AND your qmgr uses linear logs, then an image of the message will be found in one of the log segments - if you keep them.
|
The messages were persistent, but unfortunately, the qmgr uses circular logging.
| bruce2359 wrote: |
How long ago was this message allegedly put to a queue? Do you retain log files that go back that far?
Ask the app team to tell you exactly what the data in the alleged message contained (a name, an address, something like that); then use a search/find utility to look digitally through the log segments. |
One message was put on 5th Sept. 2012 and two other messages were put on 14th Sept. 2012.
The application team provided the message content details like Transaction Ref No. (TRN), transaction amount, etc. The TRNs usually begin with the string 'SAA'. Now, I can see a several entries of transactions starting with SAA in the dmpmqlog output. But those do not match with the TRNs which were provided to us.
I doubt if those messages were put in the queue at all. What do you guys say? |
|
| Back to top |
|
|
| sayleep |
| Posted: Sat Sep 22, 2012 10:26 pm Post subject: Re: Interpreting the output of dmpmqlog |
|
|
Newbie
Joined: 21 Sep 2012
Posts: 8
|
| Vitor wrote: |
How is the application team supporting this allegation? Can they prove the message was sent?
|
There is nothing concrete from the application side to support this allegation. We asked if there are any application log where any MQPUTs can be seen for those "missing" messages. They say that they do not maintain any application logs as such. They just get acknowlegement codes once the message is transmitted to the queue.  |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Sep 23, 2012 3:49 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
At this point your circular logs have probably been overwritten...
Which means you cannot show the message in the logs.
So you have no proof you received it and you have no proof somebody removed it from the queue...
Damned it you do, damned if you don't...
You might want to check anyways if there may be application contention for messages on the queue. What happens if app a gets a message intended for app b?
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Sun Sep 23, 2012 7:14 am Post subject: Re: Interpreting the output of dmpmqlog |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sayleep wrote: |
| They just get acknowlegement codes once the message is transmitted to the queue. |
Suggest that this code be augmented with the reason code from the MQPUT (typically zero) rather than just an acknowledgement that their code attempted the put.
I find this works for me:
"I'm the postal service. I take messages out of the box, and deliver them to the destination box. If the other party doesn't take them out of the destination box that's not my business any more that the postal service cares that Uncle Bubba got his card late because he was too drunk to leave the house for two days. Likewise if you want the postal service to investigate why his card never showed, the first thing they want is prove is that you really posted it & it's not in the pocket of your car. So if you want me to find out what happened to your message, prove to me you posted it."
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Sep 23, 2012 8:33 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
And check for the strange behavior where you don't receive the 2085 or 2087 anymore with one of the higher V7 versions? I was discussed over the last 2 weeks on the board.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| sayleep |
| Posted: Sun Sep 23, 2012 8:55 pm Post subject: |
|
|
Newbie
Joined: 21 Sep 2012
Posts: 8
|
| fjb_saper wrote: |
And check for the strange behavior where you don't receive the 2085 or 2087 anymore with one of the higher V7 versions? I was discussed over the last 2 weeks on the board.
|
Can you post the URL of this thread? |
|
| Back to top |
|
|
|
|
