| Author |
Message
|
| DrajMQ |
 Posted: Sat Mar 29, 2014 3:03 am Post subject: DLQ reason 134742016 |
 |
|
Newbie
Joined: 29 Mar 2014
Posts: 7
|
| Hi friends, can anybody tell me what happens exactly while the message gets en-queued or de-queued from a local queue. Is there any kind of buffering while this procedure takes place? I have a 2 GB file to be transferred over the queues, where messages can be of max 4 MB. While the messages are put to queue (they are segmented) some of the messages go to DLQ with reason 134742016. I searched for this reason on google but couldn't find anything. Any help would be appreciated. Thank you |
|
| Back to top |
|
 |
| PaulClarke |
| Posted: Sat Mar 29, 2014 4:54 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
| Quote: |
| Is there any kind of buffering while this procedure takes place? |
It depends what you mean exactly. If you are asking "is it possible to put messages to a queue faster than they are being processed?" then the answer is yes. An application, such as the one you describe ought to have retry processing to cope with this situation.
As for what does 134742016 mean I would have thought this was a question for the Developers of the putting application. What is the putting application ?
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| DrajMQ |
| Posted: Sat Mar 29, 2014 10:17 am Post subject: |
|
|
Newbie
Joined: 29 Mar 2014
Posts: 7
|
Thanks for the reply Paul.
Sorry for not framing my question in a proper format.
Below is the scenario in which I am getting the DLQ reason 134742016:
I have one application on the source side which picks a 2+ GB file from a directory - actually 5 files of max 500 MB separately - each file is picked separately and place on the remote queue (where it is segmented with 4MB max msg size) which points to a local queue on the destination side, where a similar application will pick the messages and place the file in a directory.
I am getting first 3 parts of the files successfully on the destination side, but for the 4th file I can see that some of the messages go to DLQ instead of the required local queue with reason 134742016. I don't think this number has to do anything with my application (I am not sure though, your advice will be very helpful). The 4th file is of 500 MB, so I should get around 130 messages of 4 MB each. I can see that I have messages with seq number from 1 to 122 in my required local queue and the messages with seq number from 123 to 129 go to DLQ all with reason 134742016. The last message which has less size (almost 300 KB) fits in the required local queue. And adding to my surprise I am getting the 5th file successfully at the destination. So the 4th file sits in queue, as the application won't pick the messages as some of its messages are in DLQ.
Initially I though this might be a space issue for the file system, but when checked there is more than enough space on the file system to acquire the file.
I am looking forward for your thoughts in this scenario (I hope I am clear now.  ) |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Sat Mar 29, 2014 11:01 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Are you sure you are reading the DLQ reason code correctly ? - use something like MO71 to look at it.
134742016 is 0x0808 0000 which, if we byte swap it is....
0x0000 0808 which in decimal is 2056 which is....
MQRC_Q_SPACE_NOT_AVAILABLE
which does sound very likely.
I suspect your application received MQRC_Q_SPACE_NOT_AVAILABLE from MQ and, not unreasonably, put the message to the DLQ. One might expect your application to do a little retry processing first but ultimately it is reasonable that it put it to the DLQ. What you should probably have, for this and other reasons, is a Dead Letter Queue handler which will periodically retry the puts for you.
Of course this just educated guesswork and the truth may be much stranger.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| DrajMQ |
| Posted: Sun Mar 30, 2014 2:17 am Post subject: |
|
|
Newbie
Joined: 29 Mar 2014
Posts: 7
|
Thanks a lot Paul. That had surely put some light in our investigation.
Going further I have two questions for this scenario as per my next findings:
1. Why is that the Reason in DLQ is mentioned as 134742016 instead of mentioning the true reason 2056 (MQRC_Q_SPACE_NOT_AVAILABLE).
2. I checked the file system space for /var/mqm/ . It has 4 GB of total space.
While putting the messages to the required local queue, the queue file size goes upto 1.95 GB. But even after the messages from the queue are de-queued completely, the queue file size remains 1.95 GB for some duration (almost for the next half hour) even when there are no messages in this queue. Why is this so?
_________________
Thank,
Dhiraj Khot |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Sun Mar 30, 2014 2:33 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Hi DrajMQ,
Well the answers are:
1. The reason code in the DLQ header it is just a number (ie. an MQLONG). How it is reported depends purely on the application you are using to display the DLQ. I don't think you've mentioned what that is so I can't comment. If you use MO71 I think you'll find that it does report the reason code as something like MQRC_Q_SPACE_NOT_AVAILABLE. In fact version 7.5.2 will display the DLQ reason from the queue browse window rather than having to display each message.
2. Generally speaking it would be highly inefficient if the queue file constantly grew and shrank as each message was put and got from the queue. So, the Queue Manager adds a level of hysteresis. Take a look at http://www-01.ibm.com/support/docview.wss?uid=swg21110841 for a description of some of the behaviours. I don't doubt there are other pages you can find this is just the first one that came up when I did a Google search.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Mar 30, 2014 4:34 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
|
| Back to top |
|
|
| tczielke |
| Posted: Sun Mar 30, 2014 5:49 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
"1. Why is that the Reason in DLQ is mentioned as 134742016 instead of mentioning the true reason 2056 (MQRC_Q_SPACE_NOT_AVAILABLE)."
If you are reading this value from a raw binary dump of the message like from amqsbcg or a trace and your message was generated from a computer using the x86 processor, then this Reason will appear in little Endian format (lowest significant byte to highest significant byte, starting at the beginning address of the 4-byte binary word). Almost every other processor in the world will store it in big Endian format (highest significant byte to lowest significant byte, starting at the beginning address of the 4-byte binary word). But the x86 processor is somewhat unique in this way.
For example, if the x86 processor is told to store the hex value x'12345678' at address x'001000A0', it will store it in the following fashion (least significant byte to most significant byte):
x'001000A0' = x'78'
x'001000A1' = x'56'
x'001000A2' = x'34'
x'001000A3' = x'12'
If the SPARC processor (which is a big endian processor) is told to store the hex value x'12345678' at address x'001000A0', it will store it in the following fashion (most significant byte to least significant byte):
x'001000A0' = x'12'
x'001000A1' = x'34'
x'001000A2' = x'56'
x'001000A3' = x'78
One consequence of this x86 little endian design decision is that mutli-byte binary values like the 4 byte Reason in a DLQ intuitively appear backwards to the casual viewer who is not aware of this anomaly and view the data in a raw binary dump. It sounds like you may be reading the raw binary display of the DLQ reason code that was generated on an x86 processor, and you need to reverse the byte ordering to correctly convert it to its decimal value, like Paul mentioned. Using a tool that properly formats it, like Paul mentioned, would be a good idea, as well. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sun Mar 30, 2014 6:38 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9470
Location: US: west coast, almost. Otherwise, enroute.
|
| PaulClarke wrote: |
| So, the Queue Manager adds a level of hysteresis. |
A word seldom used in casual conversation. I do enjoy being sent to the dictionary. http://en.wikipedia.org/wiki/Hysteresis
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Sun Mar 30, 2014 10:40 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
That was a nice explanation of Endianess. I brought back a lot of memories. I spent many years working on Little Endian CPU's, namely PDP-11 and VAX at machine code level I always found Little Endian more logical than Big Endian.
We mostly read left to right. but numbers have their Most significant bits on the left, the LSB is on the right. Little endian bit patterns reflect this especially when loading values into the system through the CPU Front panel. When pushed I can remember the PDP-11/70 front panel bootstrap codes. Perhaps I should retire now?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| tczielke |
| Posted: Sun Mar 30, 2014 4:10 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
Hi smdavies99,
Thanks for the note. One other note for DrajMQ and others to help with reading MQ data structures and handling endianness. The MQ data structures that I have seen like DLQ, MD, etc. start with a 4-byte StrucId that has an eye-catcher like "DLQ ", "MD ", etc. followed by a 4-byte Version field. This Version field is an MQLONG and would be in the range like 1-10. If DrajMQ goes back and looks at the Version field of the DLQ structure, the value would probably be x'01000000'. Since that number is way to large to be a Version number, it is a good clue that the DLQ data structure if formatted in little endian and 4-byte binary numbers need to be "reversed" to be read correctly. For example, the Version id x'01000000' should be reversed to x'00000001', which would be 1, the only valid Version number for a DLQ structure. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 31, 2014 4:59 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| PaulClarke wrote: |
| So, the Queue Manager adds a level of hysteresis. |
A word seldom used in casual conversation. I do enjoy being sent to the dictionary. http://en.wikipedia.org/wiki/Hysteresis |
This is now my Word Of The Day
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Mar 31, 2014 6:09 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
I like this comment from wikipedia best
| Quote: |
| Not to be confused with Hysteria |
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 31, 2014 7:01 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zpat wrote: |
I like this comment from wikipedia best
| Quote: |
| Not to be confused with Hysteria |
|
That's my Word Of Every Other Day
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| DrajMQ |
| Posted: Mon Mar 31, 2014 7:45 pm Post subject: |
|
|
Newbie
Joined: 29 Mar 2014
Posts: 7
|
Thanks a lot guys for such a detailed explanation.
Special thanks to Paul. You had been a great help for this. I went through the document which you referred for the 2nd point. After reading it I was wondering whether the following case is valid/invalid:
1. If at present my Q file size is 1.5 GB but no messages in the queue, and if I put some messages which accounts to be more than 500 MB, lets say 800 MB, then the Q file would increase gradually upto 2 GB and then throw the Q_SPACE_NOT_AVAILABLE error. (Though the local queue doesn't have data more than 500 MB)
OR
2. If at present my Q file size is 1.5 GB but no messages in the queue, and if I put some messages which accounts to be more than 500 MB, lets say 800 MB, then the Q file would be over-written by some of the data resulting in no Q_SPACE_NOT_AVAILABLE error?
Thanks,
Dhiraj Khot |
|
| Back to top |
|
|
|
|
