| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| How is the MQMD MsgId Calculated? |
« View previous topic :: View next topic » |
| Author |
Message
|
| mkza.swe |
 Posted: Fri Jul 13, 2012 5:51 am Post subject: How is the MQMD MsgId Calculated? |
 |
|
Novice
Joined: 04 Jun 2012
Posts: 16
|
I am using MQ and Message Broker.
We are looking to log some information about messages, and save some messages passing through the flows.
We have no good business identifier to sort messages on, and would want to make this solution more general if possible as well, so it can be reused.
The MQMD.MsgId would be great to use in filenames and IDs and some filters etc. However 24 Bytes is pretty long. Since we don't expect so much traffic, I'm thinking if it would be possible to truncate the MQMD and only save a part of it in the logs/files etc to save some space and make easier to read.
Files dont get so readable when you append something like this to them:
414d5120514d574d42564552312020207186ab4f232d7c19
From examining the MsgId byte sequence, it looks like it is like a normal counter, it goes up by one for each message (as far as I can see), meaning that just by keeping the last 8 hexadecimal characters I could identify 4 Billion different messages, which would be much longer than the lifetime of the messages we are saving anyway.
Do you see any problem with this? I can not find anyway where it actually specifies how the MsgId is built up in detail. It just sais that it's a uniqueue byte sequence. Is it a real counter or something else, where the message id's can not be predicted as easy as a counter, like if it was a hash or something? |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Fri Jul 13, 2012 6:02 am Post subject: Re: How is the MQMD MsgId Calculated? |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mkza.swe wrote: |
| We have no good business identifier to sort messages on, and would want to make this solution more general if possible as well, so it can be reused. |
What? There is no field in your application data that uniquely identifies one message (transaction) from another? Account number? Employee name? Department number? Device id? Data/Time? Please describe your data? What is the application?
If there is nothing in your app data on which to sort, why do you want to sort it?
Look at CorrelId in the MQMD. It is exactly the same size/shape as MsgId. The most common use of MsgId is for applications that use the request-reply model.
MQMDs are handled by qmgrs differently from your application message data in distributed queuing.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| stuartjohnston1967 |
| Posted: Fri Jul 13, 2012 6:24 am Post subject: |
|
|
Novice
Joined: 12 Jul 2010
Posts: 11
|
I agree with Bruce, use something else or use/change the correlID
I remember in the mist of time that the MessageID was explained and its format broken down (System clock in their somewhere)
Details would either be in the programming reference manual or the intercommunication manual maybe.
There are third party products which trace end-to-end message flows.
Stuart |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jul 13, 2012 6:40 am Post subject: Re: How is the MQMD MsgId Calculated? |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mkza.swe wrote: |
Do you see any problem with this?
|
Yes. Best-practice tells us to use the MQMD fields for their proscribed purpose.
| mkza.swe wrote: |
| I can not find anyway where it actually specifies how the MsgId is built up in detail. |
Your application use of this field is a choice. You can set the value of MsgId to any 24 Bytes of your choosing. The definition from the manuals (and InfoCenter) tells you that the MsgId generated will be unique (for this qmgr). Inevitably, this leads us to the question 'what if I have two qmgrs with the same name?'
Before you pour concrete around your misunderstanding and misuse of the MsgId field, I'd strongly suggest that you research (read) the APR and APG manuals (or equivalent InfoCenter) for supported uses of MsgId (and CorrelId).
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mkza.swe |
| Posted: Fri Jul 13, 2012 7:15 am Post subject: |
|
|
Novice
Joined: 04 Jun 2012
Posts: 16
|
We are using business ids, like order number etc, and timestamps to differentiate different messages sent regarding same order etc. The problem is that the sending application can not generate a timestamp for the message, or a "last updated" field or something like that, so it's difficult to differentiate between messages that regard the same order for instance.
So for timestamps, we are stuck with generating them in the MB, which is not as good. Like in cases of retries of messages (backout threshold), then the timestamp will be different between retries. Then using MsgId would be easier to see that it's the same message, since it is maintained during retries.
I'm sure it can be done in other ways too, but I was curious of how the MsgId would work and if it would be forgivable to "misuse" it a bit by truncating it (only in logs and filenames etc, wouldn't actually edit the MQMD field of the message).
I think that it might work technically, as messages are grouped by message flow for this solution, so it doesn't matter for this case if a truncated MsgId collides with a MsgId in another flow. Also all flows always have the same sender and receiver QMgr.
Thanks for all your replies. |
|
| Back to top |
|
|
| mkza.swe |
| Posted: Fri Jul 13, 2012 7:30 am Post subject: |
|
|
Novice
Joined: 04 Jun 2012
Posts: 16
|
I found what I was looking for, on how the MsgId is built up. I must have missed it the first time I browsed that page:
A MsgId generated by the queue manager consists of a 4-byte product identifier (AMQ␢ or CSQ␢ in either ASCII or EBCDIC, where ␢ represents a blank), followed by a product-specific implementation of a unique string. In WebSphere® MQ this contains the first 12 characters of the queue-manager name, and a value derived from the system clock. All queue managers that can intercommunicate must therefore have names that differ in the first 12 characters, in order to ensure that message identifiers are unique. The ability to generate a unique string also depends on the system clock not being changed backward. To eliminate the possibility of a message identifier generated by the queue manager duplicating one generated by the application, the application must avoid generating identifiers with initial characters in the range A through I in ASCII or EBCDIC (X'41' through X'49' and X'C1' through X'C9'). However, the application is not prevented from generating identifiers with initial characters in these ranges.
http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0m0/topic/com.ibm.mq.csqzak.doc/fr13190_.htm#fnsrc_1 |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jul 13, 2012 7:32 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Look at the context fields in the MQMD: put date and put time.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Jul 13, 2012 7:51 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| bruce2359 wrote: |
| Look at the context fields in the MQMD: put date and put time. |
 All this is a little bit complicated to just differenciate by timestamp when one is provided on the MQMD... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mkza.swe |
| Posted: Fri Jul 13, 2012 9:40 am Post subject: |
|
|
Novice
Joined: 04 Jun 2012
Posts: 16
|
| bruce2359 wrote: |
| Look at the context fields in the MQMD: put date and put time. |
Aye, thanks! Didn't think of that one at first  |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
