| Author |
Message
|
| jamesyu |
 Posted: Tue Jan 19, 2010 1:28 pm Post subject: MQMD Expiry - why it is treated as approximate? |
 |
|
Acolyte
Joined: 31 Jan 2007
Posts: 70
|
Hi All,
Below is the MQMD Expiry definition from the MQ spec. One thing I don't understand that, it says "the expiration time is treated as approximate". What does it mena? Does it mean the message can be expired less than the value specified in Expiry in MQMD? Or it could be later than the value specified in Expiry?
Why is it "approximate"?
-----------------------------------------------------------------------------
This is a period of time expressed in tenths of a second, set by the application that puts the message. The message becomes eligible to be discarded if it has not been removed from the destination queue before this period of time elapses.
The value is decremented to reflect the time that the message spends on the destination queue, and also on any intermediate transmission queues if the put is to a remote queue. It can also be decremented by message channel agents to reflect transmission times, if these are significant. Likewise, an application forwarding this message to another queue might decrement the value if necessary, if it has retained the message for a significant time. However, the expiration time is treated as approximate, and the value need not be decremented to reflect small time intervals.
When the message is retrieved by an application using the MQGET call, the Expiry field represents the amount of the original expiry time that still remains.
--------------------------------------------------------------------------------
Thanks! |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Jan 19, 2010 1:58 pm Post subject: Re: MQMD Expiry - why it is treated as approximate? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jamesyu wrote: |
Why is it "approximate"?
|
Because as it says, intermediate steps might change the value. You can be certain that once the time expires, and not before, the message becomes ineligible for get opererations.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mvic |
| Posted: Tue Jan 19, 2010 2:02 pm Post subject: Re: MQMD Expiry - why it is treated as approximate? |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| jamesyu wrote: |
| Why is it "approximate"? |
Well, all timings on computers are approximate.
Consider a program call to sleep(x seconds). It will return control to your code x seconds after you call it. Let's say that's precisely x seconds. Not approximately x, but precisely x. Control is now back with your code after x seconds. Then precisely a little time later your next line of code can run. (Processors take time to execute each instruction).
That's a total of (x + a little bit) before your next line of code can run.
x was only an approximate length of time if what you really cared about was when your next line of code ran.
Calculation of a message's age is approximate for similar reasons.
In order to calculate whether a message is expired, its expiry time needs to be compared against MQ's record of how long it has had that message. Some useful examples of the details of this were in the quoted section of the manual.
Because there are approximations involved in calculating the elapsed time (especially if the message moves throughout a network), the calculation of age cannot ever be precise. It is therefore approximate.
Hope this helps. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jan 19, 2010 2:21 pm Post subject: Re: MQMD Expiry - why it is treated as approximate? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mvic wrote: |
| jamesyu wrote: |
| Why is it "approximate"? |
Well, all timings on computers are approximate.
Consider a program call to sleep(x seconds). It will return control to your code x seconds after you call it. Let's say that's precisely x seconds. Not approximately x, but precisely x. Control is now back with your code after x seconds. Then precisely a little time later your next line of code can run. (Processors take time to execute each instruction).
That's a total of (x + a little bit) before your next line of code can run.
x was only an approximate length of time if what you really cared about was when your next line of code ran.
Calculation of a message's age is approximate for similar reasons.
In order to calculate whether a message is expired, its expiry time needs to be compared against MQ's record of how long it has had that message. Some useful examples of the details of this were in the quoted section of the manual.
Because there are approximations involved in calculating the elapsed time (especially if the message moves throughout a network), the calculation of age cannot ever be precise. It is therefore approximate.
Hope this helps. |
Gosh I wish I'd thought of this explaination. It's so much clearer and better than mine
 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Jan 19, 2010 2:24 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
I agree with mvic.
And here's another prespective. They mention the word "approximate" when they are telling you not to worry about decrementing it for small values.
If you get a messages with 10 seconds of expiry, and it takes you 51 milliseconds to process the message, does it really matter if you agonize on whether to drop expiry for the related outgoing messages from 100 to 99? Do you round up and drop a tenth of a second off? What if it took you 49 milliseconds? Now do you round down and not take off a tenth? Because Expiry is expressed in such huge increments, tenths of a second, as soon as you pass thru a process that takes more or less than exactly 100 milliseconds you are already off. You are now approximate. And on todays systems even milliseconds are a rather large lenth of time.
I would bet that if Expiry was being created and designed today, it would be expressed in milliseconds, like the newer JMS Time To Live parm which is milliseconds itself.
We have seen this with WMB. We sometimes see the Expiry value of the outgoing messages greater than what the incoming message was, because the Broker rounded up the value! We confirmed this with a PMR and its discussed on mqseries.net in a thread a couple of years ago.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jamesyu |
| Posted: Tue Jan 19, 2010 2:39 pm Post subject: |
|
|
Acolyte
Joined: 31 Jan 2007
Posts: 70
|
| PeterPotkay wrote: |
I agree with mvic.
We have seen this with WMB. We sometimes see the Expiry value of the outgoing messages greater than what the incoming message was, because the Broker rounded up the value! We confirmed this with a PMR and its discussed on mqseries.net in a thread a couple of years ago. |
This is exactly whht I am doing now in a message flow to copy the Expiry from the original request message to the response message before sending it out to the sender (request and reply). The sender wants to have toe same Expiry specified in the request message back in the response message....
Has it been fixed in WMB 6.1? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Jan 19, 2010 3:37 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Approximate vs absolute.
"Approximate" contemplates a message in transit from an xmit queue, across a network, and into a destination queue. There will be a time when a message exists in both xmit queue and destination queue until both channel ends commit or backout messages.
| Quote: |
| This is exactly what I am doing now in a message flow to copy the Expiry from the original request message to the response message before sending it out to the sender (request and reply). The sender wants to have toe same Expiry specified in the request message back in the response message.... |
Reread the portion of the manual you quoted regarding when Expiry in the MQMD will be decremented. By the time your consuming application gets the message, Expiry will be LESS than when the requesting application created the message - by however many 10ths of seconds it sat in a queue.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mvic |
| Posted: Tue Jan 19, 2010 4:11 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| PeterPotkay wrote: |
| Because Expiry is expressed in such huge increments, tenths of a second, as soon as you pass thru a process that takes more or less than exactly 100 milliseconds you are already off. |
I don't quite buy the idea that a 10th of a second is huge, in this context.
When would someone care about tenths? (Not a rhetorical question!) 
For example when would someone declare a message as expirable after 0.1 second? Or 10.2 seconds?
| Quote: |
| You are now approximate. And on todays systems even milliseconds are a rather large lenth of time. |
Well, yes. But you are always approximate.
About milliseconds... if (say) you were able to specify message lifetime in milliseconds and cared enough about the precision to set (say) 10.345 milliseonds, does it matter to you if it expires after 10.346 ?
So maybe I just don't get it. I don't get why people would worry over tenths, when it comes to setting a message lifetime. I also don't get (x100 !) why people would worry over thousandths.
Someone educate me  |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Jan 19, 2010 4:23 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
I suppose the folks at the MQ factory pondered how to differentiate message expiry values. I don't have a strong feeling one way or the other about 10ths or 100ths or 1000ths, as they are just increments. It's one of those choice things.
I'd say 10 seconds is too long (maybe) for an ATM transfer message to sit in a request queue. Or perhaps 30 seconds is too long. Certainly 2 minutes is too long...
In any case, the OP wants to have the reply msg to have the exact same Expiry value from the original request message. Not possible. The original Expiry value is not kept in the MQMD - only the time remaining.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Jan 19, 2010 5:24 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| mvic wrote: |
| PeterPotkay wrote: |
| Because Expiry is expressed in such huge increments, tenths of a second, as soon as you pass thru a process that takes more or less than exactly 100 milliseconds you are already off. |
I don't quite buy the idea that a 10th of a second is huge, in this context.
|
Yup, its huge in computing time, and the Olympics, but more than enough for basing decisions on as it relates to when is a message no longer valid.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Jan 19, 2010 6:11 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| jamesyu wrote: |
| PeterPotkay wrote: |
I agree with mvic.
We have seen this with WMB. We sometimes see the Expiry value of the outgoing messages greater than what the incoming message was, because the Broker rounded up the value! We confirmed this with a PMR and its discussed on mqseries.net in a thread a couple of years ago. |
This is exactly whht I am doing now in a message flow to copy the Expiry from the original request message to the response message before sending it out to the sender (request and reply). The sender wants to have toe same Expiry specified in the request message back in the response message....
Has it been fixed in WMB 6.1? |
There is nothing to fix. Its been a couple of years, but as I remember it...
The Expiry Time used in the flow is an absolute time calculated from the incoming message header's Expiry Interval compared to the current time. This absolute time is stored in the message (not an MQ message at this point mind you). When the message leaves the MQOuput node, the calculation is made again to determine what MQMD.Expiry should be - by comparing that saved absolute time and the current time.
The Expiry time in the MQMD is in tenths of a second, and the absolute time and current time are much more precise. There needs to be rounding as the code translates. And the calculation is done twice. 2 cases of rounding allows for 2 instances of the wrong decision to be made. The wrong decision being rounding down when dealing with a message with very small expiry. You would not want the broker to round down and expire a message prematurely. So IBM played it safe and rounds up. In some cases for very fast flows this causes the Expiry in the MQMD of the outgoing message to be a tenth of a second bigger than the incoming message.
Occasionally having a message with a extra tenth of a second is probably never a problem. Having a message occasionally expire too early, even if its only less than a 1/10 of a second too early, while still not a big deal, is less desireable.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Wed Jan 20, 2010 4:20 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| bruce2359 wrote: |
...I'd say 10 seconds is too long (maybe) for an ATM transfer message to sit in a request queue. Or perhaps 30 seconds is too long. Certainly 2 minutes is too long...
|
By ATM, do you mean a monetary transfer transaction? If yes, these are "update" MQ messages and they should never use Expiry, and they should be Persistent.
Expiry should only be used on "inquiry" or "information" MQ messages that can be lost (expired) without any impact to the business.
_________________
Glenn |
|
| Back to top |
|
|
|
|
