| Author |
Message
|
| jefflowrey |
 Posted: Mon Oct 03, 2005 9:45 am Post subject: |
 |
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Can you be more specific, perhaps about what a "loaded queue" is versus a regular queue?
Does this mean that, if I attempt to put a message to a queue that has reached it's maxqdepth, but messages are expired on that queue, then the expired messages will be removed to make room for the new one?
I think Peter and I are both of the opinion that the only thing that removes expired messages in both v5 and v6 on distributed is a GET that would otherwise have retrieved the message. And to our reading, markt's comments contradict that. Specifically his comment about ExpiryInterval says to me that v6 on distributed will perform the same functions that z/OS does - where it will periodically purge expired messages without requiring a GET - but you can't use ExpiryInterval to TUNE this.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
 |
| wschutz |
| Posted: Mon Oct 03, 2005 11:04 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
| Quote: |
Specifically his comment about ExpiryInterval says to me that v6 on distributed will perform the same functions that z/OS does - where it will periodically purge expired messages without requiring a GET - but you can't use ExpiryInterval to TUNE this.
|
Interesting, if I put some messages on the queue with a short exiry (1 second), I see those messages a minute later in the curdepth. When I returned a hour later, they were gone. Soo....... looks like you interpertation is correct 
_________________
-wayne |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Mon Oct 03, 2005 11:26 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| jefflowrey wrote: |
| markt's comments contradict that. Specifically his comment about ExpiryInterval says to me that v6 on distributed will perform the same functions that z/OS does - where it will periodically purge expired messages without requiring a GET - but you can't use ExpiryInterval to TUNE this. |
it must use some value or interval somewhere, I can't believe anyone hardcoding parameters these days... 
interesting to see what comes out of this discussion and if something changed in V6 distributed, then where is it documented... 
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| hopsala |
| Posted: Mon Oct 03, 2005 1:28 pm Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| Michael Dag wrote: |
| it must use some value or interval somewhere, I can't believe anyone hardcoding parameters these days... |
I wish you were right, then I could change the queue buffer size for queues that serve large messages. 
And let's not forget checkpoints (every 10000 transactions), the recently-gotten message chain (512) and more... |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Oct 03, 2005 1:44 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Hey Now!!!! This time I waited before browsing the queue that had 100 expired messages, and at around the 4 minute mark (after they expired), the q depth dropped to zero! It is repeatable.
So, what is really going on behind the scenes in 6.0 with expired messages? To me this is a pretty substantional change to not have been spelled out in detail. A lot of times a q depth of expired messages was at least a clue something was wrong. Now if a message goes to a DLQ or exception q and expires and blinks away, we get paged, only to go find an empty q, and then curse the monitoring system for "false" alerts. Yeah it was annoying to browse a queue only to get a 2033, but at least we knew something was in there becasue we saw the depth.
Hmmmm, I think this will be a good thing in the end, but its gonna take some getting used to. (and some understanding in detail!)
-Peter
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqrules |
| Posted: Mon Oct 03, 2005 4:08 pm Post subject: |
|
|
Centurion
Joined: 01 Jun 2005
Posts: 100
Location: US
|
On z/OS you can set the ExpiryInterval attribute on the qmgr, which will check the queues at that interval for expired messages and get rid of them. This could be a good or bad thing...
mr |
|
| Back to top |
|
|
| Volodya |
| Posted: Mon Oct 03, 2005 11:17 pm Post subject: |
|
|
Novice
Joined: 04 Mar 2004
Posts: 22
Location: Moscow
|
I put 100, 1000 and 10000 messages (1Kb) on a queue with a short expiry on MQ 6.0 - Windows 2000 Server.
Behaviour seems the same as 5.3.
I put 100000 messages and expired messages were cleared in queue
when queue storage has exceeded 20Mb for the first time and 25Mb in subsequent times.
And it is correct !
This effect is harmful at greater flows of messages.
There is no big necessity to clean queue often.
It can affect on perfomance.
Threshold 20Mb quite reasonable in my opinion at work with greater flows messages.
_________________
Vladimir Makushkin, WebSphere MQ administrator |
|
| Back to top |
|
|
| markt |
| Posted: Tue Oct 04, 2005 3:50 am Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 508
|
a) it is new function in V6
b) the equivalent of ExpiryInt is not an exposed parameter.
c) the precise behaviour of expiry has NEVER been documented, at least apart from the z/OS ExpiryInt. The only guarantee is exactly the same as it's always been - that you will never be able to GET an expired message. The behaviour has changed several times internally, so any "rules" that people discovered - eg it required a destructive get or a browse, that might vary by platform and by release - were not part of the defined produc externals.
d) a loaded queue is (to a first approximation) a queue where someone has it open or has only recently closed it.
e) if a queue does not get loaded (because noone opens it) then any expired messages on the queue continue to sit on the queue. The queue manager doesn't want to go to the expense of setting up all the memory and the I/O for a queue that noone's actually using.
So even with the new internals you cannot be certain that a message will be preemptively removed when expired. It's just a lot more likely. |
|
| Back to top |
|
|
| hopsala |
| Posted: Tue Oct 04, 2005 6:08 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| markt wrote: |
d) a loaded queue is (to a first approximation) a queue where someone has it open or has only recently closed it.
e) if a queue does not get loaded (because noone opens it) then any expired messages on the queue continue to sit on the queue. The queue manager doesn't want to go to the expense of setting up all the memory and the I/O for a queue that noone's actually using.
So even with the new internals you cannot be certain that a message will be preemptively removed when expired. It's just a lot more likely. |
However, even though it is undocumented and thus risky (pending changes in other versions - undocumented implicitly means unsupported), all those that had written an "expiry message cleaner" that MQGETs (BROWSE or other methods) from the queue, can now just MQOPEN and immediately close, and that will make sure all expired msgs are taken care of. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Oct 04, 2005 9:57 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
And I assume MQOPEN just for MQINQ is enough, as that is all I had open on the queue (using the little WatchQ Support Pac) watching the depth refresh every second.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Tue Oct 04, 2005 10:52 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| markt wrote: |
| a) it is new function in V6 |
nice
| markt wrote: |
| b) the equivalent of ExpiryInt is not an exposed parameter. |
not so nice
| markt wrote: |
| c) the precise behaviour of expiry has NEVER been documented, at least apart from the z/OS ExpiryInt. The only guarantee is exactly the same as it's always been - that you will never be able to GET an expired message. The behaviour has changed several times internally, so any "rules" that people discovered - eg it required a destructive get or a browse, that might vary by platform and by release - were not part of the defined produc externals. |
and it could change again? (good reason for not documenting something...)
| markt wrote: |
| d) a loaded queue is (to a first approximation) a queue where someone has it open or has only recently closed it. |
BTW does this also apply to the MCA opening the queue???
| markt wrote: |
| So even with the new internals you cannot be certain that a message will be preemptively removed when expired. It's just a lot more likely. |
true, but then you could "see" the messages before they expired...
nice trick question for the V6 certification test
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| hopsala |
| Posted: Tue Oct 04, 2005 2:21 pm Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| PeterPotkay wrote: |
| And I assume MQOPEN just for MQINQ is enough, as that is all I had open on the queue (using the little WatchQ Support Pac) watching the depth refresh every second. |
Actually any sort of MQOPEN should be sufficient, since once you MQOPEN a queue it is "loaded", I have this excerpt out of the "v5.3 Internal presentation" by the illustrious Paul Clark of IBM (courtesy of wayne):
| In the MQOPEN processing chapter, Paul Clark wrote: |
The DAP performs the operations needed to make the physical (local) queue available. This is termed loading the
queue. It involves opening the file containing the underlying message data and allocating the shared memory buffers
and other shared resources necessary for the queue to be used. Of course, if the queue is already loaded, this work
can be avoided |
Granted this is 5.3 behavior, and undocumented behavior at that, but really this is cross-version internal processing, and i'm quite sure it hasn't changed in v6. |
|
| Back to top |
|
|
| hopsala |
| Posted: Tue Oct 04, 2005 2:23 pm Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| Michael Dag wrote: |
| markt wrote: |
| d) a loaded queue is (to a first approximation) a queue where someone has it open or has only recently closed it. |
BTW does this also apply to the MCA opening the queue??? |
Yes. An MCA is an MQ appl like any other, give or take a few mq calls only available for MCA's, MQCONN-MQOPEN-MQGET processing is normal. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Nov 09, 2006 9:29 am Post subject: |
|
|
Guest
|
| Expiry is documented in substantial detail in the Application Programming Reference (sc34-6596-00), Chapter 12, page 151. |
|
| Back to top |
|
|
|
|
