| Author |
Message
|
| Volodya |
 Posted: Thu Sep 29, 2005 11:54 pm Post subject: |
 |
|
Novice
Joined: 04 Mar 2004
Posts: 22
Location: Moscow
|
I have found one more way of the decision of expired message problem.
We process very much greater flows of messages with Expiry Messages
(approximately 10 million messages for a day through one queue).
Our server fell if there is a big flow of messages with Expiry Messages.
There's no equivalent command available on distributed platforms as the command for z/OS named REFRESH QMGR TYPE(EXPIRY) NAME(*).
These expired messages are discarded when an application does an MQGET(browse/non-browse) of these messages.
But using MQGET in a cycle for all messages lowers performance at work with WMQ. Thus MQGET allows to solve a problem partially.
WMQ should release disk space as soon as the message "dies".
I tested Garbage Collector for Expiry Messages on WMQ V6.
It work fine.
On WMQ V6 the queue manager automatically delete expired messages from loaded queues.
_________________
Vladimir Makushkin, WebSphere MQ administrator |
|
| Back to top |
|
 |
| PeterPotkay |
| Posted: Fri Sep 30, 2005 7:36 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Volodya wrote: |
I tested Garbage Collector for Expiry Messages on WMQ V6.
It work fine.
On WMQ V6 the queue manager automatically delete expired messages from loaded queues. |
What is Garbage Collector for Expiry messages?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Volodya |
| Posted: Sun Oct 02, 2005 9:45 pm Post subject: |
|
|
Novice
Joined: 04 Mar 2004
Posts: 22
Location: Moscow
|
"Garbage Collector for Expiry messages" is an automatic clearing disk memory.
It is my own definition.
Expiry messages ARE NOT discarded automatically on WMQ V5.3, disk space is overflown for NT server.
Under disk space I understand C:\Program Files\IBM\WebSphere MQ\Qmgrs\QM1\QUEUES\StockQueus .
When I have seen that WMQ V6 released disk space as soon as the message "dies" I have named this mechanism "Garbage Collector" by analogy with UNIX , C/C++ , Java , etc.
In my opinion this definition describes correctly the mechanism realized for problem of Expiry messages on WMQ V5.3.
_________________
Vladimir Makushkin, WebSphere MQ administrator |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Mon Oct 03, 2005 12:44 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
I think Peter wants to know what tool you use to achieve this garbage collecting (I do too  )
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| Volodya |
| Posted: Mon Oct 03, 2005 1:22 am Post subject: |
|
|
Novice
Joined: 04 Mar 2004
Posts: 22
Location: Moscow
|
I used own very simple program with parameter md. Expiry = 10;
I sent 1000000 messages ( length 1Kb ) and observed next effects on WMQ V6.
1) Expiry messages "died" in queue.
2) Disk space (C: \Program Files \IBM \WebSphere MQ \Qmgrs \QM1 \QUEUES \StockQueus) varied dynamically from 10Mb up to 20Mb. Disk space = 2048 bytes in the end of experiment.
Such experiment on WMQ V5.3 gave next result - disk space grew up to 1.5Gb and was not released !
_________________
Vladimir Makushkin, WebSphere MQ administrator |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Mon Oct 03, 2005 1:41 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
ok, so you don't have something to do garbage collection on MQ 5.3 right?
you just observed the filesystem to grow.
On MQ V6 you observe expired message to "die" and so filestorage is released. How / What did you do to observe the "dying" messages?
it could be the new Eclipse tooling is doing a get with browse to "view" the message on the queue causing them to "die" when expiry is set.
maybe the old 5.3 MMC does not do that.
It's guessing for me right now, as far as I know there is nothing in the V6 distributed code (as opposed to z/OS) to make expired messages go away.
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| Volodya |
| Posted: Mon Oct 03, 2005 4:00 am Post subject: |
|
|
Novice
Joined: 04 Mar 2004
Posts: 22
Location: Moscow
|
On MQ 5.3 I make refresh queue only (no browse) on server Windows NT and I looked the size of a file.
Old 5.3 MMC shows only growth of messages up to 1000000 and Windows tools show growth of a file for queue.
On MQ 6 I make the same.
On MQ 6 I make refresh queue (no browse) on other server Windows NT and I looked the size of a file. MQ Explorer showed dynamics of dying and from 3000 up to 5000 messages was in queue. About the size of a file I have written above.
Michael wrote:
it could be the new Eclipse tooling is doing a get with browse to "view" the message on the queue causing them to "die" when expiry is set.
I do not know what mechanisms used IBM but it works and this opportunity can be recommended for use.
_________________
Vladimir Makushkin, WebSphere MQ administrator |
|
| Back to top |
|
|
| markt |
| Posted: Mon Oct 03, 2005 4:36 am Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 508
|
>> as far as I know there is nothing in the V6 distributed code (as opposed to z/OS) to make expired messages go away.
But there is ... messages are automatically expired from loaded queues. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Oct 03, 2005 4:43 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| markt wrote: |
>> as far as I know there is nothing in the V6 distributed code (as opposed to z/OS) to make expired messages go away.
But there is ... messages are automatically expired from loaded queues. |
ExpiryInterval is still a z/OS only parameter...
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| hopsala |
| Posted: Mon Oct 03, 2005 5:17 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| Volodya wrote: |
| Such experiment on WMQ V5.3 gave next result - disk space grew up to 1.5Gb and was not released |
Queue files disk space in v5.3 is not released upon destructively MQGETing messages, but is reused; only when you restart the qm, or issue "CLEAR QL" will these files decrease in size.
See WMQ5.3 WinXP / Queue Files - Research - experiment 4 and summary, and Peter's+Roger's response. |
|
| Back to top |
|
|
| markt |
| Posted: Mon Oct 03, 2005 6:36 am Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 508
|
>> ExpiryInterval is still a z/OS only parameter...
Indeed it is. But I said "automatically". There is no external tuning on Distributed platforms, but it happens anyway. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Oct 03, 2005 7:11 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| markt wrote: |
>> ExpiryInterval is still a z/OS only parameter...
Indeed it is. But I said "automatically". There is no external tuning on Distributed platforms, but it happens anyway. |
I remember seeing comments about this, in the announcement or something...
but I've been unable to find references in the actual product documentation... Maybe I haven't looked hard enough?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| wschutz |
| Posted: Mon Oct 03, 2005 8:15 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
afaik, there's nothing new in v6 for expired messages.....
_________________
-wayne |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Oct 03, 2005 9:17 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
For what its worth, I put 100 messages on a queue with a short expiry on MQ 6.0 - Linux.
Behaviour seems the same as 5.3. Even though the queue has nothing but expired messages, the depth still shows 100. And as soon as I get/browse, and not until I get/browse, do the messages finnally blink away.
mark, are we missing something?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| wschutz |
| Posted: Mon Oct 03, 2005 9:40 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
| Quote: |
mark, are we missing something?
|
I don't read anything in his comments that imply new function. He's correcting a statement:
| Quote: |
>> as far as I know there is nothing in the V6 distributed code (as opposed to z/OS) to make expired messages go away.
But there is ... messages are automatically expired from loaded queues. |
That was true in v5 and still true in v6..... 
_________________
-wayne |
|
| Back to top |
|
|
|
|
