| Author |
Message
|
| PeterPotkay |
 Posted: Tue Dec 04, 2007 10:46 am Post subject: AMQ7234 Loading Queues |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Test Case
MQ 6.0.2.1
Windows 2003 SP1
Load up a queue with 150,000 persistent messages. Make them about 1K in size or larger.
Open up an amqsputc session to the queue and put a new message to the queue. This completes immediately as expected.
Restart the QM.
Open up the amqsputc session and try to put another message to that same q. While you wait and wait and wait for that MQPUT to complete, go look at the QM error log. You will see this message over and over:
| Code: |
-------------------------------------------------------------------------------
12/1/2007 23:55:02 - Process(4624.7) User(mqadmin_ad1) Program(amqzlaa0.exe)
AMQ7234: 10000 messages from queue 'MY.LOCAL.QUEUE' loaded on queue manager 'QM1'.
EXPLANATION:
10000 messages from queue MY.LOCAL.QUEUE have been loaded on queue manager QM1.
ACTION:
None.
-------------------------------------------------------------------------------
|
It loads it at the rate of about 10 seconds for every 10,000 messages. Only when the q is fully loaded will the MQPUT complete. This is obviously a problem as for deep queues the MQPUT can take minutes to complete.
http://www-1.ibm.com/support/docview.wss?rs=171&context=SSFKSJ&dc=DB520&uid=swg21251301&loc=en_US&cs=UTF-8&lang=en&rss=ct171websphere
The messages in the log are new as of MQ 6.0, but is the loading behavior itself new to MQ 6.0? Is there anyway around this? The app area wants to know what's wrong with MQ - why is it so slow after a restart? If you really want to see if drag, do a q list in MO71 for all queues as soon as the QM starts. The QM doesn't give a response until all the queues have been loaded. If you have lots of queues with lots of messages*, you find yourself waiting a very long time.
*Yes, I know you are not supposed to have lots of queue with lots of messages all the time. But the fact is a QM can have lots of queues and any queue can have lots of messages at any given point in time, and if the QM is restarted at that point…...
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| markt |
| Posted: Tue Dec 04, 2007 11:09 am Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 508
|
Queues are loaded the first time they're opened after a restart. This has always been the behaviour - the message is new to indicate something is going on and that the queue manager is not simply hung.
I've not looked at the loading performance so can't say if 1000 messages a second is unexpectedly slow - it sounds like it might be, but it might depend on the hardware and possibly if the messages are parts of segments/groups when more work has to be done. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Dec 04, 2007 12:33 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Mark,
The messages are not part of any group. Unfortunatly I don't have a 5.3 QM to play around with anymore to compare the performance. I gotta think I would have noticed this during the 3 or so years we were at 5.3 if this was so slow in that version.
One of our test QMs has about a dozen queues with about 60-70K messages each (logging type stuff used in developement). If I restart the QM and then immedialty do a q list for all queues it takes over 5 minutes to come back.
I opened a PMR. The initial response was "Works as designed, don't have a lot of mesages in your queues." 
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Scubie |
| Posted: Mon Jun 24, 2013 9:42 pm Post subject: AMQ7234 - configurable |
|
|
Apprentice
Joined: 09 Oct 2001
Posts: 26
Location: Pretoria, South Africa
|
I know this thread is old but I have a similar problem with 5 queues which are not used very often but each time 20-30k messages must be loaded first taking up to 30 minutes!
Is there any way to configure this behavior? i.e. that MQ does not page the queue out of memory or at least the time interval before it does so?
Thanks
_________________
________________________________
Carlo Henrico
Live fast, die young, enjoy a good looking corpse! |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Tue Jun 25, 2013 4:00 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
http://www-01.ibm.com/support/docview.wss?uid=swg21251301
Technote: AMQ7234 is issued periodically in WMQ
Question
What is the meaning of the AMQ7234 messages in WebSphere MQ (WMQ)?
Example: AMQ7234: 10000 messages from queue 'MY.LOCAL.QUEUE' loaded on queue manager 'QM1'.
Answer
These messages are informational, and can be safely ignored.
The processing involved and meaning of the AMQ7234 messages:
When a queue is unreferenced for a period of activity, then MQ unloads the queue from memory. This process is known as "unloading the queue" .
When the queue is next referenced the queue index needs to be rebuilt in memory. This is known as loading the queue.
As the queue is loaded, message AMQ7234 is issued periodically (by default every 10,000 messages).
So, depending upon the access pattern, AMQ7234 messages can occasionally occur for queues containing over 10,000 messages.
_________________
Glenn |
|
| Back to top |
|
|
| Scubie |
| Posted: Tue Jun 25, 2013 10:47 pm Post subject: Copy/Paste |
|
|
Apprentice
Joined: 09 Oct 2001
Posts: 26
Location: Pretoria, South Africa
|
Hi gbaddeley
Not sure if you were attempting to answer my question, if so, please read it again. If not, please ignore this...
_________________
________________________________
Carlo Henrico
Live fast, die young, enjoy a good looking corpse! |
|
| Back to top |
|
|
| Andyh |
| Posted: Wed Jun 26, 2013 12:20 pm Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
When the queue manager is started, a minimal memory instantiation of each queue known to the queue manager is created. When a queue is first referenced (e.g at MQOPEN) and index for the queue is created in memory, and this involves reading the control data (e.g MQMD) for each message (not the message payloads) from disk.
If the queue is subsequently unreferenced for a prolonged period of activity it may be unloaded, after which it would then need to be reloaded at the next reference.
Loading the queue would normally be limited by disk read performance, only loading at 1000 msg/sec suggests that you have a significant issue with disk read performance (typically each message loaded will involve reading around 500 bytes, thus you're only managing to read approx 500KB/sec.
What sort of technology are you using for the file system where the MQ data is stored ? |
|
| Back to top |
|
|
| Scubie |
| Posted: Wed Jun 26, 2013 2:20 pm Post subject: Technology |
|
|
Apprentice
Joined: 09 Oct 2001
Posts: 26
Location: Pretoria, South Africa
|
Hi Andyh
This is on an HP-UX Itanium server connected via fibre to an EVA. The queues are on the EVA.
Cheers
Carlo
_________________
________________________________
Carlo Henrico
Live fast, die young, enjoy a good looking corpse! |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Wed Jun 26, 2013 3:31 pm Post subject: Re: AMQ7234 - configurable |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| Scubie wrote: |
| I know this thread is old but I have a similar problem with 5 queues which are not used very often but each time 20-30k messages must be loaded first taking up to 30 minutes! |
Why are so many messages kept in MQ? MQ is designed to be a temporary store and forward mechanism. Messages should be consumed by an application as soon as practical.
Does the "problem" occur when starting the queue manager, or while it is running?
30 minutes seems like a long time. I would expect 5 queues of 30K messages each to take at most 1 - 2 minutes to index.
| Quote: |
| Is there any way to configure this behavior? i.e. that MQ does not page the queue out of memory or at least the time interval before it does so? |
I don't think so. You can raise a Service Request with IBM to ask the question.
_________________
Glenn |
|
| Back to top |
|
|
| Tibor |
| Posted: Thu Jun 27, 2013 1:22 am Post subject: Re: AMQ7234 - configurable |
|
|
Grand Master
Joined: 20 May 2001
Posts: 1033
Location: Hungary
|
| Scubie wrote: |
| ... I have a similar problem with 5 queues which are not used very often but each time 20-30k messages must be loaded first taking up to 30 minutes! |
This amount of time sounds dreadfully, because a similar case in our environment (AIX) is less than 30 seconds...  |
|
| Back to top |
|
|
|
|
