| Author |
Message
|
| PankajS |
 Posted: Fri Feb 25, 2005 5:00 am Post subject: How the logging process works ? |
 |
|
Voyager
Joined: 27 Dec 2004
Posts: 82
|
Hi friends,
We say that the performance of the MQ system decreases if we have the persistent queues, as the Persistent message is written in to the log.
but even the non-persistent message is written in to the files(we have file systems for the queues)
How can we say that the Persistent messages gives us performance penalty(In terms of response time)?
How two things are different?
Does it make any difference to the performance if I have 20 primary log files instead of default 3
please get me out of this doubt  |
|
| Back to top |
|
 |
| bower5932 |
| Posted: Fri Feb 25, 2005 7:21 am Post subject: |
|
|
Jedi Knight
Joined: 27 Aug 2001
Posts: 3023
Location: Dallas, TX, USA
|
I think that you might be confusing the difference between persistent messages and units of work. You can write a persistent message either in or out of a unit of work and it will be written to disk so that the message will survive a qmgr shutdown/restart.
Units of work refer to transactions and the logs are where information is kept on these in case you have to recover them.
To be honest, persistent messages are usually done inside of a unit of work. The bottom line is that you'll need to analyze your situation and determine which of your messages should be persistent or not and which belong in a unit of work or not. |
|
| Back to top |
|
|
| pgorak |
| Posted: Fri Feb 25, 2005 7:31 am Post subject: |
|
|
Disciple
Joined: 15 Jul 2002
Posts: 158
Location: Cracow, Poland
|
I believe a persistent message is written to the logs even if it is put outside UOW. After all, you can replay the logs and have the message back, no matter if it was put inside or outside UOW.
The performance hit is twofold:
1. A persistent message is written to the queue file AND to the log (+ some additional information concerning UOW or any technical stuff not legible for a human being)
2. A persistent message is always written to the disk, whereas a non-persistent message MAY be consumed while still in queue buffer (yes, there is a buffer for persistent messages but don't ask me how this works - anyone knows the internals?)
Piotr
_________________
***
IBM Certified Solution Developer WebSphere MQ 5.3 |
|
| Back to top |
|
|
| bower5932 |
| Posted: Fri Feb 25, 2005 8:13 am Post subject: |
|
|
Jedi Knight
Joined: 27 Aug 2001
Posts: 3023
Location: Dallas, TX, USA
|
| pgorak wrote: |
| 2. A persistent message is always written to the disk, whereas a non-persistent message MAY be consumed while still in queue buffer (yes, there is a buffer for persistent messages but don't ask me how this works - anyone knows the internals?) |
I can't say that I know the internals, but this subject comes up at the T&M conference. Basically, there is a buffer that contains messages that have just been written to the queue. It turns out that most messages that are written are usually retrieved in a short period of time. The buffer allows WMQ to give you the message out of memory rather than have to get it off of disk. |
|
| Back to top |
|
|
| bower5932 |
| Posted: Fri Feb 25, 2005 8:27 am Post subject: Re: How the logging process works ? |
|
|
Jedi Knight
Joined: 27 Aug 2001
Posts: 3023
Location: Dallas, TX, USA
|
| PankajS wrote: |
| but even the non-persistent message is written in to the files(we have file systems for the queues) |
This can happen if the memory buffer that is used to hold the messages fills up. WMQ has to do something with these messages, and they start to spill into the file system. If you are putting/getting messages quickly and not encountering a backlog, you shouldn't see non-persistent messages written to the file system.
| pgorak wrote: |
| I believe a persistent message is written to the logs even if it is put outside UOW. After all, you can replay the logs and have the message back, no matter if it was put inside or outside UOW. |
You are correct. Re-reading my append, I wasn't very clear. What I was trying to convey is that the persistence of a message has nothing to do with a UOW. As an aside, I do remember seeing a demo once with persistent messages and UOWs. The interesting thing was the performance difference between putting a single message and committing and putting a batch of messages with a single commit.
Hopefully, this clears and not clouds. |
|
| Back to top |
|
|
| jsware |
| Posted: Fri Feb 25, 2005 9:57 am Post subject: |
|
|
Chevalier
Joined: 17 May 2001
Posts: 455
|
| pgorak wrote: |
| I believe a persistent message is written to the logs even if it is put outside UOW. After all, you can replay the logs and have the message back, no matter if it was put inside or outside UOW. |
I think writing persistent messages outside a UOW to the logs only really needs to happen when using linear logs. Since circular logs don't support media recovery, you only need to write to the log what could be backed out, so anything outside a UOW does not need to go to the log.
Additionally I would expect anything written inside a UOW - even non-persistent messages would be written to the log, even with circular logging, so that it can be backed out if necessary. - Remember that non-persistent messages might spill out of the buffer into the queue file and would need removing if backed out.
Writes to the queue files are not forced to disk like the logs. Thus a machine crash could result in a damaged object.
I've seen persistent messages written inside a UOW improve performance. I guess by only requiring a write to the log, not the queue file - thus your MQPUT call can return quicker and a separate process lazy writes the queue files. If the machine crashes, the log (circular or linear) can be used to reconstruct the unwritten portion of the queue file. Only linear logging can recover a damaged object - but during a normal shutdown (possibly even a pre-emptive one) I would expect the various file writing processes to write all their outstanding data.
I have seen evidence of these performance improvements when doing some tests. Persistent messages written inside a UOW are faster than outside because the QMgr has less work to do.
_________________
Regards
John
The pain of low quaility far outlasts the joy of low price. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Feb 25, 2005 2:18 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7716
|
|
| Back to top |
|
|
| PankajS |
| Posted: Fri Feb 25, 2005 11:19 pm Post subject: |
|
|
Voyager
Joined: 27 Dec 2004
Posts: 82
|
Thanks guys ....  |
|
| Back to top |
|
|
|
|
