| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| disc read/write for a message via MQ |
« View previous topic :: View next topic » |
| Author |
Message
|
| bruce2359 |
 Posted: Tue Nov 20, 2012 3:53 pm Post subject: |
 |
|
Poobah
Joined: 05 Jan 2008
Posts: 9399
Location: US: west coast, almost. Otherwise, enroute.
|
| PeterPotkay wrote: |
| An MQPUT with no syncpoint of a persistent message to a queue where there is an outstanding Get With Wait with no syncpoint will have the persistent message pass between apps with no writes to a log or queue file, if I recall correctly. |
I've heard this, too. But it does not seem to conform to the need to make a log image.
Consider the case of the consuming app ready-and-able to consume the persistent message outside a UofW, but fails to do so for any of the usual reasons - including qmgr and o/s failure. At restart, no logged image of the persistent message.
This from http://www.ibm.com/developerworks/websphere/library/techarticles/0605_mcnamara/0605_mcnamara.html
A persistent message, outside the current unit of work, is written to disk on every put and get operation.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
 |
| PeterPotkay |
| Posted: Tue Nov 20, 2012 4:31 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
Bruce,
Note the age of the article you refer to. Its using MQ 5.3. I first remember hearing about this direct pass from putter to getter bypassing the log under specific circumstances as a performance optimization at version MQ 6.0.
As for the risk of losing that message in this scenario, its like Eddie Murphy said "You brought that s*** upon yourself." The app chose to use no syncpoint and must live with the consequences. But it must be a very, very small window where there is this risk. Remember, its only skipping the logs if there is an outstanding get with wait and the message can pass thru the QM in memory and immediately from putter to getter.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| markt |
| Posted: Wed Nov 21, 2012 1:15 am Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 504
|
Even in this I/O avoidance case, there's NO risk of MQ persistence rules breaking down - there's no window when a properly-put message might get lost.
The MQPUT does not receive its return code saying the message was put until after the MQGET has successfully completed - and as everyone knows, it's only once the MQPUT has got a good RC/CC that you can say the message was accepted by MQ for processing.
If the box were to crash during the handover, then there would have been no RC to the putter, and therefore there could be no persistence guarantee.
It's never recommended to use persistent messages outside syncpoint, so this particular optimisation may not help many people, but it's the same path as gets done for non-persistent messages (which is a much more common situation). |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Nov 21, 2012 1:45 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| markt wrote: |
| The MQPUT does not receive its return code saying the message was put until after the MQGET has successfully completed... |
That seems counter-intuitive to me - waiting getter or not, and outside syncpoint, I'd expect the logical sequence of events to include the queue manager passing back the RC0 to the putter before 'releasing' the message to the getter.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Nov 21, 2012 9:26 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9399
Location: US: west coast, almost. Otherwise, enroute.
|
| PeterPotkay wrote: |
Bruce,
Note the age of the article you refer to. Its using MQ 5.3. |
The same narrative is repeated for v7 here: http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/index.jsp?topic=%2Fcom.ibm.mq.csqzal.doc%2Ffg10560_.htm
| Quote: |
A persistent message, outside the current unit of work, is written to disk on every put and get operation. See Committing and backing out units of work.
In WebSphere MQ on UNIX systems, WebSphere MQ for z/OS, and WebSphere MQ for Windows, a persistent message within the current unit of work is logged only when the unit of work is committed (and the unit of work could contain many queue operations). |
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| eva555 |
| Posted: Thu Nov 22, 2012 3:22 am Post subject: |
|
|
Novice
Joined: 10 Sep 2012
Posts: 15
|
Thanks all for the replies.... I believe I should have just replied "It depends" and explained the various circumstances under which a persistent msg would be written to a log.
Although I have one more confusion about the Logwrite integrity parameter in qm.ini.... I read in IBM site that if its triple write, its like writing thrice. This write....is it in the buffer or the disc? |
|
| Back to top |
|
|
| Andyh |
| Posted: Fri Nov 23, 2012 2:25 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 237
|
| markt wrote: |
Even in this I/O avoidance case, there's NO risk of MQ persistence rules breaking down - there's no window when a properly-put message might get lost.
The MQPUT does not receive its return code saying the message was put until after the MQGET has successfully completed - and as everyone knows, it's only once the MQPUT has got a good RC/CC that you can say the message was accepted by MQ for processing.
If the box were to crash during the handover, then there would have been no RC to the putter, and therefore there could be no persistence guarantee.
It's never recommended to use persistent messages outside syncpoint, so this particular optimisation may not help many people, but it's the same path as gets done for non-persistent messages (which is a much more common situation). |
The above is not true. The a persistent message put outside syncpoint can be passed to a waiting MQGET, again outside of syncpoint, with no further serialization and no loss of message integrity. The MQPUT could complete before the MQGET, or vice-versa.
Consider an MQGET getting a persistent message off a queue, without MQGMO_SYNCPOINT. At the moment the message linkage information in the queue manager is updated this message has been destructively got.
If the application (or queue manager) should fail after this point the message will NOT be recovered and the message will have been 'lost'. Thus when the queue manager sees an MQGET of a persistent message outside of syncpoint it recognizes the possiblity of message loss and acts accordingly, for example allowing such a message to pass from a putter to a getter in a fast and loose manner.
Note also that processing persistent message inside syncpoint is not only essential for message integrity, but it is generally more efficient as the distributed queue manager is heavily optimized for this case (the other case isn't very sensible and isn't given much consideration). |
|
| Back to top |
|
|
| Andyh |
| Posted: Fri Nov 23, 2012 2:36 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 237
|
| eva555 wrote: |
Thanks all for the replies.... I believe I should have just replied "It depends" and explained the various circumstances under which a persistent msg would be written to a log.
Although I have one more confusion about the Logwrite integrity parameter in qm.ini.... I read in IBM site that if its triple write, its like writing thrice. This write....is it in the buffer or the disc? |
TripleWrite can result in one forced log operation consuming three separate I/O operations, but on a busy system there is typically minimal overhead to using this option. This option controls how the queue manager rewrites partial pages in the recovery log. On a busy system with sufficient concurrent activity there will be VERY few partial pages written and hence the option then has minimal performance impact.
Whether you can use TripleWrite of SingleWrite depends upon whether rewriting a 4KB page to the recovery log can result in any outcome other than the entire original page being left asis, or the new page completely replacing the original page, for example in the event of a power outage or other write failure. If you've got a very heavily serialized workload and a deep knowledge of the underlying I/O subsystem there can be some benefit to using this option, but for the other 99% of us TripleWrite is the better option. |
|
| Back to top |
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
