| Author |
Message
|
| monkeydluffy |
 Posted: Mon Feb 29, 2016 6:56 am Post subject: choosing the appropriate LogWriteIntegrity |
 |
|
Newbie
Joined: 11 Feb 2016
Posts: 9
|
Hi All,
Sorry to post on something which has been discussed before. I am bit confused on the LogWriteIntegrity setting. We have a new PROD server which has not yet gone live with MQ and IIB installed in it. We are now configuring the queue managers and brokers in them. The queue manager log and data path is on a SAN drive which is write cache enabled. With a write cache enabled disk, we can go with Single Write and the customer EAI architect also prefers this setting.
However recently there was a disk failure. Since this is a new queue manager, it did not have any objects in it. And it started back without issues.
I have referred to the link
http://www.ibm.com/developerworks/websphere/library/techarticles/0712_dunn/0712_dunn.html
| Quote: |
Level of log write
Overview
You can specify the method that the queue manager logger uses to reliably write log records. The method used is specified in the Log stanza of the queue manager configuration using the LogWriteIntegrity parameter. Possible values are:
SingleWrite
Some hardware guarantees that if a write operation writes a page and fails for any reason, a subsequent read of the same page into a buffer results in each byte in the buffer being either:
The same as before the write, or
The byte that should have been written in the write operation
On this type of hardware (for example, a SAN write cache enabled disk), it is safe for the logger to write log records in a single write because the hardware assures full write integrity. This method provides the best performance.
DoubleWrite
Default method used in WebSphere MQ V5.2 and available for back-compatibility purposes only.
TripleWrite
Default method. When hardware that assures write integrity is not available, you should write log records using the TripleWrite method because it provides full write integrity. Systems running with high volumes ( > 1000 messages per second) will see little difference between SingleWrite and TripleWrite, because it is only the last 4k block in each log write that may be subjected to three writes.
If you are satisfied as the result of a discussion with your disk provider that the device on which one which the log is located can assure write integrity, then use SingleWrite for best performance. If you change the value, you must restart the queue manager to bring the change into effect.
|
And referred to the thread
http://www.mqseries.net/phpBB2/viewtopic.php?t=68685&sid=e403b5bb97c1eefed063a02e0254178e
Two questions I am thinking to myself, got confused, and so decided to seek help here.
Was wondering if this system was live, could QM objects have been damaged and in that case would Triple Write be the preferred way?
Does choosing Single Write provide a huge performance gain to overlook the risks associated?
I need some sort of solid justification on which I discuss with the architect on setting it to Triple Write.
The log setting is circular with 40960 log file pages, 10 primary, 10 secondary. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Mon Feb 29, 2016 7:19 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Okay, so I'm slightly confused here (bad day in real life).
You have queue manager files on a hardware-protected SAN disc.
You had a disk failure.
The queue manager restarted correctly (even a new queue manager has objects in it, 70+ come from the crtmqm command and there are the logs of course)
Based on this successful recovery from disc failure, you're now looking for a solid justification to change the log integrity setting from one that clearly works to one that works but can be slower.
Do I have that about right? Because it doesn't sound right.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| umatharani |
| Posted: Mon Feb 29, 2016 8:33 am Post subject: |
|
|
Apprentice
Joined: 23 Oct 2008
Posts: 39
|
* Neither Singlewrite nor Triplewrite will be helpful if your disk is not reliable and queue manager recovery logs got damaged.
* If your disk guarantees write integrity as documented in the MQ manual, then you can go ahead with SingleWrite. Regarding the performance, this depends on how your application is designed, particularly whether they use persistent message or non persistent message. If your application only uses non persistent messages then I think the performance gain is not much when you use SingleWrite because the log records will not be written often(For example, log records will be written for persistent messages, during checkpoint etc). If your application is using persistent messages then it is worth running tests with Single and Triple write methods to understand the performance difference though SingleWrite provides better performance than TripleWrite.
Thanks, mahesh |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Feb 29, 2016 8:43 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| umatharani wrote: |
| * Neither Singlewrite nor Triplewrite will be helpful if your disk is not reliable and queue manager recovery logs got damaged. |
It doesn't help to keep your front door locked during an earthquake.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Feb 29, 2016 9:34 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
|
| Back to top |
|
|
| monkeydluffy |
| Posted: Mon Feb 29, 2016 9:22 pm Post subject: |
|
|
Newbie
Joined: 11 Feb 2016
Posts: 9
|
Thanks for all the replies.
To Vitor - Thanks for rectifying my statement about no objects in empty queue manager. Maybe I am thinking too much into this LogWriteIntegrity. I am just worried on the scenario had this happened after the system went live. There will be many more custom MQ objects created and lots of transactions flowing through.
To umatharani - Thanks for your point on this. I understand that even Triple Write would not have helped had the queue manager recovery logs got damaged. So no point in going to a slower setting, expect that almost all of the messages here are non-persistent which brings to the point of very little performance benefit with SingleWrite
To mqjeff - Thanks for validating that.
To bruce - Thanks for the link.
| Quote: |
| If you are in ANY doubt that your storage infrastructure makes the required guarantees now and in the future in all circumstances, you should use TripleWrite. |
The disk is write cache enabled. This the EMC vendor has guaranteed. That and with the above discussion, I guess I should not try to justify going to TripleWrite. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Feb 29, 2016 10:09 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Another consideration: are your n-p messages consumed immediately after they are MQPUT to a queue? If so, it is unlikely that any will actually be written to the queue-holding file system.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| monkeydluffy |
| Posted: Mon Feb 29, 2016 10:19 pm Post subject: |
|
|
Newbie
Joined: 11 Feb 2016
Posts: 9
|
Thanks bruce.
Nice point. Yes the n-p messages are all consumed immediately. As per design this EAI queue manager will not store any messages, the queue being only used for request and reply.
| Quote: |
| If so, it is unlikely that any will actually be written to the queue-holding file system. |
If message is not being written to the queue file, is all the PUT and Get operation on the queue happening at the buffer level.
 |
|
| Back to top |
|
|
| umatharani |
| Posted: Wed Mar 09, 2016 6:58 pm Post subject: |
|
|
Apprentice
Joined: 23 Oct 2008
Posts: 39
|
Each queue is defined with a fixed buffer size. If the queue is near empty then the messages will not go to the queue file. If the q depth increases then the queue buffer will become full and then message will be written to the queue file. This behavior is same for both persistent and non persistent messages. The queue buffer sizes for persistent and non persistent messages can be tuned and the details are documented in MQ performance reports.
Thanks,
mahesh |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Mar 09, 2016 8:34 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| monkeydluffy wrote: |
Thanks bruce.
Nice point. Yes the n-p messages are all consumed immediately. As per design this EAI queue manager will not store any messages, the queue being only used for request and reply.
| Quote: |
| If so, it is unlikely that any will actually be written to the queue-holding file system. |
If message is not being written to the queue file, is all the PUT and Get operation on the queue happening at the buffer level.
|
All MQPUTs and MQGETs are to/from a queue buffer in virtual storage.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
