| Author |
Message
|
| bruce2359 |
 Posted: Thu Oct 16, 2014 10:17 am Post subject: TripleWrite |
 |
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
I'm in need of an explanation of LogWriteIntegrity=TripleWrite.
Yes, I've searched and read. I'm presuming that the value TripleWrite implies that something will be written three times. But what? What's missing for me is an understanding of exactly what happens three times with TripleWrite?
Does stuff in the log buffer(s) get written three times to disk? To the same place on disk? Or, does the stuff destined for the log file get written to three buffers, then to disk? Or, does the stuff written to disk get read back hree times to validate that it was written successfully?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
 |
| Andyh |
| Posted: Thu Oct 16, 2014 10:45 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 237
|
In the early days of MQ there were only two options, SingleWrite and DoubleWrite.
SingleWrite depended upon the atomicity of writing 4KB pages to the MQ recovery log, DoubleWrite did not.
A potential data integrity issue was subsequently discovered with the algorithm used to implement DoubleWrite, and the TripleWrite algorithm was implemented to overcome that issue. DoubleWrite no longer serves any useful purpose.
Hence the current situation is that SingleWrite should only be used if the file-system/device hosting the MQ recovery log guarantees the atomicity of 4KB writes (that is when a write of a page fails for ANY reason the only two possible states should be either the before image, or the after image and that no intermediate state should be possible), in all other cases TripleWrite should be used.
On a busy system with sufficient concurrency the queue manager will try to only write full pages of log data, and if a high percentage of full pages is achieved then there will be minimal performance difference between SingleWrite and TripleWrite.
On a system with little concurrency there would typically be a significant performance advantage to SingleWrite, however the correct solution would typically be to increase concurrency rather than to use SingleWrite.
In in ANY doubt, you should use TripleWrite. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Oct 16, 2014 10:52 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
Thanks for all that, Andy; but how does it work?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| tczielke |
| Posted: Thu Oct 16, 2014 12:53 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA
|
FYI - This is from an excerpt from the MQ LISTSERV where Andy replied to my question about TripleWrite vs. Single Write, which adds some more detail.
| Quote: |
There is a lot of misunderstanding surrounding SingleWrite/TripleWrite. In a well designed high throughput persistent messaging environment, there should be a high degree of concurrency, and in such an environment the overhead of TripleWrite should be minimal.
TripleWrite/SingleWrite only behave differently in the case of partial pages. The queue manager tries to avoid writing partial pages (this is much better at V7.1 and beyond) and only writes a partial page if progress cannot be made through writing a full page (i.e all outstanding persistent activity requires the last incomplete page to be written).
Unless you really know what you're doing you should steer clear of SingleWrite. A detailed understanding of how the low level I/O for your particular adapter and device is required in order to understand if SingleWrite really is safe.
For the tiny percentage of people with a high throughput persistent workload with little concurrency, and a sufficiently detailed understanding of the low level I/O to understand whether a partial 4KB page could ever be written to disk during a failing write, there would be potential advantages to exploiting this option." |
|
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Oct 16, 2014 1:22 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
Yep, already read that; but how does it work? Triple what?
I believe I understand the intent, namely: to ensure log write integrity with disk hardware that can/cannot ensure write integrity. All of the stuff I've researched/read has merely echoed the intent, and how to pick Single or Triple.
I suppose at some time in the distant past that the number three was somehow involved.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| tczielke |
| Posted: Thu Oct 16, 2014 4:22 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA
|
TripleWrite does sound like some type of algorithm that involves a write, a validation/error check, a second write, a validation/error check, and then a third write, but of course that is pure speculation on my part. The information that Andy provides is enough for me to make an informed decision. In our case, to use TripleWrite.
When Andy mentions things like 4K pages, this starts to sound like direct I/O where when MQ issues the write, the data would be transferred directly from the MQ process address space to the device driver (i.e. bypassing any type of kernel buffer that a normal write would travel through). But this is really getting down into the internals, and the information that Andy has there is enough for me to make an informed decision. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Oct 16, 2014 11:33 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5849
Location: UK
|
If you are using modern SAN devices - I would be surprised if TripleWrite was really necessary.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| Andyh |
| Posted: Fri Oct 17, 2014 12:41 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 237
|
With "TripleWrite" up to three write calls can be needed write a single 4KB log page.
With "SingleWrite" a single write call is needed to write a single 4KB log page.
As already explained, in a well designed system with sufficient concurrency there will be minimal difference in performance between SingleWrite and TripleWrite.
If you know exactly how the underlying filesystem and device handle ANY write failure (and can therefore guarantee the atomicity of 4KB writes), and have a strongly serialized workload, then SingleWrite would be a viable option.
If you're in any doubt you should be using TripleWrite.
If you're after a general performance boost for persistent messages then I'd suggest you try to keep relatively current on MQ versions.
V8 > V7.5 > V7.1 >> V7.0
(V8 is perhaps a bit new for general production deployment, but I'd be particularly interested to hear of any V8 performance observations). |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Oct 17, 2014 1:24 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5849
Location: UK
|
Hopefully NVRAM, striped data, UPS and Synchronous Remote Data Facility (to our second data centre) would cover it!
We found TripleWrite caused a large performance degradation over SingleWrite - possibly because of our use of SRDF.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Oct 17, 2014 3:14 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
http://www.ibm.com/developerworks/websphere/library/techarticles/0712_dunn/0712_dunn.html
| Quote: |
Level of log write
Overview
You can specify the method that the queue manaGer logger uses to reliably write log records. The method used is specified in the Log stanza of the queue manager configuration using the LogWriteIntegrity parameter. Possible values are:
SingleWrite
Some hardware guarantees that if a write operation writes a page and fails for any reason, a subsequent read of the same page into a buffer results in each byte in the buffer being either:
The same as before the write, or
The byte that should have been written in the write operation
On this type of hardware (for example, a SAN write cache enabled disk), it is safe for the logger to write log records in a single write because the hardware assures full write integrity. This method provides the best performance
DoubleWrite
Default method used in WebSphere MQ V5.2 and available for back-compatibility purposes only.
TripleWrite
Default method. When hardware that assures write integrity is not available, you should write log records using the TripleWrite method because it provides full write integrity. Systems running with high volumes ( > 1000 messages per second) will see little difference between SingleWrite and TripleWrite, because it is only the last 4k block in each log write that may be subjected to three writes.
If you are satisfied as the result of a discussion with your disk provider that the device on which one which the log is located can assure write integrity, then use SingleWrite for best performance. If you change the value, you must restart the queue manager to bring the change into effect.
|
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Oct 17, 2014 4:06 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9396
Location: US: west coast, almost. Otherwise, enroute.
|
| Andyh wrote: |
With "TripleWrite" up to three write calls can be needed write a single 4KB log page.
With "SingleWrite" a single write call is needed to write a single 4KB log page. |
Ah, we're getting somewhere. The number three is involved.
In what circumstances would (up to) three write calls be needed to write the 4K log buffer page to disk? Why would more than one be necessary?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Oct 17, 2014 5:00 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| In what circumstances would (up to) three write calls be needed to write the 4K log buffer page to disk? Why would more than one be necessary? |
I am not now nor have I ever been a hardware specialist, but I understand that under some circumstances with some kinds of hardware, the disc is written as a direct stream with no cache at the hardware level to provide the greatest access speed. So if the hardware is under load (for example) the write operation may not complete properly; kind of "it's better to do it fast than do it right". So it may take three attempts to get the data down. Why 3 rather than 5 or an unlimited number of attempts? No clue; the hardware wizards have decided that is a magic number. Presumably that's what "the algorithm" works out to.
As previously discussed, some hardware (like SAN) is built for "it's better to do it right than do it fast" and have hardware buffers & other exotic dohickies to ensure integrity.
I know we have MQ data files on SAN for HA, I know we use TripleWrite and I know our performance is well under our SLA.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| tczielke |
| Posted: Fri Oct 17, 2014 5:12 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA
|
| One other thing to consider. Even if you do the proper analysis and research, work with your disk provider, and confirm you are safe to use SingleWrite, how do you account for sometime in the future when that same queue manager is replatformed to another disk infrastructure that does not meet the SingleWrite requirements? Would you have the operational procedures in place to catch that and then change your queue manager from SingleWrite to TripleWrite before being replatformed? Or how do you prevent a disk administrator from accidentally/intentionally making an administrative change that causes you to be no longer covered for SingleWrite? I am not saying you can't properly use SingleWrite from an operational stand point. But it does have its ongoing operational challenges. Since TripleWrite gives us the performance we need and log write integrity is very important, we really had only one option for us. TripleWrite. |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Oct 17, 2014 6:00 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5849
Location: UK
|
Seems unlikely that a new infrastructure would be inferior.
Any such move would involve considerable storage preparation, and would have HA and SAN considerations as a key part of planning.
You could always put a comment in the qm.ini file if you were worried about it.
Seems odd to sacrifice useful performance on the basis that someone might be incompetent in the future, in an unlikely scenario.
On the rare occasion that I replatform an existing QM, I take every chance to improve its performance and robustness - and review its setup - generally raise limits where possible etc to allow for growth.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| tczielke |
| Posted: Fri Oct 17, 2014 6:04 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA
|
| tczielke wrote: |
| Since TripleWrite gives us the performance we need and log write integrity is very important, we really had only one option for us. TripleWrite. |
That statement is too strong. A better wording would have been, "Since TripleWrite gives us the performance we need and log write integrity is very important, we felt the best option for our environment was TripleWrite." |
|
| Back to top |
|
|
|
|
