| Author |
Message
|
| KeeferG |
 Posted: Thu May 08, 2008 12:29 am Post subject: NPMCLASS v PERSISTENT |
 |
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
Ever since MQ 5.3.6 and the introduction of NPMCLASS I have been struggling with when to use persistent messages. I understand the benefits when using internal disks but what do I gain when using SAN. As far as I can see if I use NPMSPEED(NORMAL) on my channel and NPMCLASS(HIGH) on my queue then I get persistent message behaviour. The only difference that I see is that I am not protected against corrupted objects on system startup but if I am using a SAN then what are the chances of this.
I am intersted to hear other peoples thoughts on this as I am about to have another persistent v non-persistent discussion with some developers and it would be good to have other peoples views on the matter.
Cheers
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Thu May 08, 2008 12:43 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
|
| Back to top |
|
|
| Vitor |
| Posted: Thu May 08, 2008 12:54 am Post subject: Re: NPMCLASS v PERSISTENT |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| KeeferG wrote: |
| As far as I can see if I use NPMSPEED(NORMAL) on my channel and NPMCLASS(HIGH) on my queue then I get persistent message behaviour. |
No you don't. You change the circumstances and odds of the queue manager discarding the messages.
| KeeferG wrote: |
| The only difference that I see is that I am not protected against corrupted objects on system startup but if I am using a SAN then what are the chances of this. |
You're also not protected against a queue manager crash / forced restart. Non-persistent messages may or may not survive this. Persistent messages always will
| KeeferG wrote: |
| I am intersted to hear other peoples thoughts on this as I am about to have another persistent v non-persistent discussion with some developers and it would be good to have other peoples views on the matter. |
Persistent messages should be used where the message content cannot or should not be reproduced. Non-persistent messages should be used for everything else.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| KeeferG |
| Posted: Thu May 08, 2008 1:07 am Post subject: |
|
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
We have been using NPMCLASS(HIGH) and NPMSPEED(NORMAL) on some of our transmit queues for a while for and never lost a message yet. I guess there may be a loss when writing to my local queue during a failure although we haven't had one of these on production for over three years. We have had applications fail and cause us to bring down our queue managers in a controlled fashion but that wouldn't cause a lost message.
I think I need to do a performance test of persitent v non persistent when NPMSPEED(NORMAL) is set.
On a similar vein for LogWriteIntegrity. SINGLE v TRIPLE on a SAN.
The reasons for these questions is that I work in a very high performance environment and these extra milliseconds make a big difference.
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Thu May 08, 2008 1:13 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
| KeeferG wrote: |
I think I need to do a performance test of persitent v non persistent when NPMSPEED(NORMAL) is set.
|
If the messages are persistent, NPMSPEED wont be taken care.
It will be taken care only when Non persistent messages are going through.
Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu May 08, 2008 1:15 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| KeeferG wrote: |
| The reasons for these questions is that I work in a very high performance environment and these extra milliseconds make a big difference. |
Then you've made a design decision, based on business imperatives, that you'll trade absolute message safety for improved performance on a day to day basis.
I see that as a valid decision to make, provided it's properly informed. As it seems to be here.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Thu May 08, 2008 1:22 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
If there is any mismatch between the sender and reciever NPM attributes
chances of sequence errors are very high.
NPMSPEED(FAST) :-improve the throughput of non-persistent messages
chances of loosing messages are bit high in this scenario.
Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| KeeferG |
| Posted: Thu May 08, 2008 1:40 am Post subject: |
|
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
The majority of our setup is left at default so NPMCLASS(NORMAL) and NPMSPEED(FAST).
We typically have 1000 mps over our channels and queues reading up to 5000 mps so we have to be very selective where we change these settings.
As a result all of our logging is not tuned for persitent messages so I am loathe to start adding it.
Of course we have flagged those systems where NPMSPEED(FAST) has been set to look at logs but fortunately these channels have much lower message rates. 400-500 mps
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Thu May 08, 2008 1:50 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
| KeeferG wrote: |
The majority of our setup is left at default so NPMCLASS(NORMAL) and NPMSPEED(FAST).
We typically have 1000 mps over our channels and queues reading up to 5000 mps so we have to be very selective where we change these settings.
|
I guess, this includes Persistent and NPMessages, thats the reason the NPMSPEED is set to FAST.
Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| mvic |
| Posted: Thu May 08, 2008 2:31 am Post subject: Re: NPMCLASS v PERSISTENT |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| KeeferG wrote: |
| As far as I can see if I use NPMSPEED(NORMAL) on my channel and NPMCLASS(HIGH) on my queue then I get persistent message behaviour. |
NPMCLASS is a property of the queue. It refers to a level of service offered to non-persistent messages which have reached a queue. So some consequences from this: NPMCLASS on Q1 doesn't have any bearing on non-persistent messages which are in transit towards Q1, or in transit away from Q1. It also doesn't have any bearing on messages which are on Q2 (it's possible NPMCLASS wasn't tuned up on Q2). Message persistence is a way to define - within the message itself - the importance of that message. |
|
| Back to top |
|
|
| KeeferG |
| Posted: Thu May 08, 2008 3:05 am Post subject: |
|
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
I would rather say resilience of the message than importance.
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu May 08, 2008 6:23 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
A fundamental question: do any of your applications create non-persistent messages? If not, these settings have no effect.
The issue of non-persistent vs. persistent, as discussed here and in other posts, is one of your organizations aversion to risk of loss of business data.
NPMCLASS(HIGH) will help if the qmgr shuts down gracefully. NPNSPEED(FAST) ensures that non-persistent messages are sent outside units of work - and therefore might be lost.
There is a triad (triangle) with Cost, Time and Quality. You only get two of the three. If you want Time (speed), you must sacrifice either Cost or Quality.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| KeeferG |
| Posted: Thu May 08, 2008 6:41 am Post subject: |
|
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
We currently go through around 1,000,000 message per minute during our peak loads of which maybe 1000 are persistent so these settings are used very selectively.
I have always worked on high performance systems which is why I typically avoid persistent messages. I know that there are times when persitence is absolutely required and in most cases the overhead isn't really noticable
I was actually after other peoples preferences in high performance systems and where do they typically draw the line between persistence and non persistence.
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu May 08, 2008 7:02 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| Ever since MQ 5.3.6 and the introduction of NPMCLASS I have been struggling with when to use persistent messages. |
This was your original post.
It's an application design issue. Programmers specify in the MQMD whether a message is 1) persistent, or 2) non-persistent, or 3) the message should take on the attribute of the first queue definition at MQOPEN.
With option 3), a system admin has some control of the persistence attribute of a message.
Processing of non-persistent messages is faster since they are not logged, and sent outside u-of-w's across channels with NPMSPEED(FAST). For this speed, you risk message loss. How much risk? If your applications are written to keep track of request messages sent and replies received AND to deal with missing or duplicated request/replies; then your risk is reduced. If not, what is the business plan for dealing with lost messages?
so, what issue are you struggling with?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| KeeferG |
| Posted: Thu May 08, 2008 7:32 am Post subject: |
|
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
Perhaps I should have worded it better.
I fully understand the workings of persistent v non persistent as well as NPMCLASS and NPMSPEED.
Since the addition of NPMCLASS I have since little reason to use persistent messaging in the applications that I deal with.
Developers initial response tends to always be persistent please as does the business analysts yet as most critical messages are put under syncpoint (or should be for performance) and NPMCLASS allows for messages to survive restart then that covers most failure scenarios for critical messages. The obvious one it doesn't cover is object corruption but using SAN greatly reduces this risk anyway.
On applications that are sending messages to queue managers outside of the companies control then yes persistent messaging is great. Likewise for systems where you do not trust the administrators to manage the MQ correctly but in a well controlled SAN based system where applications are well written then the need seems to disappear somewhat.
All this stuff is good practice for my talk with development next week.
If performance is not an issue then persistent is an easy choice to make.
Cheers guys
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
|
|
|
