| Author |
Message
|
| jeevan |
 Posted: Fri Jun 26, 2009 3:33 pm Post subject: when mq write message on Disc |
 |
|
Grand Master
Joined: 12 Nov 2005
Posts: 1432
|
Our MQ topology is as follows:
There are gateway queue managers reside in wintel box
There are backend ( our terminonlogy) queue managers - reside on unix box
They are cluster
the actual physical queue reside on backend queue managers
When application connect to windows queue manager, would there be any disk usage at all?
The cluster queues are persistent
app connect to -windows and put a message to a cluster queue. Is there a reason this activity uses disk ( write this mesasge to disc /log) first ?
Thanks |
|
| Back to top |
|
 |
| PeterPotkay |
| Posted: Fri Jun 26, 2009 4:08 pm Post subject: Re: when mq write message on Disc |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| jeevan wrote: |
The cluster queues are persistent
|
Queues are not persistent, messages are or are not.
If the messages are persistent, they will go to disk.
If the messages are not persistent, they won't got to disk. In most cases.
| jeevan wrote: |
app connect to -windows and put a message to a cluster queue. Is there a reason this activity uses disk ( write this mesasge to disc /log) first ?
|
Are the apps use syncpoint, which uses the logs, which uses the disk?
Are the apps using persistent messages?
Are the apps putting enough non persistent messages to overflow the default q buffer?
Non persistent messages put outside of syncpoint via an XMITQ that has a running sending channel with a NPMSPEED of FAST that is keeping the XMITQ at zero or close to it will probably eliminate disk I/O for each individual message.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jeevan |
| Posted: Sat Jun 27, 2009 3:54 pm Post subject: Re: when mq write message on Disc |
|
|
Grand Master
Joined: 12 Nov 2005
Posts: 1432
|
| PeterPotkay wrote: |
| jeevan wrote: |
The cluster queues are persistent
|
Queues are not persistent, messages are or are not.
If the messages are persistent, they will go to disk.
If the messages are not persistent, they won't got to disk. In most cases.
| jeevan wrote: |
app connect to -windows and put a message to a cluster queue. Is there a reason this activity uses disk ( write this mesasge to disc /log) first ?
|
Are the apps use syncpoint, which uses the logs, which uses the disk?
Are the apps using persistent messages?
Are the apps putting enough non persistent messages to overflow the default q buffer?
Non persistent messages put outside of syncpoint via an XMITQ that has a running sending channel with a NPMSPEED of FAST that is keeping the XMITQ at zero or close to it will probably eliminate disk I/O for each individual message. |
Peter,
Thank you for your reply and query for bigger scenario. Let me repeat the scenario.
There are two group of queue manager in cluster.
First group( windows based) where requesting application connect and put the message
The other group of queue manager ( we call backend), where responding application connect, pick up the message and respond.
The backend queue managers hold the actual request queue but they are cluster queue so the requesting application can connect the frond queue manager and put the message in the queue.
my question for the frontend queue manager server. As the messages are put in to cluster queues, will MQ use disk I/O in the frontend queue managers?
I am asking so much focused because we are debugging a application slowness problem and considering all possibility. The windows folks says he observed some io problem in these virtual mq servers which are using old SAN storage.
I do not know for sure, but these messages I am referring are request/reply so most probably they are not persistent messages.
Thanks |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sat Jun 27, 2009 6:49 pm Post subject: Re: when mq write message on Disc |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| jeevan wrote: |
| my question for the frontend queue manager server. As the messages are put in to cluster queues, will MQ use disk I/O in the frontend queue managers? |
Non persistent messages put outside of syncpoint via an XMITQ that has a running sending channel with a NPMSPEED of FAST that is keeping the XMITQ at zero or close to it will probably eliminate disk I/O for each individual message.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sat Jun 27, 2009 9:25 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
I'm a bit confused. Are your applications client-binding applications?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Sun Jun 28, 2009 1:04 am Post subject: Re: when mq write message on Disc |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jeevan wrote: |
First group( windows based) where requesting application connect and put the message
The other group of queue manager ( we call backend), where responding application connect, pick up the message and respond. |
So there are no manually defined channels in here? The "frontend" applications put to cluster queues hosted by the "backend" queue managers? No point to point for the replies, that sort of thing?
Then it comes down to my esteemed associate's comment around syncpoint which you've not answered definitively. Saying they're request/replies and "probably" non-persistent is a bit woolly. Look. Check.
Investigate. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sun Jun 28, 2009 9:22 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
More information will help us help you.
What versions of mq?
What hardware and o/s versions? Number of processors?
Are these servers and qmgrs new? Any performance problems/issues prior to this application?
Are these new applications? Have they been modified recently?
Is this a new cluster?
Are the applications server-bindings apps? Or client-bindings?
Is the responding application triggered?
Does the requesting application wait on a dynamic reply-to-queue for the reply message? Or some other technique?
How many messages per-second? Or per-minute?
What size are the messages? 100meg? 1k?
In general, logging of persistent messages is not a performance issue, especially for server-bindings apps, point-to-point channels, small (a relative term) message sizes.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Sun Jun 28, 2009 4:29 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| bruce2359 wrote: |
More information will help us help you.
What versions of mq?
What hardware and o/s versions? Number of processors?
Are these servers and qmgrs new? Any performance problems/issues prior to this application?
Are these new applications? Have they been modified recently?
Is this a new cluster?
Are the applications server-bindings apps? Or client-bindings?
Is the responding application triggered?
Does the requesting application wait on a dynamic reply-to-queue for the reply message? Or some other technique?
How many messages per-second? Or per-minute?
What size are the messages? 100meg? 1k?
In general, logging of persistent messages is not a performance issue, especially for server-bindings apps, point-to-point channels, small (a relative term) message sizes. |
Don't overload the poor guy with questions. It is unlikely he will answer all of these. Start with a simple leading question and then build from there.
_________________
Glenn |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sun Jun 28, 2009 7:25 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
It's a test.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Sun Jun 28, 2009 11:38 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| gbaddeley wrote: |
| Don't overload the poor guy with questions. It is unlikely he will answer all of these. |
We live in hope.
| gbaddeley wrote: |
| Start with a simple leading question and then build from there. |
I think jeevan is well used to our little ways by now.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Sun Jun 28, 2009 11:39 pm Post subject: Re: when mq write message on Disc |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Vitor wrote: |
| Investigate. |
I personally stand by this.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jeevan |
| Posted: Wed Jul 01, 2009 9:59 am Post subject: |
|
|
Grand Master
Joined: 12 Nov 2005
Posts: 1432
|
Please see the answer in Italics
| bruce2359 wrote: |
More information will help us help you.
What versions of mq?
mq. 6.0.2.2
What hardware and o/s versions? Number of processors?
wintel front end /sparc backend
Are these servers and qmgrs new? Any performance problems/issues prior to this application?
These are in place since Last July (july 2008)
Are these new applications? Have they been modified recently?
They are been running and no change recently
Is this a new cluster?
No.
Are the applications server-bindings apps? Or client-bindings?
No. Client connection
Is the responding application triggered?
No.
Does the requesting application wait on a dynamic reply-to-queue for the reply message? Or some other technique?
It wait for on a local queue
How many messages per-second? Or per-minute?
I do not have stats on that.
What size are the messages? 100meg? 1k?
1 k
In general, logging of persistent messages is not a performance issue, especially for server-bindings apps, point-to-point channels, small (a relative term) message sizes. |
I already mentioned, both the qmgr application connect to put message and wait for a response and the queue manager responding application connect to pick up the messages and respond are in the same cluster.
Thank you very much |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jul 01, 2009 10:06 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Yes.
There will be disk usage. |
|
| Back to top |
|
|
| jeevan |
| Posted: Wed Jul 01, 2009 1:22 pm Post subject: |
|
|
Grand Master
Joined: 12 Nov 2005
Posts: 1432
|
| bruce2359 wrote: |
| I'm a bit confused. Are your applications client-binding applications? |
Is it the same as saying client connection? Yes. the app and mq queue managers are not in the same box. I am not sure though whether I answered your question. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Jul 01, 2009 1:39 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
I guess I'm confused by the answers to my long list of questions, and what conclusions it led me to - likely incorrectly.
If your messages are persistent, they will be logged multiple times. At the very least when the msg is put to the destination queue and when the msg is mqget from the destination queue.
You didn't say if the destination queue is local to the qmgr where the requesting app mqconnects, only that the queue is clustered. Does an instance of the clustered queue exist on the qmgr where the app mqconnects?
It appears that your app is using a modified request-reply model. Modified in the sense that the requesting app does not create a dynamic queue for the replying app to send the reply msg. I gather that some pre-defined queue is used for the reply msg. Yes?
If no local instance of the request clustered queue exists on the front end qmgr, then the msg ends up in the cluster xmit queue, then it is logged at mqput by the app, and mqget by the mca. The msg will be logged again at the destination qmgr at mqput to the destination queue, and again when mqget by the replying app.
The return trip will entail the same logging - disk writes.
Client apps have additional network flows as the qmgr sends CC/RC for each and every MQ call. Client apps are not the stellar performers that a server-bindings app (app and qmgr on the same o/s instance).
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
