| Author |
Message
|
| ravishankarn |
 Posted: Mon Apr 07, 2008 1:35 am Post subject: Some strange suggestion during a problem |
 |
|
Novice
Joined: 06 Feb 2007
Posts: 14
|
Hi all,
Please help us to have your exter suggestion on below issue:-
1) We have front end application called "Gemini".
2) Gemini puts messages to "remote queue" R1. From R1 , a Xmit queue takes message. From there it goes to channel. From channel it goes to local queue at CICS. Then CICS processes it.
3) After processing, CICS puts the message to "remote queue" R2. From R2, an Xmit Queue at CICS takes the message, puts into channel, received by Gemini's local queue.
It works wonderfully. When we are doing a stress testing, we have seen that some times, the messages are getting stuck at Xmit queue at CICS side.
We analysed the messages, and an explanation we got for this is that whenever we are pumping 30K messages a lot from Gemini, the problem occurs. If it is the size issue, our argument is that it should happen at Gemini's Xmit queue side also, right??
We are not convinced!!! 1000 messages of 30K size - 30GB in total - will it slow down mq processing. Then we have difficulty to believe in the stability of the product .
It might be something else, can anyone here advise us, whether such a problem had been experienced by anyone? What could be the issue?
Best
Ravion.  |
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Mon Apr 07, 2008 1:44 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
Its important to know the max queue depth and max message length of XMITQ's at both the sides. (it might not be the same  )
Performance wise, this product is the best in Market.
Assume if the application Gemini is taking time to process the message coming from CICS. 
Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Apr 07, 2008 1:57 am Post subject: Re: Some strange suggestion during a problem |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ravishankarn wrote: |
| 2) Gemini puts messages to "remote queue" R1. From R1 , a Xmit queue takes message. From there it goes to channel. From channel it goes to local queue at CICS. Then CICS processes it. |
Strictly speaking, the message goes directly to the xmitq. There's no storage associated with a remote queue definition.
| ravishankarn wrote: |
| It works wonderfully. When we are doing a stress testing, we have seen that some times, the messages are getting stuck at Xmit queue at CICS side. |
Do you mean stuck, or do you mean not transmitted immediately but arrive eventually? I'm assuming the latter.
| ravishankarn wrote: |
| We analysed the messages, and an explanation we got for this is that whenever we are pumping 30K messages a lot from Gemini, the problem occurs. If it is the size issue, our argument is that it should happen at Gemini's Xmit queue side also, right?? |
Wrong. If the messages are being delivered eventually, then the likely explaination is that the channel can't deliver the messages immediately because the target queue is full. Possible explainations include (but are not limited to) this Gemini thing not pulling messages off fast enough and the queue filling up. Once the queue empties and space is available more messages will flow.
| ravishankarn wrote: |
| We are not convinced!!! 1000 messages of 30K size - 30GB in total - will it slow down mq processing. Then we have difficulty to believe in the stability of the product . |
30GB of message data is nothing. If you've got a local queue, and have not bothered to tune it & just defined it then the max depth will be 5000 messages. If you're stress testing by sending 1000 messages a second this will fill in 5 seconds unless this Gemini thing is very sharp at removing them.
Also you don't have a stability problem. If the product was unstable it would crash under the load, or start dropping messages. You've provided no evidence this is happening. All WMQ guarantees is that all messages will be delivered according to their settings; it doesn't guarantee how fast that happens.
| ravishankarn wrote: |
| It might be something else, can anyone here advise us, whether such a problem had been experienced by anyone? What could be the issue? |
The issue sounds like you're doing a stress test in an environment not set up to handle the stress. What is the expected / required enqueue/dequeue rate on both sides? Do you have enough storage (queue depth) to handle the lag between this rate and the transmission rate? What action should either the Gemini application or the CICS app take in the event of stress? How is stress detected by either application? What is your monitoring? Is it working? What does this Gemini thing run on - Windows desktop or big Unix box? How fast is it processing? Does it need more memory? Is either or both app single or multi threaded? Is there a database involved on either side? Does it deadlock under load? Does either app not use the database indexs properly and exponentially slow under load?
etc
etc
etc
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Mon Apr 07, 2008 2:07 am Post subject: Re: Some strange suggestion during a problem |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
| ravishankarn wrote: |
We are not convinced!!! 1000 messages of 30K size - 30GB in total - will it slow down mq processing. |
If its persistenet messages.......chances are there
Regards
gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die
Last edited by Gaya3 on Mon Apr 07, 2008 2:48 am; edited 1 time in total |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Apr 07, 2008 2:39 am Post subject: Re: Some strange suggestion during a problem |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Gaya3 wrote: |
| ravishankarn wrote: |
We are not convinced!!! 1000 messages of 30K size - 30GB in total - will it slow down mq processing. Then we have difficulty to believe in the stability of the product .
|
If its persistenet messages.......chances are there
|
I'm assuming that what you mean here is that "persistent messages are slower than non-persistent messages", which is true of course.
IMHO it seems unlikely that the logging process adds sufficient overhead to the throughput to create the problems indicated, especially given the moderate volumes quoted. Also persistent messaging increases the stability of the product, not lessens it. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Mon Apr 07, 2008 2:47 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
There is no question of Stability in the matter of IBM MQ.
I forgot to clear out that part, my mistake
Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Apr 07, 2008 6:43 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
1000 x 30,000 = 30,000,000
30 MB, not GB.
This is nothing for MQ. Look at your logs to see what the channel errors are during the XMITQ backup. If there aren't any, then the CICS app could be putting the reply messages but not immedialty commiting them, which causes the XMITQ depth to rise.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Apr 07, 2008 6:51 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
1000 x 30,000 = 30,000,000
30 MB, not GB. |
Clearly there's more than one of us needing  this morning.
I fully agree - that's nothing.
I also agree that the points raised my mathmatically adept associate are easily contenders for my etcs!
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Apr 07, 2008 7:05 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| Vitor wrote: |
| Clearly there's more than one of us needing this morning. |
Except it's not *morning* for you...  
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Apr 07, 2008 7:14 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jefflowrey wrote: |
| Vitor wrote: |
| Clearly there's more than one of us needing this morning. |
Except it's not *morning* for you... |
Alright, alright, so I'm going through a rough patch.... 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
