| Author |
Message
|
| manojMQ |
 Posted: Tue May 29, 2012 11:53 am Post subject: Delay in message consumption from MQ while using MDB in WAS7 |
 |
|
Newbie
Joined: 29 May 2012
Posts: 4
|
Hello All,
I am new to this forum and relatively new to MQ. I have tried searching in various topics and googling but never got the answer for the topic.
I have an MQ Queue configured on a remote server and using an Activation Spec to read messages from MQ.
Now there is no problem with processing of the messages as such. But the time delay between the last printed timestamp ( at the end of OnMessage method in MDB) and the time at which the onMessage is called again on the same MDB is varying anywhere from 10ms-30 ms.
I have configured to use only one MDB instance per Queue configures as i need to preserver the order in which the messages are received and processed. So when i place 100 - 150 messages on MQ, the MDB timestamp shows the delay of 10-30ms between each message consumption.
I tried using Native MQ to connect to the same MQ queue and read & commit messages, in that case there is no delay. So obviously there is some configuration that could be done. Any help would be appreciated. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Tue May 29, 2012 6:20 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Have you tried doing a receivewait(1000), instead of an MDB?
What's the deal with the 10 to 20 ms? Is this a problem?
What kind of an application are you trying to build?
What is your transactional behavior?
Think about all this and let us know 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| manojMQ |
| Posted: Wed May 30, 2012 6:50 am Post subject: |
|
|
Newbie
Joined: 29 May 2012
Posts: 4
|
We are using this for a trade processing application where we need to process trades as soon as they arrive. 20-30ms is not acceptable for just waiting to fetch the messages while they are already available on MQ.
The Architect of the project already decided to go with MDB and hence we have designed to use MDB to read messages from MQ. Once the trade executions are replied back from stock exchanges to APPIA, APPIA ( fix engine) puts these messages on MQ from where MDB reads and processes the messages to be sent to CICS region.
We do not want process the messages in batch since some times the bunch of messages in single batch might belong to different clients and failure of one of them should not impact the other executions getting through to CICS. Please advise if there is anything we could do to fetch the messages with much lesser latency. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed May 30, 2012 7:03 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| Vitor |
| Posted: Wed May 30, 2012 7:10 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| manojMQ wrote: |
| We do not want process the messages in batch since some times the bunch of messages in single batch might belong to different clients and failure of one of them should not impact the other executions getting through to CICS. |
If the messages belong to different clients, why do you need to preserve message sequence as indicated in your original post?
Moving aside from the question of batch, I repeat the question about transactionality. How many messages form a unit of work? How wide is the unit of work?
MDB is more inefficient than native MQ because there's another layer involved but the difference (as you've proved) is marginal; 30ms as worst case latency is not that much especially in the kind of trade processing you're talking about. What's killing your throughput (and hence causing your latency problem) is that you're single threading and all the 30ms delays are adding up to a point where a message can be waiting for a second or so which is unacceptable. So again I ask; where's the affinity? Why is the MDB single threaded to preserve order?
An answer of "Because that's how the architect of the project decided it should be configured" is not acceptable in this context.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| manojMQ |
| Posted: Wed May 30, 2012 7:24 am Post subject: |
|
|
Newbie
Joined: 29 May 2012
Posts: 4
|
Hi Vitor,
Thank you for the explanation but here are the explanation:
1.There is not specified number of transactions per unit of work as in one case a client may order a batch of 100 transactions with 100 units each but in another case it may be just one transaction with any number of units of stock.
2. While processing the 100(or any number of) transactions of a single client there is a possibility that one of the transaction's might intend to cancel previous few transactions, hence we need to preserver the order of processing per client at any situation. So we designed only one MDB instance per MQ queue.
3. So as per your explanation the problem comes only when a number of messages comes in MQ for the same client.
It would be really helpful if there is any other configuration we could do to reduce the latency occuring withing single MDB to read from MQ. Or anyone could explain where is this delay occuring? |
|
| Back to top |
|
|
| manojMQ |
| Posted: Wed May 30, 2012 7:27 am Post subject: |
|
|
Newbie
Joined: 29 May 2012
Posts: 4
|
[quote="Vitor"]
If the messages belong to different clients, why do you need to preserve message sequence as indicated in your original post?
[/quote]
Vitor, We are using a specific hashing algorithm to determine which queue the message should be put in while the execution comes back from te FIX engine APPIA. Hence eventhough the messages for the same client would always be on the same queue, there is a high possibility that messages for some other client also would be on the same MQ queue. Since we do not want to configure separate MQ queue for every client we have, we have implemented in this way. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed May 30, 2012 7:46 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| manojMQ wrote: |
| 2. While processing the 100(or any number of) transactions of a single client there is a possibility that one of the transaction's might intend to cancel previous few transactions, hence we need to preserver the order of processing per client at any situation. So we designed only one MDB instance per MQ queue. |
So that group of transactions has affinity not the entire message stream.
| manojMQ wrote: |
| It would be really helpful if there is any other configuration we could do to reduce the latency occuring withing single MDB to read from MQ. |
Given the affinity requirement above, you can use the WMQ group facility to group the messages that need to stay in sequence, ensure those are processed in sequence while other groups are processed by other threads.
| manojMQ wrote: |
| Or anyone could explain where is this delay occuring? |
In the JMS layer your MDB is exploiting.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed May 30, 2012 7:48 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| manojMQ wrote: |
| Vitor, We are using a specific hashing algorithm to determine which queue the message should be put in while the execution comes back from te FIX engine APPIA. |
Why? What does that gain you? That the WMQ facilities don't?
| manojMQ wrote: |
| Hence eventhough the messages for the same client would always be on the same queue, there is a high possibility that messages for some other client also would be on the same MQ queue. Since we do not want to configure separate MQ queue for every client we have, we have implemented in this way. |
So put each client's transaction(s) in a given group. Accepting that a given client could have multiple groups through the day.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed May 30, 2012 5:51 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
And as a gentle reminder, you would should take into consideration that the 20 to 30 ms could well be the time needed for the commit...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
