| Author |
Message
|
| garimella |
 Posted: Tue Jan 31, 2006 10:48 am Post subject: Getting more than one message from queue using get method |
 |
|
Newbie
Joined: 31 Jan 2006
Posts: 5
|
G'day all,
We are using MQSeries to establish asynchronous communication between java clients and c++ servers. I am looping till reasonCode is MQRC_NO_MSG_AVAILABLE using c++ api to retrieve messages one by one from the queue.
My question is, is there any option I can use to get more than 1 message per get call instead of 1 message per get call?
Thank you,
Sridhar. |
|
| Back to top |
|
 |
| HubertKleinmanns |
| Posted: Tue Jan 31, 2006 10:52 am Post subject: |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
Sridhar,
messages are always read one by one, but you are able, to use syncpointing, to read more than one message within a unit of work. This means you still read the messages one by one, but you commit (or backout) all messages together.
Is this, what you want?
Hubert |
|
| Back to top |
|
|
| garimella |
| Posted: Tue Jan 31, 2006 11:08 am Post subject: |
|
|
Newbie
Joined: 31 Jan 2006
Posts: 5
|
Hubert,
Thank you very much for your quick reply. What I want is, getting messages in batch mode. I mean I should get an array of messages with one get call based on some type of configuration setting.
For example if I configure batch size as n. Then get call should retrieve n messages( if queue contains >=n messages ) if I pass empty array Message[] so that I can loop this array. This will reduce no of get calls to queue.
Thank you,
Sridhar. |
|
| Back to top |
|
|
| vennela |
| Posted: Tue Jan 31, 2006 11:34 am Post subject: |
|
|
Jedi Knight
Joined: 11 Aug 2002
Posts: 4055
Location: Hyderabad, India
|
|
| Back to top |
|
|
| HubertKleinmanns |
| Posted: Tue Jan 31, 2006 11:48 am Post subject: |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
| garimella wrote: |
Hubert,
Thank you very much for your quick reply. What I want is, getting messages in batch mode. I mean I should get an array of messages with one get call based on some type of configuration setting.
For example if I configure batch size as n. Then get call should retrieve n messages( if queue contains >=n messages ) if I pass empty array Message[] so that I can loop this array. This will reduce no of get calls to queue.
Thank you,
Sridhar. |
I guess, you mean something like batch size in channel definitions. Channels are normal MQ applications, which read messages one by one under syncpoint control (as I described before). They commit or backout the whole batch as a unit.
There is not batch option for the MQGET call.
Hubert |
|
| Back to top |
|
|
| mvic |
| Posted: Tue Jan 31, 2006 12:24 pm Post subject: Re: Getting more than one message from queue using get metho |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| garimella wrote: |
I am looping till reasonCode is MQRC_NO_MSG_AVAILABLE using c++ api to retrieve messages one by one from the queue.
My question is, is there any option I can use to get more than 1 message per get call |
Is your C++ app remote from your server, or on the same machine? And, about your messages, how big are they, and are any / all of them persistent? |
|
| Back to top |
|
|
| garimella |
| Posted: Wed Feb 01, 2006 1:23 am Post subject: |
|
|
Newbie
Joined: 31 Jan 2006
Posts: 5
|
mvic,
Our c++ app is local to server. At present the message size is 10kb and it can grow to 25kb max in future. All messages are persistent. We cant afford to lose any message if there is any queue manager restart as its a trading system.
Thank you,
Sridhar. |
|
| Back to top |
|
|
| mvic |
| Posted: Wed Feb 01, 2006 2:45 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| garimella wrote: |
| Our c++ app is local to server. At present the message size is 10kb and it can grow to 25kb max in future. All messages are persistent. We cant afford to lose any message if there is any queue manager restart as its a trading system. |
Thanks for the info. Certainly there is a theoretical advantage to batching in a server-bound app, but in practice I think the benefit would be tiny. This is especially true in the case of persistent messages.
For persistent messages in the server-bound case, the majority of time taken by an MQGET is spent fetching and writing data to disk. This is potentially milliseconds, depending of course on I/O and disk speed / seek times. The in-memory operations involved in making and servicing the API call itself will be negligible in comparison.
If performance is critical, and the application design allows it (ie. each message can be processed entirely independently from the others) then one technique for speeding things up is to have more copies of the app getting from the queue and processing the contents. But I don't know your application design so this may not be appropriate.
EDIT: clarify 2nd paragraph.
EDIT: clarify 2nd paragraph (again).
Last edited by mvic on Wed Feb 01, 2006 3:02 am; edited 2 times in total |
|
| Back to top |
|
|
| wschutz |
| Posted: Wed Feb 01, 2006 2:46 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
Why do you need this? Granted, it would be a nice feature (which you could implement as a class), but aside from that, whats the need? Are there performance problems?
_________________
-wayne |
|
| Back to top |
|
|
| HubertKleinmanns |
| Posted: Wed Feb 01, 2006 2:57 am Post subject: |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
Another possible solution may be performed by the applications. The sender app could concatenate several messages and put them as one MQ message to the queue. You will have some more administration data in the message, to split the data on the receiving side, but it would increase the transport of the messages via network a little (because you will have only one MQ-MD of about 400 bytes instead of 10, 100 or what you want). If you use a very slow network (and if your sending application does the concatenation), this could increase the speed.
Hubert |
|
| Back to top |
|
|
| garimella |
| Posted: Wed Feb 01, 2006 9:12 am Post subject: |
|
|
Newbie
Joined: 31 Jan 2006
Posts: 5
|
mvic,
Thanks for your explanation. We have already designed the app to have multiple processes(not multiple threads due to our legacy system constraint. So many shared resources ) to get messages from same queue. I thought of using this option if its available.
wschutz,
I dont have any performance stats as we are still in the implementation stage. Sure I will update once I have stats.
Hubert,
Its good idea to concatenate several messages and put them as one MQ message.
Thank you,
Sridhar. |
|
| Back to top |
|
|
| wschutz |
| Posted: Wed Feb 01, 2006 9:37 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
| Quote: |
| Its good idea to concatenate several messages and put them as one MQ message. |
Allow me to offer a differing opinion:
Unless there is a good reason to do this (and usually performance is the reason cited), doing this makes your applications more complex and difficult to maintain. You need to develop a method for indicating how many logical messages you've packaged into a physical message.
If you want to start sending messages to an application that handles "only 1 message at a time", then that application needs to change. If you want to insert a broker function (transformation, pub/sub), that becomes more complex.
If you want to use clustering to do workload balancing, you might impact the ability to effectively use that (ie, if you batch 20 logical messages to a physical message, and you only generate one phyiscal message, all 20 logical messages go to the same queue manager)....
So, before adding that complexing in, just make sure you really need it. 
_________________
-wayne |
|
| Back to top |
|
|
|
|
