| Author |
Message
|
| marcin.kasinski |
 Posted: Mon Dec 04, 2006 1:59 am Post subject: MQ Application design consideration |
 |
|
Sentinel
Joined: 21 Dec 2004
Posts: 850
Location: Poland / Warsaw
|
Hi,
We have little problem,
Some of our applications send dynamic data via MQ which reach max message size.
We don’t want to change this attribute to solve this problem but we are considering use of message segmentation.
We have a lot of adapters written in C, JAVA and JMS.
Problem is that segmentation is not implemented in JMS, so we have to in JMS adapter combine segmented message.
Is there other way to solve this “JMS segmentation problem” ?
Best regards
Marcin |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Mon Dec 04, 2006 3:07 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
So what you're saying is that you would rather change many many programs to implement segmentation, and write a NEW program to handle the "JMS segmentation problem", instead of implementing a relatively simple configuration change on your MQ network?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| marcin.kasinski |
| Posted: Mon Dec 04, 2006 3:38 am Post subject: |
|
|
Sentinel
Joined: 21 Dec 2004
Posts: 850
Location: Poland / Warsaw
|
| jefflowrey wrote: |
| So what you're saying is that you would rather change many many programs to implement segmentation, and write a NEW program to handle the "JMS segmentation problem", instead of implementing a relatively simple configuration change on your MQ network? |
Yes,
At first we would like to change 4 adapters which sent messages reach max message size.
Then in new release of other applications we are going to implement segmentation.
But we still have this “JMS segmentation problem”.
I know that we can make very simple configuration change but we would like to optimize resources. MQ will lock resources for this new max message size.
We have tens of QMs and we will have to do it for all of them.
Another thing is future. With segmentation managed by QM message length is let’s say is not important from application point of view. With configuration change which you suggest and with gradually increasing message length I can have the same problem (reach max message length) in 1 or 2 year.
Of course these are only plans and I’m looking for advice.
Best Regards
Marcin |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Dec 04, 2006 3:51 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| marcin.kasinski wrote: |
| MQ will lock resources for this new max message size. |
What resources do you believe will be locked? Have you noticed this as a resource issue?
| marcin.kasinski wrote: |
| We have tens of QMs and we will have to do it for all of them. |
This kind of change can be scripted.
| marcin.kasinski wrote: |
| with gradually increasing message length I can have the same problem (reach max message length) in 1 or 2 year. |
Don't forget the maximum message length supported by MQ has increased drastically in recent releases, and can reasonably be expected to increase in the future. You don't mention if you're using client code or server / bindings code, the limits on which are different.
My 2 cents.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| marcin.kasinski |
| Posted: Mon Dec 04, 2006 4:28 am Post subject: |
|
|
Sentinel
Joined: 21 Dec 2004
Posts: 850
Location: Poland / Warsaw
|
| Vitor wrote: |
Don't forget the maximum message length supported by MQ has increased drastically in recent releases, and can reasonably be expected to increase in the future. You don't mention if you're using client code or server / bindings code, the limits on which are different.
My 2 cents. |
So you are saying is that I can set max message size attribute to max system supported value and forget about this.
Why this parameter exist in MQ ? There should be reason
Is there any performance or another impact of this attribute ?
Most of our applications use client code.
We rarely use bindings mode.
Best Regards
Marcin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Dec 04, 2006 4:35 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| Vitor wrote: |
| Don't forget the maximum message length supported by MQ has increased drastically in recent releases, and can reasonably be expected to increase in the future. |
It has been 104857600 since version 5.0.
There are no announced plans that I am aware of to change that, at all.
| marcin.kasinski wrote: |
| Another thing is future. With segmentation managed by QM message length is let’s say is not important from application point of view. With configuration change which you suggest and with gradually increasing message length I can have the same problem (reach max message length) in 1 or 2 year. |
Change everything to support the maximum message length of 100MB.
If you still reach that in two years, *then* you can think about segmentation. And in the meantime, you can *keep* thinking about segmentation, and implement it in new code where appropriate.
Also, JMS may not support automatic segmentation, but I believe it supports Grouping.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Dec 04, 2006 4:41 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| marcin.kasinski wrote: |
Why this parameter exist in MQ ? There should be reason
Is there any performance or another impact of this attribute ?
|
It exists to provide flexibility. For instance - if you know that the largest message going through your system is 30Mb and you want to prevent a looping program causing larger messages to be sent, then the limit can be artifically set lower to cause such messages to be rejected. Other uses limited by the imagination of the administrator. 
Where it comes into play is with log sizes & queue storage. The larger the messages the more space needed to hold them. No matter how you resolve your problem, you'll need space and a logging strategy that can cope with the message size you're using. 60Mb of message data is 60Mb of message data in one message or several segments.
And I still think this would be easier than trying to set JMS to use segmentation. Again, that's an opinion. Other views are equally valid.
| marcin.kasinski wrote: |
Most of our applications use client code.
We rarely use bindings mode.
|
Why? Bindings offers performance advantages over client mode (at the smplest level reduced network usage) and the use of UOW - handy with all that dynamic data flowing round I'd have thought.
Be aware that client connections have a lower maximum message size - this may well restrict your choice of options.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Dec 04, 2006 4:49 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jefflowrey wrote: |
It has been 104857600 since version 5.0.
|
Even client connections? 
I turn my back for a few years..... 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Dec 04, 2006 4:57 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
The MaxMsgLn parameter for channels of type CLNTCONN has had a maximum value of 104857600.
You have generally had to use either a client table or MQCONNX to use that, however, as the default client connection you get from using MQSERVER was (and I believe still is) still using a MaxMsgLn of 4 mb.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Dec 04, 2006 5:03 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Ah - so only slightly mistaken - I knew the 4Mb still figured in clients somewhere.
My messages tend to be XML that seldom troubles the 4Mb mark, so the question does not often come up for me.
Again MQSeries.net delivers the knowledge!
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Dec 04, 2006 5:11 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Marcin -
Yes, the current practice I recommend is to configure the entire MQ network to support the maximum 104857600, and then forget about it.
There are relatively few types of business data that should generally be larger than 4 MBs - and it's always better with messaging to package data into the smallest logical piece of work.
The cases where people need to ship larger messages tend to be where they are shipping batches of records, or they are trying to move files (which are really batches of records) or they are working with image data.
If you're not doing file transfer or moving images, you should probably look at ways to work with smaller logical messages, rather than looking at segmentation or grouping.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
