| Author |
Message
|
| anveshita |
 Posted: Wed Nov 09, 2011 9:33 am Post subject: Queue size considerations |
 |
|
Master
Joined: 27 Sep 2004
Posts: 254
Location: Jambudweepam
|
We have vendor product that creates an object and loads the object to queue. Whenever the object size is large, it fails with a MQ error. I think the code is written in C# and the object contains the tags for eg: firstname xyz,lastname abc similar to it. I find passing the tag name is a redundant exercise, when the program reading the queue know the object. This for sure reduces the size of the message. We have increased the size to 20 MB for now, what will we have to do when we cross 100MB.
Any thoughts? Please let me know |
|
| Back to top |
|
 |
| Vitor |
| Posted: Wed Nov 09, 2011 9:42 am Post subject: Re: Queue size considerations |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| anveshita wrote: |
what will we have to do when we cross 100MB.
|
Get the vendor to use segmentation? And, obviously, modify the receiving app accordingly.
Another alternative would be to see if the 100Mb message can be logically split into a number of business relevant messages, e.g. one message per customer transaction.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Nov 09, 2011 9:56 am Post subject: Re: Queue size considerations |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Vitor wrote: |
| ...Get the vendor to use segmentation? And, obviously, modify the receiving app accordingly... |
So long as he's not on z/OS...
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Nov 09, 2011 10:04 am Post subject: Re: Queue size considerations |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| exerk wrote: |
| Vitor wrote: |
| ...Get the vendor to use segmentation? And, obviously, modify the receiving app accordingly... |
So long as he's not on z/OS... |
If the OP is running C# on z/OS that's it's own can of worms.
And if he is on z/OS trying to move 100Mb+ of data round the site sys progs will offer alternatives. Ever wonder why z/OS doesn't have segmentation? 
It sounds a lot like the file is just a dump of something and will stand being split. I find it hard to believe anything which starts with a last name and a first name then goes on for 100Mb (or 20Mb) of pure data related to the guy with that name, which can't be segmented and/or grouped.
XML you can hit 20Mb+ without much trouble. But that's because you can't remove the tag names as has been done here. Self describing data comes at a price. 
_________________
Honesty is the best policy.
Insanity is the best defence.
Last edited by Vitor on Wed Nov 09, 2011 10:05 am; edited 1 time in total |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Nov 09, 2011 10:05 am Post subject: Re: Queue size considerations |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| anveshita wrote: |
...what will we have to do when we cross 100MB.
Any thoughts? Please let me know |
100MB is the maximum physical message size. (Note that I did not say "logical message size.")
If you can't segment the message(s), then consider compressing the application data portion of the message so it's <100MB.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Nov 09, 2011 10:06 am Post subject: Re: Queue size considerations |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Vitor wrote: |
| If the OP is running C# on z/OS that's it's own can of worms. |
It's been a long week already 
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| anveshita |
| Posted: Wed Nov 09, 2011 10:10 am Post subject: Re: Queue size considerations |
|
|
Master
Joined: 27 Sep 2004
Posts: 254
Location: Jambudweepam
|
| exerk wrote: |
| Vitor wrote: |
| ...Get the vendor to use segmentation? And, obviously, modify the receiving app accordingly... |
So long as he's not on z/OS... |
While the above options are there, I am not sure what is the purpose of using the tags as part of the message if the receiver already know the format in which to expect the message. ( I find I have the same thing against XML being slow).. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Nov 09, 2011 10:18 am Post subject: Re: Queue size considerations |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| anveshita wrote: |
| While the above options are there, I am not sure what is the purpose of using the tags as part of the message if the receiver already know the format in which to expect the message. |
This assumes that all the tagged fields exist (or missing fields are delimited by a spare comma) and appear in the same order each time. It's clearly possible to create such a file (you've obviously done it) but it limits flexibility.
Note that WMB comes with a MRM parser specifically to handle such tagged fields. This is how common the format is.
| anveshita wrote: |
| ( I find I have the same thing against XML being slow).. |
XML is not slow. XML is large. XML parsers can be slow, but most (like the one under the XMLNSC domain) are not. In today's wired world big XML documents are not the cumbersome format they might have been 20 or even 10 years ago. It also has the advantage you don't need to know the format to interpret it; wonderful if you're trying to find the sender of a message that's arrived in a DLQ.
Also, web service architecture is here to stay. Ay least until the Next Big Thing. So learn to love XML and juggle your WSDLs.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Nov 09, 2011 10:23 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Message grouping? We've just used that on a new MQ and WMB project and works well. |
|
| Back to top |
|
|
|
|
