| Author |
Message
|
| Steve Tudor |
 Posted: Wed Oct 19, 2005 7:42 am Post subject: Large Reference Messages |
 |
|
Newbie
Joined: 10 Oct 2005
Posts: 7
|
We are currently trying to use Websphere MQ to move a large number of image files from one server to another.
We have implemented a Reference Message solution using C# and everything is working just fine.
The question I would like to ask is this - We are considering zipping all of the images from a single directory into a single zip file and transferring this one file, rather than transfer 1000 or so separate image files - The thing is that this file will be between 200 and 500 MB in size (yes up to half a gig). Is this a really bad idea, or do you think that MQ will take this in it's stride with no problems at all.
We will obviously be running some tests of our own but won't have access to the live environment for quite some time, so I was wondering if you guys had any experience or opinions on this? |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Wed Oct 19, 2005 7:46 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
If you're using reference messages (if I understand reference messaging correctly), then MQ doesn't care about the size of the file being moved. All you're actually moving via MQ is information about where the file is and where it will end up.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Steve Tudor |
| Posted: Wed Oct 19, 2005 7:58 am Post subject: |
|
|
Newbie
Joined: 10 Oct 2005
Posts: 7
|
Jeff
Thanks for this.
Just one small correction, the files themselves are actually being moved via MQ as well.
If you have a look at the reference messaging examples that are supplied with MQ the amqsxrm.dll MsgExit is responsible for chopping the file up into pieces sending them across MQ as messages and reassembling them at the other end.
Does this new information change your opinion?
Steve |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 19, 2005 8:07 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
No, it doesn't change my opinion. I haven't looked at the exit, though. A quick review to establish that it doesn't care about file size might be helpful for you.
And you might want to review it's transactional behavior, as well, to ensure that it doesn't attempt to put all of the "sub" messages into the same transaction.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Steve Tudor |
| Posted: Wed Oct 19, 2005 8:12 am Post subject: |
|
|
Newbie
Joined: 10 Oct 2005
Posts: 7
|
| Quote: |
And you might want to review it's transactional behavior, as well, to ensure that it doesn't attempt to put all of the "sub" messages into the same transaction
|
I would be interested in understanding why you make this suggestion.
To a newbie it would seem that I would want all of the sub messages in the same transaction. The sub messages contain portions of a file which are no good unless they are ALL recevied. Isn't this an ideal demonstration of when to use a transaction, we either want all sub messages or none of them.
Am I missing something??? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 19, 2005 8:27 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Well, it's not as important if they aren't persistant messages, necessarily.
But transactions use up space in your transaction logs... and a 1/2 gig transaction is very big for MQ transaction logs....
And you really don't need all these messages in a transaction, and potentially not in a transaction at all!
You just need good solid communication between both ends of the exit, to ensure that you can receive all pieces, resend any lost pieces, and that the file on the sending side is not deleted until necessary. And MQ is good solid communication... 
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
