ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Process one big zip file by WMB

Post new topic  Reply to topic
 Process one big zip file by WMB « View previous topic :: View next topic » 
Author Message
KoGor
PostPosted: Mon Dec 26, 2016 8:16 am    Post subject: Process one big zip file by WMB Reply with quote

Voyager

Joined: 09 Nov 2005
Posts: 81
Location: Moscow,Russia.

Hi all!

I've quite a challenge task (for me) to solve with a WMB. There is an automated system that send a large zip file to WMB by WMQ (MQ Group used). Inside this zip there are routing info and description of files in archive. [img]http://take.ms/hn7sj[/img]

Route.xml contains sender and reciepient addresses.
Envelope.xml contains descripition of attached files (size, properies and etc)
Attached files can be also zip container with envelope.xml but we don't extract them, they are just files to WMB.

Attached file can be large - about 2-4GB (scanned documents).

Tasks for WMB:

1) get a MQ message (one big Zip file ) from one queue;
2) Extract only Route.xml and Envelope.xml;
3) Find routes in DB based on addresses from Route.xml
4) Put a message to DB
5) Propagate a message to several queues.

I have sugested using MIME as a container as soon as it supported by WMB instead of ZIP archive. With a MIME also there is a way to send binary data without encoding. But customer don't want to use MIME and want to use ZIP.

So the question is how to handle zip file inside message flow. Is there way to extract only route an envelope.xml and do not load whole message to memmory? May be there is a better way to handle such large files with a broker? Anyone had solved a similar problem with a large files in a broker?

Also I 've found information that it's not possible to handle more than 2GB message in WMB.

http://www.ibm.com/developerworks/websphere/library/techarticles/1601_crighton-trs/1601_crighton.html

Thank you in advance!
Back to top
View user's profile Send private message
mqjeff
PostPosted: Wed Dec 28, 2016 1:46 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

WMB/IIB does support MIME documents using the MIME parser.

First you have to determine if you can access some part of the zip file before you have loaded it all - i.e. can you extract the route file before you have the complete zip file.

Also, note that the developerworks article you are discussing is using the SOAP/MIME parser.

I suspect you can handle much larger BLOB data, since it doesn't have a parser associated.

You should try very hard to get the customer to put the zip file somewhere, send a trigger message of some kind to the flow, and then unzip the file on the file system, and then use FileRead to get the pieces you need.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Process one big zip file by WMB
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.