| Author |
Message
|
| ganesh3 |
 Posted: Tue Apr 04, 2006 12:34 am Post subject: Sending a file as MQ message and reading a file from MQ. |
 |
|
Apprentice
Joined: 30 Mar 2006
Posts: 35
|
Hi,
Is there a way to send a file as MQ message. Kindly let me know if it is possible, if it is not kindly let me know certain solutions which i can use. I am using IBM MQSeries 5.2 and file being sent is an XML file. Also kindly let m eknow if reading an xml file from the queue is possible.
_________________
Have a Good Day!!! |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Apr 04, 2006 12:47 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Okay:
There are a number of boxed solutions you can buy that send files over queues. PM4Data by Commercequest/Metastorm is one (and available via IBM I think), but only one and I offer it as an example only without preduice. Other here will undoubtably know of others.
But:
Do you mean an XML file as in "test.xml", ie. a single document? Why not just send the content as a normal message in the normal way? It's only text.
Finally:
5.2 is a bit long in the tooth these days. Give some thought to upgrading. Your life will be made easier.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| ganesh3 |
| Posted: Tue Apr 04, 2006 1:03 am Post subject: |
|
|
Apprentice
Joined: 30 Mar 2006
Posts: 35
|
Hi,
Thanks for the reply. Yes, i need to upload an xml file as in "test.xml" and will be using java(JMS) code to perform the same. Using TextMessages by converting the data into string and adding it to stringbuffer or an array would imply more manipulation of data and also i will be creating the xml file dynamically and uploading it and deleting it, converting it into normal messages(TextMessage, ObjectMessage, StreamMessage,..,...) will be an overhead. Kindly let me know what is PM4DATA, is it an application software or some code which is used to upload a file.
_________________
Have a Good Day!!! |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 04, 2006 1:34 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
For your requirement, PM4Data can best be described as "a sledgehammer to crack a broken nut". It's a software solution produced by the company named & sold both direct & via IBM, designed as a replacement for FTP and utilising MQ functionality for the data transfer rather than native TCP/IP.
Unless you're moving large XML documents or thousands of them I would urge you to get a quote from the vendors before you rule out JMS. It (in common with most commerical solutions) is not exactly cheap and you really don't sound like you need all the extra features it would bring.
I'm not a big JMS / Java guy but what you're describing doesn't sound like a lot of overhead. This site does exactly that - dynamically create XML requests with JMS, send them via MQ, recieve an XML reply and read it & they roll their own JMS.
Just my observation. Clearly you know your own situation best.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| ganesh3 |
| Posted: Tue Apr 04, 2006 1:44 am Post subject: |
|
|
Apprentice
Joined: 30 Mar 2006
Posts: 35
|
Hi,
If it is the case, that this site creates an xml and sends a request, is it possible to confirm that the messages being sent to MQ are infact TextMessages or stream messages or byte or object messages and not the whole xml file as a message. Kindly let me know. Yes i need to upload only two files but the conversion to stringbuffer or array will cause some delay as the file has to be sent as sooon as possible.
_________________
Have a Good Day!!! |
|
| Back to top |
|
|
| wschutz |
| Posted: Tue Apr 04, 2006 2:33 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 04, 2006 2:37 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Accept before I start that I am not now nor have I ever been a Java programmer, so I may have been asking the wrong questions or misquoting the answers. With luck, other members more skilled will correct me. 
Given that, each request / reply is a distinct XML document, stateless and complete in it's own regard. They range from a couple of lines for some of the requests to over 500Mb (!) for some of the replies. They're generated either from manual intervention on systems (e.g. user keying a request) or database triggers. Volume is in the thousands per minute range.
All the Java uses JMS to communicate with MQ. The document is parsed with JAXB (?) and send as MQSTR though my broker for transformation and routing, using the CONVERT option on the get to deal with code page issues.
Like I said, there's a high volume and many of these are servicing live users at a screen who won't wait much more than a second or so for a response.
For my information and enlightenment, you say "conversion to stringbuffer or array will cause some delay". Do you have timings on that? I'm trying to improve my Java from it's currently low level & would like to avoid any problem code structures.
Thx
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 04, 2006 2:39 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| wschutz wrote: |
| Vitor wrote: |
This site does exactly that - dynamically create XML requests with JMS, send them via MQ, recieve an XML reply and read it & they roll their own JMS.
|
Do you mean mqseries.net?
|
Apologies no - the site at which my day job is currently located.
Language skills not good at the moment!!!
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| ganesh3 |
| Posted: Tue Apr 04, 2006 3:00 am Post subject: |
|
|
Apprentice
Joined: 30 Mar 2006
Posts: 35
|
Hi,
Thanks for the replies. Yes, the stringbuffer and arrays will take time and no time is set on them but moving data from file to stringbuffer or array does take time. I have not got answers to my query as yet. I am sorry if am sounding harsh. It would be great if some java or jms expert does answer my query.
_________________
Have a Good Day!!! |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Apr 04, 2006 3:10 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ganesh3 wrote: |
Hi,
Thanks for the replies. Yes, the stringbuffer and arrays will take time and no time is set on them but moving data from file to stringbuffer or array does take time. I have not got answers to my query as yet. I am sorry if am sounding harsh. It would be great if some java or jms expert does answer my query. |
We're all just trying to get to a solution.
I meant in a general Java sense how expensive (in terms of time & CPU) is the conversion to stringbuffer or array? It's a little off topic I admit, but I'm interested to know. Certainly the experience here is that it's fast enough for production (where, as I should have made clear before, "here" is the site I'm working at not mqseries.net!
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Apr 04, 2006 3:15 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Can you explain the difference between a whole XML file and the XML data as a TextMessage?
They are really the same thing. A file is just a big long string of data - and a message is just a big long string of data. An XML file is a big long string of Text data, and a TextMessage is a big long string of test data.
You can not put the contents of the file into an MQ message of any sort - using Java, JMS or any other language - without reading the entire contents of the file and then adding that to the message.
And it will always take some time. If you use a standard web 'file upload' form, then what happens is the contents of the file is read and then written into the HTTP Request message which is sent to the server - and it always takes time. It always takes time to move files from one place to another, because the data always has to be read from the file and written to the new file.
There is no method in the MQ API for Java or in JMS for adding a file object to a message - even if there was, all it would do is read the file and write it to the message. So you will have to write this code yourself.
Likewise, there is no method to turn a message object into a file. So, again, you will have to write this code yourself.
On the other hand, I know that several people over the years have posted sample code that does just this to this forum right here on MQSeries.net. So you can search for something like "java.io" and see what shows up.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Apr 04, 2006 12:24 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Quote: |
| Yes i need to upload only two files but the conversion to stringbuffer or array will cause some delay as the file has to be sent as sooon as possible |
This will take a lot less time than to get the output from the DB / Request in java, stream it to a file, read it from the file and put it into MQ /JMS.
You need to look at your provider an stream the data directly into the TextMessage and send it over. Remember to make your sessions transacted if needed.
I would not sent any message greater than 4MB but segment using JMS grouping and have a program at the other end that will hande reassembly.
(check if grouping flag is set and get all messages with correlid = message id of first message... Now if the message is not the first you need to have some logic about how to get the messages before the one you hit.) (best bet is to put the fragments to a db and check periodically until the set is complete, then extract it and assemble it.)
And be realistic. If the volume for transfer is 500 MB + you WILL NOT be able to transfer it over the Internet in less than a second...(overhead + lag time ++++)
If you need fast, reliable, auditable file transfer over MQ get PM4DATA
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
