| Author |
Message
|
| pbravic |
 Posted: Fri Jul 16, 2004 11:50 am Post subject: transferring very large messages from MQ to FTP |
 |
|
Apprentice
Joined: 24 Nov 2003
Posts: 42
|
hi all
as discussed in
http://www.mqseries.net/phpBB2/viewtopic.php?t=16357&highlight=mq+ftp
our requirment is to FTP the messages from MQ.
We are in the process of having a tool or bespoke development
In case of bespoke development, intially we thought of using a java wrapper arround the MQ-FTP client MA0P, When started with Proof of Concept we are facing input-stream problems and in java forums also doesn't recomend calling external program.
Mr jefflowrey gave some inputs, but I am looking for more info before doing Proof of Concept
Currently for bespoke development we were planning to use the MQ and FTP API combo.
Here some times the message sizes are arround 100 meg. Considering putting application will
segment the messages and put them in single unit of work
My further queries on bespoke development are
1) Does any one had similar kind of experience
2) For transferring very large segmented messages we need to group them first and uset the FTP APIs to transfer it to the respective FTP server
Am I going in right approach
For the tool based approach
If any one can suggest a best approach or the tool already available it would be great help to us
Thanks and Regards
Ravi |
|
| Back to top |
|
 |
| Michael Dag |
| Posted: Fri Jul 16, 2004 12:48 pm Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
There are a number of commercially available FTP over MQ tools.
Why re-invent the wheel?
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| pbravic |
| Posted: Fri Jul 16, 2004 10:16 pm Post subject: |
|
|
Apprentice
Joined: 24 Nov 2003
Posts: 42
|
Hi
Thanks we are having bespok as a option only
we have anlayzed the below tools
)XPressFTP - http://www.isisworks.net/
ii)SPAZIO FTOM / FTFI - Primeur Group
iii)CommerceQuest’s Data Integrator 4.0 - www.commercequest.com
Only SPAZIO is havin the tool nearest to requirement (i.e for tranfering messages from MQ to the respective FTP server)
If you are aware of better solution (tool ) , in the market pls let us know , we will look into that.
Your inputs will be more valuable to us.
Thanks and Regards
Ravi |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Sat Jul 17, 2004 1:51 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sat Jul 17, 2004 5:34 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Michael - That means that it can receive files over FTP and load them to a Queue.
It seems like from
| pbravic wrote: |
| (i.e for tranfering messages from MQ to the respective FTP server) |
that pbravic wants to extract files from MQ and transmit them over FTP.
This is a lot simpler to code. In fact, I think the FTPSend WMQI node support pack has included source code, and may be a good place to start. (But note, I believe this support pack has been withdrawn and is only available through the withdrawn page).
Also, I bet that IBM has an Adapter that will do this.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Sat Jul 17, 2004 8:43 am Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| jefflowrey wrote: |
Michael - That means that it can receive files over FTP and load them to a Queue.
It seems like from
| pbravic wrote: |
| (i.e for tranfering messages from MQ to the respective FTP server) |
that pbravic wants to extract files from MQ and transmit them over FTP.. |
the Q Liner tool also does the oposite, so
from using FTP client to FTP server/MQ and
MQ/FTPServer to xxx using FTP client
| jefflowrey wrote: |
This is a lot simpler to code. In fact, I think the FTPSend WMQI node support pack has included source code, and may be a good place to start. (But note, I believe this support pack has been withdrawn and is only available through the withdrawn page).
Also, I bet that IBM has an Adapter that will do this. |
possibly. the 'beauty' of Q Liner is that it is MQ/FTP integrated and not a 2 step process with an adapter.
What it looks like coming from any machine using FTP is,
you FTP to another machines directory, but the FTP server is not writing the data into a directory but straight onto a Queue.
the other way around as an MQ user you deliver the data to a Queue where the data 'sits' and the FTP server offers the data directly from the
Queue to any FTP client requesting the file.
I hope this made sense.
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sat Jul 17, 2004 10:12 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| MichaelDag wrote: |
| the Q Liner tool also does the oposite |
Ahh. You didn't say that first time around... 
| MichaelDag wrote: |
possibly. the 'beauty' of Q Liner is that it is MQ/FTP integrated and not a 2 step process with an adapter.
|
I don't necessarily think the IBM connectors/adapters are "two step processes".
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| pbravic |
| Posted: Mon Jul 19, 2004 3:44 am Post subject: very large message segmentation |
|
|
Apprentice
Joined: 24 Nov 2003
Posts: 42
|
Hi MichaelDag and jefflowrey
Thanks for your inputs, we will get in touch with MQ Software
apart from this thread it would be helpfull , if you were able to through some light on the below thinking as well. I need some pros and cons for the convincing my management
For bespoke develoment - mq ftp adapter
For large messages (100 mb +)
If the putting application segments the messagess and drops in to the queue. If the getting application waits for all the messages, after collecting then if the ftp process fails , I think we can't backout the messages as before (segmented)
from the application ref manual:
You cannot, however, put or get segmented messages in a global unit of work.
In total
When a transfer of very large messages (100 mb +) from MQ to FTP, if there is a process failure , if the segmentation is in place what will the status of message if ftp process fails?
Apart from that, the message buffer and application level performance will need to be taken care.
This may be a remote chance of going for bespoke coding , but I would like to know from the Masters what are the issues on segmentation for very large messages and issues on failure of these messages?
Thanks and Regards
Ravi |
|
| Back to top |
|
|
|
|
