| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| File Transfer (Yet Again) |
« View previous topic :: View next topic » |
| Author |
Message
|
| CodeCraft |
 Posted: Sun Feb 23, 2003 7:20 am Post subject: File Transfer (Yet Again) |
 |
|
Disciple
Joined: 05 Sep 2001
Posts: 195
|
All,
I've been reading various on file transfer using MQSeries. I am about to engage on a project in which FTP is being used to move data between an application on Ingres (Solaris) and an application on Progress (SCO).
I do not (that I know of) have ODBC as an option on these systems, nor, do I have an option for XA support to co-ordinate reads/updates on the respective databases and MQ.
Therefore, whatever way I approach the problem it will be a loosely coupled "connection", though more reliable than FTP and of course will introduce trickle feeding and proper abstraction of the current point to point link to provide a reusable/generic source service (via WMQI).
As part of the implementation I want to do the following:
1. Get a platform neutral method of shipping files between Solaris and SCO via MQ, so I assume Java utilites would be the best.
2. Assuming that, provide a Java "put" utility which will take the contents of a file and put it on a queue, and, a Java "get" utility which will take the contents of a queue and put it in a file.
3. Use triggers to kick off scripts on the source side to:
Extract data, write it to a file and call the Java "put" with the file.
4. Use triggers to kick off scripts on the target side to:
Call the Java "get", and then process the file contents (update target).
Questions/comments:
- Is there any severe critisms of the approach (I appreciate the need for some controls to confirm that the update actually happened, or not).
- Is there any Java utils already out there in the FreeWare world which already perform these functions. This would be like a Java implementation of IH02 but would also require a "get" partner.
- Any issues are Java/MQ Support on SCO?
- I also appreciate a Java programmer could possibly knock these up reasonably quickly but if it's out there already, I'd like to find out!
Thanks. |
|
| Back to top |
|
 |
| zpat |
| Posted: Mon Feb 24, 2003 12:18 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Why use files? Why not have the endpoint applications directly generate and consume messages as events occur. You don't have to have XA to do that (and you certainly can't have XA to flat files).
You can still use syncpoints without XA and should do so to protect data against loss. For example you might write from the file to the queue under a single syncpoint so that the messages "appear" on the queue at the same time and a triggerred process can read all of them.
Using files tends to build in a batch mentality with all the attendant problems of scheduling, avoiding processing the files twice or not at all and so on.
If you don't find Java samples (and can't write them) then you might consider using Perl code on these platforms. |
|
| Back to top |
|
|
| CodeCraft |
| Posted: Mon Feb 24, 2003 1:14 am Post subject: File Transfer, Why? |
|
|
Disciple
Joined: 05 Sep 2001
Posts: 195
|
The source and target applications are 4GLs which don't readily allow MQ enablement without much work. Ideally the event that would drive the MQPut would be the same event that performs an update in the source applications database. To do it right the database update and MQPut (or target MQGet and resultant update) would be done as one UOW on each side to guarantee consistency. This is not feasable in these environments and hence producing complex feed specific programs would not add much value.
The use of files in this case would not reflect a batch mentality - it would be merely a time driven (eg: every few seconds) on on the fly write to a temporary file followed by a call to a generic piece of code to move the data. The file in this case is the only easy way to pass data from the application to a program which can 'put' the data. It might also be possible to do this by streams/pipes without physical creation of the file, this has to be checked out.
What I really need is something which meets the requirement, not so much discussion on alternatives. There are a number of reasons why I don't want to look at alternatives (cost, systems replacements, etc). |
|
| Back to top |
|
|
| migstr |
| Posted: Mon Feb 24, 2003 6:35 pm Post subject: |
|
|
Apprentice
Joined: 05 Jun 2002
Posts: 34
Location: New York
|
|
| Back to top |
|
|
| zpat |
| Posted: Tue Feb 25, 2003 12:12 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
CommerceQuest sell a file transfer product that runs over MQSeries, should you really not want to code anything. This product has comprehensive scheduling, post-transfer execution of programs etc.
Getting a Java programmer to write a small program that monitors a file directory (there is a Java class to do this I understand) and places any files found on a queue in a single syncpoint would seem the best option.
The reason I mention a single syncpoint, is that if you trigger the target program, there is a risk it will empty the queue before all the records have arrived. Using a single syncpoint for the MQPUTs avoids that risk. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Feb 25, 2003 4:55 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
