MQSeries.net :: View topic - Design question about File to Queue process

Scooter · Posted: Thu Jun 09, 2005 1:28 pm Post subject:

We have been using Websphere MQ for over a year to process message on a transactional base. That is we put a message on a remote queue that triggers a mainframe transactions to process it. I have now been asked to come up with a design that will transfer a file from location to the next by using Websphere MQ. Basically design an alterative to FTP. It sounds simple to them but Iâ€™m having a nightmare determining the best approach for this. I hope someone can help point me in the direction for the best approach to this issue. Good news is that the design only needs to work within a Unix (AIX) environment on both ends.

One of the approaches I have come up with would be to read the file and put each record into its own message. To handle large files I would group the message into Units of work and commit each unit of work as it completes. These units of work would be put on a remote queue that would transfer the units of work to a local queue on mainframe. With this approach I would need a way to trigger the mainframe program to start processing the units of work before all the units of work are in the local queue. This would be needed because we have to indicate the message limit on the local queue and since the file being transferred could be larger then the queue limit. This is where things start to get fuzzy for me. If the sending program has any problems and doesnâ€™t completely convert the file to units of work and needs to be restarted then the receiving program will need to stop looking for more units of work until sender is restarted. Would this be a spot to use a secondary queue to relay control messages like restarts and new files messages?

The second approach would be the same as the first with the exception of using dynamic local queues that could be created to hold the size of the file that is being transferred. Each file would have its own queue. Problem with this is Iâ€™m not sure I can create dynamic local queues when using a remote queue to send the messages to another server. This approach is a lot more foreign to me but if it is a better approach then I will learn.

I know this is an odd post but Iâ€™m completely lost and not even sure either of these is the best approach to the problem. Heck, I may be so far off base that it canâ€™t even be done. I appreciate any input you guys have.