MQSeries.net :: View topic - Coordination of file handling with End Applications

MB · Posted: Wed Dec 07, 2005 1:02 am Post subject:

Hi,

A question about some Basics (or Best Practices!?) of Designing an interface involving Files in between...

We (Middleware) have WMQI v2.1 Broker followed by a JText adapter to put messages as records in a Text file. The Destination Application (Dest-App) has to process these Text files once in a day. Now, the concerns are like:

Middleware to signal the Dest-App to start:
1. If the Adapter writes to the files by the time Dest-App starts processing the file (two applications simultaneously accessing the files... one for Reading and one for Writing) the Dest-App might not see some data. For this, we thought the Source Application (Src-App) will send an End-of-Batch message towards the end; since the adapter is processing the messages in sequence, it will process the End-of-Batch towards the end and put a zero byte 'Signal1' file in the destination folder. The Dest-App keeps looking for this 'Signal1' file; once it is present, it starts processing the actual files. Once the processing is over successfully, Dest-App deletes the 'Signal1' file from the folder.
Is this fine or is there a better option?

Dest-App to signal the Middleware to Go-Ahead:
2. Now the question is other way round. While the Dest-App is still processing the Text files of the previous day, if the source application starts sending messages, the Broker and JText Adapter will process these messages and try to put to the destination Files. But since the Dest-App is still reading the files, can we stop the Middleware from writing to those files?
(a) First of all, is it the responsibility of Src-App or the Middelware to hold without sending messages?
(b) For some reasons, if the Src-App cannot be modified, how can this be handled in Middleware?
Can the Adapter check if some file is 'already open for reading' before it opens for writing?
(c) Another alternative: Dest-App says they will put a zero byte file 'Signal2' once their processing is successfully completed and delete it when they start processing (or the vice-versa). So, Middleware has to write to the files or hold from writing based on the existence of 'Signal2' file. Can the Adapter check the 'existence' of a file and act based on that?
(d) If the Adapter is able to make such decisions, for the duration it is waiting for 'Signal2', what should it do with the incoming flow of messages from the Src-App? Where shall it hold them?

(I feel this must be an old question and someone would already have answers for this. I browsed the forum but could not get any clue; May be I did not get the right search string/pattern!)

Thanks and Regards,
MB