| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| [solved]transactionality of FileOutput mqsitransit directory |
« View previous topic :: View next topic » |
| Author |
Message
|
| j.f.sorge |
 Posted: Tue Oct 29, 2013 2:59 am Post subject: [solved]transactionality of FileOutput mqsitransit directory |
 |
|
Master
Joined: 27 Feb 2008
Posts: 218
|
I'm using a ComputeNode which will do some mapping, sending the mapped data to a FileOutput node configured to "stage within mqsitransit and move on Finish File". When the complete execution groups abends the temporary file will stay within the mqsitransit directory. After flow restarts the processing, the temporary file won't be deleted. New records will be appended to that temporary file.
Is there any chance to let the temp file being deleted on abend?
If that could not be done, I'll have to map into a row and sent out that data at once which will cause a very big message in memory to be created.
Could that have performance issues in combination with the FileOutput node?
_________________
IBM Certified Solution Designer - WebSphere MQ V6.0
IBM Certified Solution Developer - WebSphere Message Broker V6.0
IBM Certified Solution Developer - WebSphere Message Broker V6.1
IBM Certified Solution Developer - WebSphere Message Broker V7.0
Last edited by j.f.sorge on Tue Oct 29, 2013 4:26 am; edited 1 time in total |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Oct 29, 2013 3:57 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Well, you couldn't have it be deleted on *abend*, because I mean... it's an abnormal end...
The question is, are you able to tell at restart that you need to finish the file?
The second question is, why are you *planning* for an abend... You should be planning for normal error conditions, and planning to make manual efforts to resolve extraordinary error conditions. And an abend should always be treated as an extraordinary error condition...
If you are somehow stuck in a situation where you the process that feeds into the FileOutput is broken because of a product defect,, such that it throws an abend, and you can't avoid it until you get a product fix or are able to upgrade...
then I'd look to mediate the situation by isolating the FileOutput process into a separate flow kicked off by MQ. Then you can move this to another EG or another Broker and get isolated from the abend conditions.
It will also force you to think about the state of the file - in terms of whether you need to put row markers or something into the messages on the queue. Which will allow you to better know when you need to finish the file. |
|
| Back to top |
|
|
| j.f.sorge |
| Posted: Tue Oct 29, 2013 4:24 am Post subject: |
|
|
Master
Joined: 27 Feb 2008
Posts: 218
|
| mqjeff wrote: |
Well, you couldn't have it be deleted on *abend*, because I mean... it's an abnormal end...
The question is, are you able to tell at restart that you need to finish the file?
The second question is, why are you *planning* for an abend... You should be planning for normal error conditions, and planning to make manual efforts to resolve extraordinary error conditions. And an abend should always be treated as an extraordinary error condition...
If you are somehow stuck in a situation where you the process that feeds into the FileOutput is broken because of a product defect,, such that it throws an abend, and you can't avoid it until you get a product fix or are able to upgrade...
then I'd look to mediate the situation by isolating the FileOutput process into a separate flow kicked off by MQ. Then you can move this to another EG or another Broker and get isolated from the abend conditions.
It will also force you to think about the state of the file - in terms of whether you need to put row markers or something into the messages on the queue. Which will allow you to better know when you need to finish the file. |
Thanks a lot for your help! I will re-write the flow in order to create the output file in a single burst.
_________________
IBM Certified Solution Designer - WebSphere MQ V6.0
IBM Certified Solution Developer - WebSphere Message Broker V6.0
IBM Certified Solution Developer - WebSphere Message Broker V6.1
IBM Certified Solution Developer - WebSphere Message Broker V7.0 |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Oct 29, 2013 5:28 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| j.f.sorge wrote: |
| mqjeff wrote: |
Well, you couldn't have it be deleted on *abend*, because I mean... it's an abnormal end...
The question is, are you able to tell at restart that you need to finish the file?
The second question is, why are you *planning* for an abend... You should be planning for normal error conditions, and planning to make manual efforts to resolve extraordinary error conditions. And an abend should always be treated as an extraordinary error condition...
If you are somehow stuck in a situation where you the process that feeds into the FileOutput is broken because of a product defect,, such that it throws an abend, and you can't avoid it until you get a product fix or are able to upgrade...
then I'd look to mediate the situation by isolating the FileOutput process into a separate flow kicked off by MQ. Then you can move this to another EG or another Broker and get isolated from the abend conditions.
It will also force you to think about the state of the file - in terms of whether you need to put row markers or something into the messages on the queue. Which will allow you to better know when you need to finish the file. |
Thanks a lot for your help! I will re-write the flow in order to create the output file in a single burst. |
Ok.
That's not remotely what I suggested, but it's what you said was your other choice.
 |
|
| Back to top |
|
|
| Esa |
| Posted: Tue Oct 29, 2013 5:29 am Post subject: Re: [solved]transactionality of FileOutput mqsitransit direc |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| j.f.sorge wrote: |
If that could not be done, I'll have to map into a row and sent out that data at once which will cause a very big message in memory to be created.
|
| j.f.sorge wrote: |
Thanks a lot for your help! I will re-write the flow in order to create the output file in a single burst. |
That solution will inevitably cause a very big message to be handled in the flow(s)...
If you are running the flow with no additional instances, you could use a shared ESQL variable that - when uninitialized - would tell your flow to read away all files from mqsitransit before starting to process any messages/files.
You can argue it that would be planning for an unexpected abend. I would rather say that it's more like programmatically patching the File Nodes' limited support for transactions.
One way to avoid abends is to run the flow in a dedicated execution group. That would prevent abends caused by other flows in the same EG from disturbing your process. If it's not your file flow itself that causes the abends. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
