| Author |
Message
|
| ruimadaleno |
 Posted: Wed Feb 08, 2017 3:16 am Post subject: FileInput node behaviour and deployment |
 |
|
Master
Joined: 08 May 2014
Posts: 274
|
Hi all,
we have built a message flow that reads files from a folder, do some transformation and integration and finnaly move the file to a destination folder.
FileInputNode (Read files from folder A)-----> transformation/mapping ---> soaprequest(integration via websevice)---> Fileoutput node (move file to destination folder)
We are trying to understand the behaviour and best practices on deploying this kind of message flows.
Now let's consider the following scenario:
There are 1200 files in Folder A. The message flow is started and is processing files.
Now we need to deploy a new message flow version (bug corrections) how to proceed ?
1) Wait for the message flow to end processing the 1200 files ?
2) Abort the 1200 files processing ? how can this be done ?
3) Deploy the message flow. This will issue a deploy command to broker that will have to wait for files processing to end,
eventually this deploy command will fail due to timeout
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Wed Feb 08, 2017 3:57 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
How about
Stop the flow first.
It will process the current file and end.
Then do the deploy
Then start the redeployed flow.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| ruimadaleno |
| Posted: Wed Feb 08, 2017 4:15 am Post subject: |
|
|
Master
Joined: 08 May 2014
Posts: 274
|
Hi smDavies, thank you for you input.
in fact that is the more logical and "catolic" way to do the deploy 
Just trying to understand what happens in this "extreme" scenario (maybe when deployment team miss some steps  ) so i can write scripts and clear instructions to deployment teams
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Feb 08, 2017 5:06 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
The deploy should cause the flow to stop processing anything after the current instance (or instances) that are running to finish their current work.
It doesn't matter how many files are in the folder. There will be 1->x files being processed at the time the deploy happens, where x == the number of instances of the flow that are actually in use.
When those 1->x instances have completed their work, no more files will be read.
When the deploy has completed, the flow will restart. It will continue processing any files that remain in the directory.
In an abnormal case - where a broker is halted badly - some of the files may still have locks on them. This would prevent them from being processed after the broker starts.
But it's either very unlikely or impossible for a file lock to remain set after the process that set them shuts down.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 08, 2017 5:50 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
The deploy should cause the flow to stop processing anything after the current instance (or instances) that are running to finish their current work.
It doesn't matter how many files are in the folder. There will be 1->x files being processed at the time the deploy happens, where x == the number of instances of the flow that are actually in use.
When those 1->x instances have completed their work, no more files will be read. |
If you have 1200 files in folder A, the FiileInput node doesn't know that. It knows that it found an individual file that matched the criteria for processing. The deploy command will signal that it shouldn't scan for another file once it's finished because a change is pending.
The biggest problem you're going to face is that some percentage of the files will be processed with one version of the flow, the rest with the new version. If we posit that you're deploying the flow because you're changing the processing from what-you-don't-want-to-happen-anymore to what-should-happen-now that's going to be an issue.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
