ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportFileInput node behaviour and deployment

Post new topicReply to topic
FileInput node behaviour and deployment View previous topic :: View next topic
Author Message
ruimadaleno
PostPosted: Wed Feb 08, 2017 3:16 am Post subject: FileInput node behaviour and deployment Reply with quote

Master

Joined: 08 May 2014
Posts: 274

Hi all,

we have built a message flow that reads files from a folder, do some transformation and integration and finnaly move the file to a destination folder.


FileInputNode (Read files from folder A)-----> transformation/mapping ---> soaprequest(integration via websevice)---> Fileoutput node (move file to destination folder)


We are trying to understand the behaviour and best practices on deploying this kind of message flows.


Now let's consider the following scenario:

There are 1200 files in Folder A. The message flow is started and is processing files.
Now we need to deploy a new message flow version (bug corrections) how to proceed ?

1) Wait for the message flow to end processing the 1200 files ?
2) Abort the 1200 files processing ? how can this be done ?
3) Deploy the message flow. This will issue a deploy command to broker that will have to wait for files processing to end,
eventually this deploy command will fail due to timeout
_________________
Best regards

Rui Madaleno
Back to top
View user's profile Send private message
smdavies99
PostPosted: Wed Feb 08, 2017 3:57 am Post subject: Reply with quote

Jedi Council

Joined: 10 Feb 2003
Posts: 6077
Location: Somewhere over the Rainbow this side of Never-never land.

How about

Stop the flow first.
It will process the current file and end.

Then do the deploy
Then start the redeployed flow.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995

Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Back to top
View user's profile Send private message
ruimadaleno
PostPosted: Wed Feb 08, 2017 4:15 am Post subject: Reply with quote

Master

Joined: 08 May 2014
Posts: 274

Hi smDavies, thank you for you input.

in fact that is the more logical and "catolic" way to do the deploy

Just trying to understand what happens in this "extreme" scenario (maybe when deployment team miss some steps ) so i can write scripts and clear instructions to deployment teams
_________________
Best regards

Rui Madaleno
Back to top
View user's profile Send private message
mqjeff
PostPosted: Wed Feb 08, 2017 5:06 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

The deploy should cause the flow to stop processing anything after the current instance (or instances) that are running to finish their current work.

It doesn't matter how many files are in the folder. There will be 1->x files being processed at the time the deploy happens, where x == the number of instances of the flow that are actually in use.

When those 1->x instances have completed their work, no more files will be read.

When the deploy has completed, the flow will restart. It will continue processing any files that remain in the directory.

In an abnormal case - where a broker is halted badly - some of the files may still have locks on them. This would prevent them from being processed after the broker starts.

But it's either very unlikely or impossible for a file lock to remain set after the process that set them shuts down.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Feb 08, 2017 5:50 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26074
Location: Texas, USA

mqjeff wrote:
The deploy should cause the flow to stop processing anything after the current instance (or instances) that are running to finish their current work.

It doesn't matter how many files are in the folder. There will be 1->x files being processed at the time the deploy happens, where x == the number of instances of the flow that are actually in use.

When those 1->x instances have completed their work, no more files will be read.




If you have 1200 files in folder A, the FiileInput node doesn't know that. It knows that it found an individual file that matched the criteria for processing. The deploy command will signal that it shouldn't scan for another file once it's finished because a change is pending.

The biggest problem you're going to face is that some percentage of the files will be processed with one version of the flow, the rest with the new version. If we posit that you're deploying the flow because you're changing the processing from what-you-don't-want-to-happen-anymore to what-should-happen-now that's going to be an issue.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker SupportFileInput node behaviour and deployment
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.