ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportProcess flow stops abnormally after certain time

Post new topicReply to topic
Process flow stops abnormally after certain time View previous topic :: View next topic
Author Message
PBS12
PostPosted: Mon Nov 12, 2018 8:30 am Post subject: Process flow stops abnormally after certain time Reply with quote

Newbie

Joined: 12 Nov 2018
Posts: 3

HI,

I have 2 separate message flows like below-

1. Trigger-- It reads one file and calls another flow(Process) for all records in the file using dynamic aggregation with 4 records at a time asynchronously. I am using File Read node for reading the file.

2. Process- This is simple REST service which does Vertica DB read and DB2 write operations. Vertica DB read is really slow. So I have 4 instances of this service

Overall interface is like a batch job.

Whenever I am triggering first flow, observing that other calls on the bus are facing latency issues. Also, after certain time , say 20-23 minutes, first flow stops to call Process flow.

This looks very strange. Verified file read by FILE READ node from archive , it has all the records.

So, this is stopping after 21 minutes? In 21 minutes whatever records are processed are fine but why its stopping after 21 minutes?

Anyone faced this kind of issues? Need expert advice.

Appreciate your comments.

-PB
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Nov 13, 2018 5:52 am Post subject: Re: Process flow stops abnormally after certain time Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25365
Location: Ohio, USA

PBS12 wrote:
1. Trigger-- It reads one file and calls another flow(Process) for all records in the file using dynamic aggregation with 4 records at a time asynchronously. I am using File Read node for reading the file.


Why are you using FileRead not FileInput? It's an odd design choice.

PBS12 wrote:
Overall interface is like a batch job.


Another good reason to use a FileInput node.

PBS12 wrote:
So, this is stopping after 21 minutes? In 21 minutes whatever records are processed are fine but why its stopping after 21 minutes?


I suspect you're running the flow out of memory, or so close to out that it starts to drag. Calling the FileRead node in a loop is the textbook recommended method to run the broker JVM out of memory.

PBS12 wrote:
Need expert advice.


If you need to read a file from start to finish like a batch job, use the FileInput node. If you need to read a single record from a file (typically with a key or other correlation) use a FileRead node.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
PBS12
PostPosted: Tue Nov 13, 2018 12:38 pm Post subject: Re: Process flow stops abnormally after certain time Reply with quote

Newbie

Joined: 12 Nov 2018
Posts: 3

Why are you using FileRead not FileInput? It's an odd design choice.

Earlier I was using FileInput Node only, after seeing this issue I switched to File Read node hoping that will help. But there was no change in behavior. Also, I have monitored heap space and even broker logs during batch. It looks normal. Only problem I observed was multiple threds were active when I checked webserver logs.

Please advice. Appreciate your feedback.
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Nov 13, 2018 1:18 pm Post subject: Re: Process flow stops abnormally after certain time Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25365
Location: Ohio, USA

PBS12 wrote:
Earlier I was using FileInput Node only, after seeing this issue I switched to File Read node hoping that will help.


Ok, that's another odd decision that it would have helped if you'd mentioned in your original post.

PBS12 wrote:
But there was no change in behavior.


So if this happens with 2 different nodes why did you ask (in your other thread) if there was a timeout on the node?

PBS12 wrote:
Also, I have monitored heap space and even broker logs during batch. It looks normal.


If the logs seem normal, what does the user trace say the flow is doing after 21 minutes? As a bare minimum, does the trace indicate that it stops at the same place each time and what is that place?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker SupportProcess flow stops abnormally after certain time
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.