| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Process flow stops abnormally after certain time |
« View previous topic :: View next topic » |
| Author |
Message
|
| PBS12 |
 Posted: Mon Nov 12, 2018 8:30 am Post subject: Process flow stops abnormally after certain time |
 |
|
Newbie
Joined: 12 Nov 2018
Posts: 9
|
HI,
I have 2 separate message flows like below-
1. Trigger-- It reads one file and calls another flow(Process) for all records in the file using dynamic aggregation with 4 records at a time asynchronously. I am using File Read node for reading the file.
2. Process- This is simple REST service which does Vertica DB read and DB2 write operations. Vertica DB read is really slow. So I have 4 instances of this service
Overall interface is like a batch job.
Whenever I am triggering first flow, observing that other calls on the bus are facing latency issues. Also, after certain time , say 20-23 minutes, first flow stops to call Process flow.
This looks very strange. Verified file read by FILE READ node from archive , it has all the records.
So, this is stopping after 21 minutes? In 21 minutes whatever records are processed are fine but why its stopping after 21 minutes?
Anyone faced this kind of issues? Need expert advice.
Appreciate your comments.
-PB |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Nov 13, 2018 5:52 am Post subject: Re: Process flow stops abnormally after certain time |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PBS12 wrote: |
| 1. Trigger-- It reads one file and calls another flow(Process) for all records in the file using dynamic aggregation with 4 records at a time asynchronously. I am using File Read node for reading the file. |
Why are you using FileRead not FileInput? It's an odd design choice.
| PBS12 wrote: |
| Overall interface is like a batch job. |
Another good reason to use a FileInput node.
| PBS12 wrote: |
| So, this is stopping after 21 minutes? In 21 minutes whatever records are processed are fine but why its stopping after 21 minutes? |
I suspect you're running the flow out of memory, or so close to out that it starts to drag. Calling the FileRead node in a loop is the textbook recommended method to run the broker JVM out of memory.
| PBS12 wrote: |
| Need expert advice. |
If you need to read a file from start to finish like a batch job, use the FileInput node. If you need to read a single record from a file (typically with a key or other correlation) use a FileRead node.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PBS12 |
| Posted: Tue Nov 13, 2018 12:38 pm Post subject: Re: Process flow stops abnormally after certain time |
|
|
Newbie
Joined: 12 Nov 2018
Posts: 9
|
Why are you using FileRead not FileInput? It's an odd design choice.
Earlier I was using FileInput Node only, after seeing this issue I switched to File Read node hoping that will help. But there was no change in behavior. Also, I have monitored heap space and even broker logs during batch. It looks normal. Only problem I observed was multiple threds were active when I checked webserver logs.
Please advice. Appreciate your feedback. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 13, 2018 1:18 pm Post subject: Re: Process flow stops abnormally after certain time |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PBS12 wrote: |
| Earlier I was using FileInput Node only, after seeing this issue I switched to File Read node hoping that will help. |
Ok, that's another odd decision that it would have helped if you'd mentioned in your original post.
| PBS12 wrote: |
| But there was no change in behavior. |
So if this happens with 2 different nodes why did you ask (in your other thread) if there was a timeout on the node?
| PBS12 wrote: |
| Also, I have monitored heap space and even broker logs during batch. It looks normal. |
If the logs seem normal, what does the user trace say the flow is doing after 21 minutes? As a bare minimum, does the trace indicate that it stops at the same place each time and what is that place?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
