| Author |
Message
|
| rhine |
 Posted: Tue Dec 15, 2015 2:42 am Post subject: MQInput new messages |
 |
|
Novice
Joined: 17 Nov 2015
Posts: 16
|
I want my MQInput node to wait a specific time lapse before generating a new event with TimeControl Node because i expect messages to be received before the subflow called by the TimeControl node ends it's process.
The MQInput receives XML messages. The problem is, the first XML gets stomped by the next one so the first one is not properly processed.
I made a java sleep to get the process delayed but the XML gets override anyway.
Does anyone have a good approach to evade that overriding? I need all the messages to be processed.
Thanks in advance. |
|
| Back to top |
|
 |
| timber |
| Posted: Tue Dec 15, 2015 4:00 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| I don't understand. Why do you need to insert sleeps or timeouts when you are using a message *queueing* system? Please explain what task the message flow is performing. |
|
| Back to top |
|
|
| rhine |
| Posted: Tue Dec 15, 2015 4:28 am Post subject: |
|
|
Novice
Joined: 17 Nov 2015
Posts: 16
|
MQInput gets an XML msg. Depending of its content, a compute decides routing it to one TimeoutControl or another.
Then the flow that processes that generated event can take few minutes before ending.
A new message can be put into the queue before the secondary flow ends its work, overriding the current process. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Dec 15, 2015 4:46 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
IF you want to setup selective delays JUST using timeout control nodes then frankly you are in for a whole heap of trouble. It does not scale. Been there, got the 'T-Shirt'.
IMHO, you need to disconnect the output from the input.
I'd put the input messages into a table with the delay time as a specific column (the Primary key). Then every so often (controlled by a fixed timeout control node) the table would be read and 1 or more messages that need to be sent onward processed and deleted from the table.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Dec 15, 2015 5:14 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| rhine wrote: |
MQInput gets an XML msg. Depending of its content, a compute decides routing it to one TimeoutControl or another.
Then the flow that processes that generated event can take few minutes before ending.
A new message can be put into the queue before the secondary flow ends its work, overriding the current process. |
As designs go, that's absolutely horrible. If your flow is:
Get Message -> Set Timer -> Timer Fires -> Something happens -> Wait for next message
then you need a much stronger latching mechanism than just a timer.
Why exactly are you using a timer node in all of this? Why doesn't the input message directly trigger this secondary process?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| rhine |
| Posted: Tue Dec 15, 2015 5:16 am Post subject: |
|
|
Novice
Joined: 17 Nov 2015
Posts: 16
|
I do not need a selective delay. I just need the messages of the Q to wait until the secondary flow has ended its work.
I can control whether is working or not with global cache, but that needs the initial flow to reach a compute node and i dont know how could the initial flow could beheave. |
|
| Back to top |
|
|
| joebuckeye |
| Posted: Tue Dec 15, 2015 5:21 am Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 365
Location: Columbus, OH
|
What does the first flow actually do?
Why not have the "secondary" flow just pull messages from the queue when it is ready? |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Dec 15, 2015 5:31 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Using sleep is a sign of bad design. However at least use the ESQL sleep function and not the Java native one. That way WMB has some control.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Dec 15, 2015 5:47 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| rhine wrote: |
| I just need the messages of the Q to wait until the secondary flow has ended its work. |
Why?
| rhine wrote: |
| I can control whether is working or not with global cache, but that needs the initial flow to reach a compute node |
Why wouldn't it?
| rhine wrote: |
| i dont know how could the initial flow could beheave. |
Why not?
From what you've described, you could as easily have:
Get message -> Decide which secondary flow to use -> Process secondary flow -> process remains of primary flow
Like that, each message is it's own thread.
So why doesn't that work for you?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| timber |
| Posted: Tue Dec 15, 2015 6:36 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| A new message can be put into the queue before the secondary flow ends its work, overriding the current process. |
There's your problem. You have a long-running process, and no way of knowing when it ends. A timeout is a horrible solution to that problem - you have no guarantee that any timeout will be long enough, so you need to set it to a huge value just to be sure.
Why can't the long-running processes send back a signal when they complete? Then you can split the flow into two parts, and trigger the next call as soon as the previous one finishes.
If I was asked to solve this problem, I would advise the company that they should redesign their systems so that processes communicate via message queues and deal with small units of work that complete quickly. |
|
| Back to top |
|
|
| rhine |
| Posted: Tue Dec 15, 2015 7:03 am Post subject: |
|
|
Novice
Joined: 17 Nov 2015
Posts: 16
|
| Vitor wrote: |
| rhine wrote: |
| I just need the messages of the Q to wait until the secondary flow has ended its work. |
Why?
Because if the iniciator does not wait, it makes a new event which the secondary flow processes, overriding the previous one.
| rhine wrote: |
| I can control whether is working or not with global cache, but that needs the initial flow to reach a compute node |
Why wouldn't it?
Because if it reached the compute, the xml message already passed through the MQInput node
| rhine wrote: |
| i dont know how could the initial flow could beheave. |
Why not?
From what you've described, you could as easily have:
Get message -> Decide which secondary flow to use -> Process secondary flow -> process remains of primary flow
Like that, each message is it's own thread.
So why doesn't that work for you? |
Because, one event overrides the previous one.
| Quote: |
| Why can't the long-running processes send back a signal when they complete? Then you can split the flow into two parts, and trigger the next call as soon as the previous one finishes. |
Yes, this could be a nice approach. I'll try.
Thanks |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Dec 15, 2015 7:09 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| rhine wrote: |
| Because if the iniciator does not wait, it makes a new event which the secondary flow processes, overriding the previous one. |
No it doesn't. Each new message is either a separate thread or isn't picked up until the first one is completed. The only reason it's "overriding" anything is because of the way you're invoking the secondary flow, i.e. it's a bug in your design.
| rhine wrote: |
| Because if it reached the compute, the xml message already passed through the MQInput node |
See above. That shouldn't matter.
| rhine wrote: |
| Because, one event overrides the previous one. |
See above.
| rhine wrote: |
| Yes, this could be a nice approach. |
That's why I suggested it.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
