| Author |
Message
|
| scravr |
 Posted: Thu Aug 15, 2013 7:50 am Post subject: Recursive in flow |
 |
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
HI to All,
Is there a limit on Recursive path between flow (or sub-flow) nodes?
In V8 independent resource artifacts (V7 style):
I have a path like MQInput->
(sub flow1): multiple-computes->
(sub flow2): multiple computes->
(sub flow3): compute1->fileread->compuete2->back to compute1 ---- loop exits based on condition in compue1
subflow3 exit back to subflow2 which call subflow3 again until exit condition in subflow2
Looks like subflow3 crash after 8 internal loops between compute1 and compute2 on first iteration from subflow2.
Is there a limit?
Thanks |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Thu Aug 15, 2013 7:54 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| scravr |
| Posted: Thu Aug 15, 2013 7:55 am Post subject: |
|
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
| I am asking about limit . not about design !!! |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Aug 15, 2013 7:59 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| scravr wrote: |
| I am asking about limit . not about design !!! |
The limit is zero. You should never use circular logic paths. It creates problems with memory management.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| scravr |
| Posted: Thu Aug 15, 2013 8:17 am Post subject: |
|
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
That's not always true.
In some cases I have loop of 300K times and all works fine. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Aug 15, 2013 8:26 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| scravr wrote: |
That's not always true.
In some cases I have loop of 300K times and all works fine. |
No.
It's always true that using wires to create a loop in a message flow will cause a crash.
It's always a bad idea to use wires to create a loop in a message flow.
always.
If you are using propagate instead of wires, then this is an entirely separate thing.
In both cases there is no fixed limit. In both cases you still need to consider how you are using memory and how you are handling the flow.
But it's still always the wrong thing to use wires, and it will always cause a crash if you do.
Sure, sometimes you can get away with only having three tires on your four wheeled car. And sometimes you can even drive several miles that way.
It's always a bad idea to do so. |
|
| Back to top |
|
|
| scravr |
| Posted: Thu Aug 15, 2013 8:55 am Post subject: |
|
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
yes, there are several propagates from each subflow to other labels.
This particular crash was in
(subflow3): compute1 function1 propagate to
(subflow4): input->fileread->compuete2 function2 calls function1 in subflow3 |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Aug 15, 2013 9:40 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| scravr wrote: |
This particular crash was in
(subflow3): compute1 function1 propagate to
(subflow4): input->fileread->compuete2 function2 calls function1 in subflow3 |
So this is the point where you ran out of whatever resource you ran out of.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Aug 15, 2013 9:43 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
There are a ton of issues with subflows in v8.0.0.1 and earlier.
Even with compiled and inlined flows.
You should absolutely positively be at 8.0.0.2 of Runtime.
You also don't say what "crash" is.
Does the EG actually halt?
Or does the flow simply stop doing work? |
|
| Back to top |
|
|
| NealM |
| Posted: Thu Aug 15, 2013 10:03 am Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
| Quote: |
There are a ton of issues with subflows in v8.0.0.1 and earlier.
Even with compiled and inlined flows.
You should absolutely positively be at 8.0.0.2 of Runtime. |
Even FP2 isn't safe if you use subflows heavily. We have several APARs on top of 8002 for that situation.
Recursive flows, subflows, nodes will eventually cause the Broker to "blow it's stack" (well, one of it's stacks) after a number of iterations. Think about how the Broker internally is keeping track of the entire path that a message goes down until it gets to that path's end, so in case there is an exception it can go backwards all the way to the input node. That particular stack is not limitless (I've blown it myself). I've found other ways to blow other stacks, but that one is the most obvious. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Aug 15, 2013 10:21 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Sure, it's entirely possible to create a rat's nest of subflow invocations that will use up the EG stack - given that each node is another subroutine call.
But doing so is an indication that you should refactor your subflows.... Just like blowing the stack of a standalone application through highly recursive subroutine calls is an indication that you should refactor your subroutines.
As the ancient sages say
| Quote: |
| first you curse, then you recurse |
|
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Aug 15, 2013 11:02 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
@scravr: I can confirm the advice already given by mqjeff. Don't put loops into message flows if you can possibly avoid it. If you are asking what the limits are then that is a strong signal that your design is fragile. I strongly advise you to find a better design.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| NealM |
| Posted: Thu Aug 15, 2013 11:19 am Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
| Quote: |
| Sure, it's entirely possible to create a rat's nest of subflow invocations that will use up the EG stack - given that each node is another subroutine call. |
Actually all it takes in 8002 is having several (ok, a bunch of) common subflows in a Library and a bunch of Broker Projects (not Applications) deployed in an EG, all of which use a few (not a bunch) of the common subflows. Turns out the product was designed for that but never thoroughly tested. Until we started migrating our v6.1 stuff that is. I guess the thought was that everybody would switch to Applications.
Anyway, we will likely join the v10 beta to ensure it doesn't happen again. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Aug 15, 2013 11:59 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| NealM wrote: |
Anyway, we will likely join the v10 beta to ensure it doesn't happen again. |
What about V9.0.0.0? Does it exhibit the same problem(s)?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| NealM |
| Posted: Thu Aug 15, 2013 12:13 pm Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
| Quote: |
| What about V9.0.0.0? Does it exhibit the same problem(s)? |
I'm sure v9000 has many of the the same issues as 8002; Same developers. I would think that for v9 they concentrated more on the new stuff, like pulling in WESB, etc. Our APARs are supposed to be in 8003 and since 9001 is due out at approximately the same time, I would expect the pertinent ones will show up there also. But that expectation is not from any hands on with v9, and there have been times in the past where fixes to the down-level broker came earlier than to the newer broker.
From our standpoint, there isn't very much new that is applicable to us in v9, so our next upgrade will be to v10. |
|
| Back to top |
|
|
|
|
