ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportFlow (seemingly) randomly drops part of XMLNSC tree

Post new topicReply to topic
Flow (seemingly) randomly drops part of XMLNSC tree View previous topic :: View next topic
Author Message
Gralgrathor
PostPosted: Mon Aug 27, 2018 3:03 am Post subject: Flow (seemingly) randomly drops part of XMLNSC tree Reply with quote

Master

Joined: 23 Jul 2009
Posts: 284

Hello, dear humans!

I'm doing something wrong.

I have a flow that consists of the following parts:

  1. Receive MQ input message -> BLOB (the message contains XML)
  2. Subflow for committing BLOB to audit flow, produces original input as output
  3. RCD to XMLNSC
  4. ESQL node that reshuffles the XMLNSC tree for further processing using a number of ATTACH/DETACH operations
  5. Five subflows that transform the reshuffled tree into Idoc, commit Idoc to queue, and then produce original input as output (via a floworder -> output node) to be used as input for the next subflow


During testing it turned out that data was lost in the second (out of five) idoc-producing subflow.

I've stepped through the thing in debug mode to figure out where this occurred, and it turns out that the loss of XMLNSC tree structure occurs at the floworder node in the subflow.

Retracing execution shows that no particular manipulation of the lost element has occurred previous to this. A recursive function runs through the tree structure looking for metadata, to remove that from the tree and attach it elsewhere. The *parent* element of the lost element was DETACHed from the tree and ATTACHed elsewhere, but that's all. The parent is intact except for the missing last element within it. Other than that, the only operation that was applied to the lost element was a CARDINALITY(element.*[]) check to see whether the element had any children for recursive processing.

To recap:
The XML is received.
Some reshuffling takes place, and the reshuffled tree looks as expected.
The reshuffled tree is submitted to a subflow, which produces an Idoc, and then goes back to the original message using a floworder node.
After the floworder node, the message tree looks *different*: data is missing.

I'm calling it random because I can't find out from looking at code or message structure what sets the lost element apart from the rest, but depending on the original XML, it's always the same bit of data that is lost.

Has anyone heard of such a thing before? Any suggestions for debugging?

UPDATE

[MQ input] -> [Subflow audit] -> [RCD to xmlnsc] -> [ESQL compute node 'restructure'] -> [floworder] (1 -> [ESQL compute node 'transform'] -> [MQ output]), (2 -> Subflow output).

Now as far as I know, there's no code that affects the contents of the Inputroot in the 'transform' node.

And yet every time the pointer arrives at this line:

Code:
            DECLARE eenheid REFERENCE TO OutputRoot;
            SET ok = getEntity(entities, unitId, eenheid); -- finds <entity> with <entity id> = unitId, and MOVEs reference <eenheid> to this <entity>
            FOR g as eenheid.*[] DO
-->            DECLARE gtinRef REFERENCE TO g;


The aforementioned element in the data is lost. REFERENCEs <eenheid> and <gtinRef> aren't referring to this element or its parent at that time.

If I remove the bit of code at which the stack data seems to change, it changes at another, unrelated point.

I've solved the problem by changing the RCD XMLNSC parser mode to 'complete', so I'm guessing that the argument stack has a problem with partially parsed message trees. Can't imagine why it would 'randomly' drop certain elements, though...
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine.
Back to top
View user's profile Send private message Send e-mail
timber
PostPosted: Mon Aug 27, 2018 12:20 pm Post subject: Reply with quote

Sentinel

Joined: 25 Aug 2015
Posts: 868

Great problem description, and I don't have any questions. Unfortunately, I don't have any explanation for this behaviour either. Well done for finding a workaround, and I suggest that you open a PMR with IBM if you want an official explanation for this behaviour.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Tue Aug 28, 2018 2:05 am Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19770
Location: LI,NY

As you are using flow order nodes, please make sure that you're not manipulating the input tree in any of the downstream nodes.

Flow order is not as strict with the input tree and this may cause some surprises on the second downstream branch.

Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Gralgrathor
PostPosted: Tue Aug 28, 2018 2:59 am Post subject: Reply with quote

Master

Joined: 23 Jul 2009
Posts: 284

fjb_saper wrote:
As you are using flow order nodes, please make sure that you're not manipulating the input tree in any of the downstream nodes

Thanks. Yes, a colleague of mine mentioned this, and it got me to take another look at the flow.

Now the input tree isn't manipulated after the floworder nodes, or anywhere after the 'reshuffle' node, but his suggestion and my subsequent debugging did show that the 'on demand' parser mode was the most likely culprit.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine.
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker SupportFlow (seemingly) randomly drops part of XMLNSC tree
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.