Posted: Wed May 08, 2019 7:21 am Post subject: Sequence node trouble
Acolyte
Joined: 11 Mar 2015 Posts: 71
Hi Experts!
I am trying to use the sequence node to get incremental counts of errors that occur while processing a file
Here is what i am trying to achieve
I am reading a text file with multiple rows, reading each row at a time. I used the modeler to define the schema.
I have few validations conditions in a compute node apart from a few schema validations that i defined while modelling the schema. If there is a validation error i propagate the message to a sequence node.
If there is a schema validation the message will propagate from the File input node's catch terminal, i have added a compute node to read the exception and pass through to the same sequence node again.
When the end of file is reached the end of data terminal triggers and i have logic that when IsEmpty is true to override the startofsequence to 0.
I have configured the sequence node as follows
Path to sequence number: default path $OutputLocalEnvironment/Sequence/Number
Start of sequence definition
Literal : 1
End of Sequence definition
Literal : 9223372036854775807
I set the persistence mode to : Persistent
Here is what is happening
When a error occurs the message propagates to the sequence node and the sequence number increments. But when InputLocalEnvironment.File.IsEmpty is True although I explicitly Set the
, the count increases after passing through the Sequence node. I verified and i am passing the local environment and message to the sequence node.
I thought that this could be because I explicit configured the sequence nodes end of sequence definition to literal. So i changed that to Predicate and set the Xpath to
Code:
$InputLocalEnvironment/File/IsEmpty = true()
But when i do this every time an error occurs and the message passes the sequence node the sequence number stays as 1
Any inputs would be appreciated. Thanks in advance!
If the only requirement is to count the errors then you could solve that without the Sequence node. One way would be to store all of the errors under a SHARED ROW variable
Code:
DECLARE validationErrorList SHARED ROW;
When the file completes, count the number of entries in the validationErrorList using CARDINALITY and do any other processing that's required. Then clear out the list and start again with the next file.
Thank you for your response. It does work using the shared row, i was just trying to figure if was doing something wrong with the sequence node and why it was behaving that way
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum