| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Compute Node Validation |
« View previous topic :: View next topic » |
| Author |
Message
|
| wolstek |
 Posted: Wed Dec 22, 2004 10:44 am Post subject: Compute Node Validation |
 |
|
Acolyte
Joined: 25 Jun 2001
Posts: 52
Location: Bristol, UK
|
I have a test flow that has effectively looks like (note I've only really put the trace nodes in to help debug)
| Code: |
MQInput(failure)--> Trace4(out)--> QueueFailure
(out)--> RCD(out)--> Trace1(out)--> Compute1 (see below)
--> Compute1(failure) --> Trace3(out)--> QueueFailure
(out)--> Trace2(out) --> MQOutput(failure)--> Trace5(out)--> Throw
(out) --> QueueOK
|
I have an XML1 defined MRM which has some value constraints in the form of enumerations
When the input message is correct it goes quite happily through to QueueOK
When the input message is invalid it goes to Queue Failure with an enumeration error as expected
However I would have expected error to have gone through Trace3 to failure; but it goes actually goes through Trace2 and fails on the MQOutput through Trace5 and thrown back to Trace4
quite happily through to QueueOK
I have the following nodes properties set, based on other posts
Compute Node: Validate = Content and Values, Failure Action = Exception
: Compute Mode = Message (and copying entire message)
MQInput Node: Validate = Content and Values, Failure Action = Exception , Timing = Immediate
Can anyone suggest circumstances why it wouldn't fail through the compute failure terminal? |
|
| Back to top |
|
 |
| Ramphart |
| Posted: Thu Dec 23, 2004 3:13 am Post subject: |
|
|
Disciple
Joined: 21 Jul 2004
Posts: 150
Location: South Africa, JHB
|
Hi,
Since you've set MQInput Node: Timing = Immediate an "INVALID" message will NEVER make it to the compute node. It will go directly to the Failure Terminal of the Input Node to Trace4.
You said that a "VALID" message makes it to the OK Queue so I can't see that you might have done something funny in the RCD node to cause a failure in the compute node. The only way you can ever get to Trace5 is if you send a "VALID" message but it fails when writing to QFailure after Trace2, that would explain why your flow will roll back via the path you've explained.
I think you must double check your scenario.
_________________
Applications Architect
Last edited by Ramphart on Thu Dec 23, 2004 10:57 pm; edited 1 time in total |
|
| Back to top |
|
|
| wolstek |
| Posted: Thu Dec 23, 2004 3:46 am Post subject: |
|
|
Acolyte
Joined: 25 Jun 2001
Posts: 52
Location: Bristol, UK
|
I know the flow looks a bit crazy, it is only to try and prove a theory that we can get the Compute validation working and failing through its failure terminal. In real life the main work is to be done as a subflow which will be called via a WebService, not through an input queue. The RCD of my flow is actually the start of a subflow, where the MQInput has the default XML domain not MRM/XML1
That aside, can you help me understand the Timing parameter, does it only apply to the MQIpnut node processing or the whole flow. I am not really sure that I understand the definition in the Message Flow manual and the difference between Complete and Immediate, they both indicate immediate validation.
| Quote: |
Set a value for Timing from the following list:
- Deferred: a message is validated only as each field is parsed. This is the default behavior.
– Immediate: a message is validated immediately, but subsets are permitted to remain unresolved (this supports Composition values Choice and Message under circumstances where the element is not yet resolved).
– Complete: the entire message is validated immediately.
|
Are you suggesting that Timing = Complete would make an INVALID message go to the failure terminal of a compute node
You say
| Quote: |
| Timing = Immediate an "INVALID" message will NEVER make it to the compute node |
. The MQInput node does not define the message set so it would not be doing any checking at MQInput time. |
|
| Back to top |
|
|
| Ramphart |
| Posted: Sun Jan 02, 2005 6:43 am Post subject: |
|
|
Disciple
Joined: 21 Jul 2004
Posts: 150
Location: South Africa, JHB
|
| Quote: |
| Are you suggesting that Timing = Complete would make an INVALID message go to the failure terminal of a compute node |
Nope - changing the timing will have the same effect in this case.
Your problem really comes in as to WHEN the parser validates your message. In your case the incoming message is never validated but the Output Message cause a problem when the parser use the MRM message set while writting to the output queue.
This is pretty much what happens in your case with an INVALID XML message (By invalid message I mean a well formed XML message but with your enumerators wrong for instance):
1. Input message to MQINPUT - Message Domain is XML so no problems here.
2. RCD resets the Message Domain, MessageSet, MessageType, MessageFormat and thats it. RCD does not do any validation EVER.
3. Your Compute Node get the Invalid message and since you set Validate to "Content and Value" the message is validated against the MQINPUT node properties (which is Message Domain = XML) hence no problem. It is important to note that it does note use the RCD properties here to validate the message. (Also note that only the fields in the INPUT message that you REFERENCE will be validated - There's a note later about this)
4. The message goes to the Compute Node Out terminal and to Trace2 and now the parser will try to write the MRM tree structure in memory to the Queue and since the MQMD properties is now MRM, etc (as specified in RCD) it uses your MRM definition to do so.
5. The outgoing message then fail validation against the MRM messageSet and goes to the failure Q of the MQOutput node.
6. Trace5 kicks in, then the Throw and the message is rolled back to Trace4.
Note the following also:
- In general the parser only validates those fields of the Input Message that you reference. This means that even if you did specify Message Domain, MessageSet=MRM, MessageType, MessageFormat on your MQInput Node and Timing = Deferred the message will still happily go thru your Compute node if you don't reference those invalid sections. ie. doing "Set OutputRoot = InputRoot" will not cause any validation to kick in.
- Just to confuse the matter. Your Trace1 node (if u do a ${Root}) will cause the Input Message to fail if your Message Domain=MRM, MessageSet, MessageType, MessageFormat on your MQINPUT Node and Timing = Deferred since ${Root} forces the parser to reference every field in the message.
| wolstek wrote: |
| Can anyone suggest circumstances why it wouldn't fail through the compute failure terminal? |
- If you want the message to fail via the Compute Node fail terminal try the following:
1. Specify on the MQInput Node your Message Domain=MRM, MessageSet, MessageType, MessageFormat and Timing = Deferred
2. Remove the Trace1 since you don't want the parser to validate the message here.
3. Compute Node: Validate = Content and Values, Failure Action = Exception
4. Specifically reference the particular field with erroneous enumerator values in the Compute Node
Like I said, the most important thing to understand is WHEN exactly the validation takes place.
_________________
Applications Architect |
|
| Back to top |
|
|
| wolstek |
| Posted: Tue Jan 04, 2005 4:53 am Post subject: |
|
|
Acolyte
Joined: 25 Jun 2001
Posts: 52
Location: Bristol, UK
|
Thanks for the detailed response
I have tried various scenarios based on your suggestions but ultimately any recorded failures against incorrect enumerations don't go through the Compute node failure. If I set MQInput to MRM... as suggested, with Timing = Complete it fails on MQInput which makes sense as it is being fully parsed there. If Timing = Deferred it flows through to the compute node which has an explicit OutputRoot.MRM.field = 'invalidvalue'. This flows through to the MQOutput and fails there.
Looking again at the Message Flow manual under the section Built-in node:Compute node it says
| Quote: |
| Select Validation in the properties dialog navigator if you want the MRM parser to validate the body of messages against the dictionary generated from the message set. The validation options you set on a Compute node apply only to any output messages that are created. Validation does not take place within the node itself, but only when the message is subsequently converted to a bit stream. |
I am therefore under the impression that the validation property on the compute node means that validation will take place but only when translated back into a bitstream i.e. at the MQOutput node, not at the Compute Node, and so the placing of this property on the Compute node is a bit misleading. I am not sure that your statement is entirely accurate, more likely that the broker does not use the new domain information until the bitstream is fully parsed (or elements of the new domain are referenced), this is backed up by the fact that it fails at the MQOutput node referencing the domain information specified in the RCD. |
|
| Back to top |
|
|
| wolstek |
| Posted: Tue Jan 04, 2005 4:54 am Post subject: |
|
|
Acolyte
Joined: 25 Jun 2001
Posts: 52
Location: Bristol, UK
|
sorry meant to say
I am not sure that your statement ...and since you set Validate to "Content and Value" the message is validated against the MQINPUT node properties (which is Message Domain = XML) hence no problem. It is important to note that it does note use the RCD properties here to validate the message. is entirely accurate, more likely that the broker does not use the new domain information until the bitstream is fully parsed (or elements of the new domain are referenced), this is backed up by the fact that it fails at the MQOutput node referencing the domain information specified in the RCD. |
|
| Back to top |
|
|
| Ramphart |
| Posted: Wed Jan 05, 2005 4:45 am Post subject: |
|
|
Disciple
Joined: 21 Jul 2004
Posts: 150
Location: South Africa, JHB
|
| wolstek wrote: |
| OutputRoot.MRM.field = 'invalidvalue' |
Is 'invalidvalue' just hard coded or is it an actual field from your input message like InputRoot.MRM.Field where field has an invalid valid?
It seems to me that your Outgoing MRM schema defines your "VALID" enum values. The way I see it is that you really need your Input message to be validated, i.e. your INPUT MRM Schema should define the valid enum values. If this is the case the message will definitely fail through your Compute Nodes failure terminal if you do an explicit mapping like
| Code: |
| OutputRoot.MRM.field = InputRoot.MRM.aField |
where aField is a wrong value.
Set your Timing = Deferred, MessageFormat to XML, RCD properties to MRM etc. and wire RCD to Compute Node, making sure that your RCD MRM Message Set defines the set of valid enums and do an explicit mapping as above. It should fail via Compute Failure node.
Pls get back to me as I've done this and got it to fail via Compute Failure node.
_________________
Applications Architect |
|
| Back to top |
|
|
| wolstek |
| Posted: Wed Jan 05, 2005 9:35 am Post subject: |
|
|
Acolyte
Joined: 25 Jun 2001
Posts: 52
Location: Bristol, UK
|
Have done as you suggested.
It doesn't seem to make a difference.It still goes down the compute out terminal and fails on the MQOutput.
Regarding referencing InputRoot I refer back to the quote before The validation options you set on a Compute node apply only to any output messages that are created.
In my eSQL I have, where Genre contains invalid data both on input and output
| Code: |
SET OutputRoot.MRM.Book[1].Title = InputRoot.MRM.Book[1].Genre;
SET OutputRoot.MRM.Book[1].Genre = 'asdfgh';
or
SET OutputRoot.MRM.Book.Title = InputRoot.MRM.Book.Genre;
SET OutputRoot.MRM.Book.Genre = 'asdfgh';
|
the former giving trace
| Code: |
2005-01-05 17:10:01.887663 4812 UserTrace BIP2568I: Node 'RunSampleChkXML.Process': Performing tree copy of 'InputRoot.MRM.Book[1].Genre' to 'OutputRoot.MRM.Book[1].Title'.
2005-01-05 17:10:01.887817 4812 UserTrace BIP2537I: Node 'RunSampleChkXML.Process': Executing statement 'SET OutputRoot.MRM.Book[1].Genre = 'asdfgh';' at (.subSampleChkXML_ComputeProcess.Main, 20.5).
2005-01-05 17:10:01.888021 4812 UserTrace BIP2566I: Node 'RunSampleChkXML.Process': Assigning value ''asdfgh'' to field / variable 'OutputRoot.MRM.Book[1].Genre'.
2005-01-05 17:10:01.888121 4812 UserTrace BIP2537I: Node 'RunSampleChkXML.Process': Executing statement 'SET OutputRoot.MRM.ProcessOptions.Update = 'UPDATED';' at (.subSampleChkXML_ComputeProcess.Main, 21.5).
|
|
|
| Back to top |
|
|
| Ramphart |
| Posted: Thu Jan 06, 2005 12:52 am Post subject: |
|
|
Disciple
Joined: 21 Jul 2004
Posts: 150
Location: South Africa, JHB
|
| wolstek wrote: |
| The validation options you set on a Compute node apply only to any output messages that are created. |
You're correct about this option in the Compute Node but we are not concerned about the Output message at this stage.
The InputMessage must be validated regardless since your InputRoot settings are: Validate : Content and Value, Failure Action : Exception, Timing : Deferred
Like I said, you need the Input Message validated and the way to do that is to reference the erroneous Input field which you did according to your last post.
Which Version of WMQI are you on and what CSD? I'm starting to suspect that your InputMessage is not validated against the options set after the RCD node.
| wolstek wrote: |
| If I set MQInput to MRM... as suggested, with Timing = Complete it fails on MQInput which makes sense as it is being fully parsed there. |
Since you've done this I'm sure that your Input MRM MessageSet is correct.
A: Try the following now:
1. Set your Timing = Deferred, MessageFormat to XML
2. RCD properties to MRM etc.
3. If you dont have Trace1 in after your RCD, put it back with ${Root} and wire to compute node.
4. Now put your Invalid Message on the Q.
5. You should only get trace1 and trace4 - if not then you know that the properties set in your RCD are not being used when validation takes place. I'm using WMQI 2.1 CSD 07 and I get Trace1 and Trace4 in this scenario.
6. If your message gets to the Compute Node then you know this is your problem as it should fail on trace1.
B: Now try this if the above got to the Compute Node
1. MessageDomian=MRM etc. Move all your RCD properties on your InputNode. Set your Timing = Deferred.
2. Remove RCD for now. Wire InputNode --> trace1 --> ComputeNode
3. Put erroneous message.
4. Now surely the message should fail on Trace1 and only Trace1 and Trace4 should be created.
5. If this is the case it confirms that your RCD properties are not used when you reference a field in the Compute Node.
C: Now try this if B produced Trace1 and Trace4 only.
1. MessageDomian=MRM etc. Set all your RCD properties on your InputNode. Set your Timing = Deferred.
2. Remove RCD and Trace1 for now. Wire InputNode --> ComputeNode.
3. Put your code:
| Code: |
| SET OutputRoot.MRM.Book[1].Title = InputRoot.MRM.Book[1].Genre; |
4. This should fail via Compute Failure Terminal.
5. If so, then you DEFINITELY know that the InputRoot properties are used in your case when validating the message in your Compute node [when you reference particular fields] and NOT you RCD properties as expected.
I remember that CSD 6 had some issues with validation and it was sorted out in CSD 7. Please get back to me.
_________________
Applications Architect |
|
| Back to top |
|
|
| wolstek |
| Posted: Thu Jan 06, 2005 5:32 am Post subject: |
|
|
Acolyte
Joined: 25 Jun 2001
Posts: 52
Location: Bristol, UK
|
I am actually running WBIMB V5 CSD4
I have finally had the message go down the compute failure node thanks to your suggestions.
Scenario A - XML defined on MQInput, with RCD, went down compute node out terminal and failed on MQOutput (Trace1, Trace2, Trace5)
Scenario B - MRM defined on MQInput, no RCD, trace after MQInput/before compute. Failed on trace parse (Trace1, Trace4)
Scenario C - MRM defined on MQInput, no RCD. no trace before compute. Failed on compute and went down compute failure (Trace3)
So this finally gets to the crux of the matter which is why the RCD does not have any effect on validation later in the flow, only on the parsing, clearly the Reset part of RCD isn't as full as one might hope. Any comments from WBIMB developers would be welcome.
It does leave me with a potential probelm later when we try to advertise the subFlow as a WebService (I am not sure how this works yet but I have been led to believe that it would have no MQInput). i.e how we can use the validation if at all as I would have set the MRM in an RCD in the subroutine. That is another investigation for when I get to that stage.
Many thanks for your help to date. |
|
| Back to top |
|
|
| shanson |
| Posted: Thu Jan 06, 2005 5:54 am Post subject: |
|
|
Partisan
Joined: 17 Oct 2003
Posts: 344
Location: IBM Hursley
|
| Quote: |
| I am therefore under the impression that the validation property on the compute node means that validation will take place but only when translated back into a bitstream i.e. at the MQOutput node, not at the Compute Node, and so the placing of this property on the Compute node is a bit misleading. |
Correct. This is stated in the documentation.
In the next release, we will be surfacing the set of validation options on other nodes. |
|
| Back to top |
|
|
| Ramphart |
| Posted: Thu Jan 06, 2005 6:00 am Post subject: |
|
|
Disciple
Joined: 21 Jul 2004
Posts: 150
Location: South Africa, JHB
|
| wolstek wrote: |
| So this finally gets to the crux of the matter which is why the RCD does not have any effect on validation later in the flow, only on the parsing, clearly the Reset part of RCD isn't as full as one might hope. Any comments from WBIMB developers would be welcome. |
I have not worked on WBIMB yet but I do find this strange as the RCD behaves as "expected" in WMQI 2.1 CSD 7.0.
Anyway - I'm glad we craked ths one. If you sort the issue out on how to use the WebService and trigger validation without an MQInput node then please post your solution here. I'd like to know how it works.
Regards.
_________________
Applications Architect |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
