| Author |
Message
|
| mqxplorer |
 Posted: Wed Dec 21, 2011 11:07 am Post subject: Saving the input xml message in Environment-validation issue |
 |
|
Master
Joined: 22 Jun 2009
Posts: 206
|
Hi,
I have a small issue which is a continuation of my thread http://www.mqseries.net/phpBB2/viewtopic.php?t=50643&highlight=
I started sometime back and a new thread http://www.mqseries.net/phpBB2/viewtopic.php?t=53591#276225 I came across in the forum while searching with error text.
I have a scenario where I am saving the incoming xml message in Environment variables to use later in the flow.
The code i wrote for that is:
| Code: |
CREATE FIRSTCHILD OF Environment DOMAIN 'XMLNSC' TYPE XMLNSC.Folder NAME 'InMsg';
SET Environment.InMsg = InputRoot.XMLNSC; |
This is working fine as long as I do not enable the validation on the input node. When I enable the validation to 'Content and Value' it is throwing an exception and the exception list says
| Code: |
Number:INTEGER:5027
Text:CHARACTER:Validation is enabled but no message set has been specified |
I took the user trace with validation enabled and it complains about the message set which is deployed in the EG along with the flow.
The text from user trace ...
| Code: |
2011-12-21 13:28:56.781990 7064 UserTrace BIP2537I: Node 'MF_Test_Validation.Compute': Executing statement ''CREATE FIRSTCHILD OF Environment DOMAIN 'XMLNSC' TYPE XMLNSC.Folder NAMESPACE '' NAME 'InMsg';'' at ('.MF_Test_Validation_Compute.Main', '6.3').
2011-12-21 13:28:56.782020 7064 UserTrace BIP2539I: Node 'MF_Test_Validation.Compute': Evaluating expression ''XMLNSC.Folder'' at ('.MF_Test_Validation_Compute.Main', '6.57'). This resolved to ''XMLNSC.Folder''. The result was ''1095233437696''.
2011-12-21 13:28:56.782043 7064 UserTrace BIP2537I: Node 'MF_Test_Validation.Compute': Executing statement ''SET Environment.InMsg = InputRoot.XMLNSC;'' at ('.MF_Test_Validation_Compute.Main', '7.3').
2011-12-21 13:28:56.783428 7064 UserTrace BIP2539I: Node 'MF_Test_Validation.Compute': Evaluating expression ''InputRoot.XMLNSC'' at ('.MF_Test_Validation_Compute.Main', '7.37'). This resolved to ''InputRoot.XMLNSC''. The result was ''ROW... Root Element Type=16777216 NameSpace='' Name='XMLNSC' Value=NULL''.
2011-12-21 13:28:56.783454 7064 UserTrace BIP2568I: Node 'MF_Test_Validation.Compute': Copying sub-tree from ''InputRoot.XMLNSC'' to ''Environment.InMsg''.
2011-12-21 13:28:56.784633 7064 Error BIP2628E: Exception condition detected on input node 'MF_Test_Validation.TEST.VALIDATION.IN'.
The input node 'MF_Test_Validation.TEST.VALIDATION.IN' detected an error whilst processing a message. The message flow has been rolled-back and, if the message was being processed in a unit of work, it will remain on the input queue to be processed again. Following messages will indicate the cause of this exception.
Check the error messages which follow to determine why the exception was generated, and take action as described by those messages.
2011-12-21 13:28:56.784660 7064 RecoverableException BIP2230E: Error detected whilst processing a message in node 'MF_Test_Validation.Compute'.
The message broker detected an error whilst processing a message in node 'MF_Test_Validation.Compute'. An exception has been thrown to cut short the processing of the message.
See the following messages for details of the error.
2011-12-21 13:28:56.784683 7064 RecoverableException BIP2488E: ('.MF_Test_Validation_Compute.Main', '7.3') Error detected whilst executing the SQL statement ''SET Environment.InMsg = InputRoot.XMLNSC;''.
The message broker detected an error whilst executing the given statement. An exception has been thrown to cut short the SQL program.
See the following messages for details of the error.
2011-12-21 13:28:56.784702 7064 ParserException BIP5027E: Validation is enabled but no message set has been specified.
This error was reported by the XMLNSC parser.
The XMLNSC parser is attempting to validate an XML message or a message tree. You must identify the message set that defines the message in one of the following places:
- In the 'Message Set' property of an input node, a Validate node, or any other node that offers this property.
- In an MQRFH2 header
- In the SET parameter of the PARSE clause in an ESQL CREATE statement |
I have some work around which is working fine...
The code I wrote for this is..
| Code: |
DECLARE propRef REFERENCE TO InputRoot.Properties;
DECLARE inCCSID INT propRef.CodedCharSetId;
DECLARE inEncoding INT propRef.Encoding;
DECLARE msgBitStream BLOB ASBITSTREAM(InputRoot.XMLNSC, inEncoding, inCCSID);
CREATE FIRSTCHILD OF Environment DOMAIN 'XMLNSC' PARSE(msgBitStream, inEncoding, inCCSID, 'OLU8Q6G002001'); |
However, I am curious to understand why 'Setting Validation ON' is complainaing about message set.
Would it be possible for someone to explain me little bit more about this?
Thanks
mqxplorer |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed Dec 21, 2011 11:50 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
you cannot validate content and value without a message set.
The XMLNSC parser will already by default validate that you have a valid XML document. For anything beyond this basic validation, you need the xsds hence a message set.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Wed Dec 21, 2011 12:15 pm Post subject: |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| fjb_saper wrote: |
you cannot validate content and value without a message set.
The XMLNSC parser will already by default validate that you have a valid XML document. For anything beyond this basic validation, you need the xsds hence a message set.
Have fun |
I actually have the message set deployed in the Execution Group where the flow is running. When I am enabling the validation on the input node, the statements I wrote to save the incoming message throws an exception saying that the 'Validation is enabled but no message set has been specified'.
| Code: |
CREATE FIRSTCHILD OF Environment DOMAIN 'XMLNSC' TYPE XMLNSC.Folder NAME 'InMsg';
SET Environment.InMsg = InputRoot.XMLNSC; |
This is not just with an invalid message... even if I send the valid message same exception is thrown while executing the statement mentioned above....
Thanks
mqxplorer |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Dec 21, 2011 12:20 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
2 points:
- if you specify a message set (not just in this context but for any reason in ESQL) call it by it's name not OLU8Q6G002001. Much easier to see which set is in use and if you delete/redefine the set you don't need to change the references. Though I accept quoting the id (OLU...) is functionally equivalent with no effect on the running of the flow.
- Where you're getting the "Validation has been enabled but no message set has been specified" (and accepting the message set is deployed to the execution group) what mechanism do you believe is at work to identify which message set deployed to the execution group should be used for the validation?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Wed Dec 21, 2011 12:34 pm Post subject: Re: Saving the input xml message in Environment-validation i |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| mqxplorer wrote: |
This is working fine as long as I do not enable the validation on the input node. When I enable the validation to 'Content and Value' it is throwing an exception ...
The text from user trace ...
| Code: |
...
You must identify the message set that defines the message in one of the following places:
- In the 'Message Set' property of an input node, ...
|
|
Does the input node specify which message set to validate the message with? |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Wed Dec 21, 2011 12:40 pm Post subject: |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| Vitor wrote: |
2 points:
- if you specify a message set (not just in this context but for any reason in ESQL) call it by it's name not OLU8Q6G002001. Much easier to see which set is in use and if you delete/redefine the set you don't need to change the references. Though I accept quoting the id (OLU...) is functionally equivalent with no effect on the running of the flow.
- Where you're getting the "Validation has been enabled but no message set has been specified" (and accepting the message set is deployed to the execution group) what mechanism do you believe is at work to identify which message set deployed to the execution group should be used for the validation? |
Thanks for the first suggestion.. I will follow it.
Regarding the second point you mentioned....looking for the message set for validation... If it is on the input node, as I specified the message set ..it know s which message set to use to validate the message which has no issues in my case. However, when you say looking for message set (I mean which message set to validate against).. do you mean to say that the statement I wrote
| Code: |
| SET Environment.InMsg = InputRoot.XMLNSC; |
is doing validation? If so, is it validating the message while copying it from InputRoot to the 'Environment.InMsg' tree?
I am asking this beacuse I am really curious to know if the validation is done while tree copying .... at least in this case....
Thanks
mqxplorer |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Dec 21, 2011 12:51 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqxplorer wrote: |
| If so, is it validating the message while copying it from InputRoot to the 'Environment.InMsg' tree? |
Tree copying? It looks to me a lot like the CREATE statement is associating the XMLNSC domain (and incumbent parser) with InMsg.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Wed Dec 21, 2011 1:33 pm Post subject: |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| Vitor wrote: |
| mqxplorer wrote: |
| If so, is it validating the message while copying it from InputRoot to the 'Environment.InMsg' tree? |
Tree copying? It looks to me a lot like the CREATE statement is associating the XMLNSC domain (and incumbent parser) with InMsg. |
I am sorry...I used the wrong word 'Tree Copying'....
As per your statement, when the ESQL statements I wrote are executed the message validation happens.. If so what is the best way to save the incoming message in Environment variables without running into "Validation is enabled but no message set has been specified" issue?
The work around I have done....
| Code: |
DECLARE propRef REFERENCE TO InputRoot.Properties;
DECLARE inCCSID INT propRef.CodedCharSetId;
DECLARE inEncoding INT propRef.Encoding;
DECLARE msgBitStream BLOB ASBITSTREAM(InputRoot.XMLNSC, inEncoding, inCCSID);
CREATE FIRSTCHILD OF Environment DOMAIN 'XMLNSC' PARSE(msgBitStream, inEncoding, inCCSID, 'OLU8Q6G002001'); |
is working fine, however, as I am making the incoming message as BLOB and then parsing it, it is a performance overhead as the input node laready parse the input message based on the message set specifications it was supplied. Parsing the same message twice is what I want to eliminate...... What is the best way to save the incoming message in Environment variables to use it later in the message flow processing?
Thanks
mqxplorer |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Dec 21, 2011 3:47 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqxplorer wrote: |
| What is the best way to save the incoming message in Environment variables to use it later in the message flow processing? |
Associate the message set
Or encode it if all you're trying to pass it down the flow.
The question is why do you need the entire message tree in the Environment?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Tue Jan 17, 2012 11:44 am Post subject: |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| Vitor wrote: |
| mqxplorer wrote: |
| What is the best way to save the incoming message in Environment variables to use it later in the message flow processing? |
Associate the message set
Or encode it if all you're trying to pass it down the flow.
The question is why do you need the entire message tree in the Environment? |
I am calling a web service which we call it a routing rules service implemented in .NET and this service returns a list of consumer specific queues based on some business rules implmented in the .NET web service. Once I get the list of queues from the web service, I need to distribute the original message to different queues.
I do not want the message to be parsed twice as per my code.... I am revisting my code for performance tuning and I see the code I wrote adds performnce overhead...
Thanks
mqxplorer |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jan 17, 2012 12:17 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqxplorer wrote: |
| I do not want the message to be parsed twice as per my code.... I am revisting my code for performance tuning and I see the code I wrote adds performnce overhead... |
If the input node specifies BLOB as you indicate above (and which I missed previously  ) then it's not being parsed by the input node.
If you want the actual XML stored in the Environment under the XMLNSC domain (which is desirable to preserve attributes and so forth) then it needs to be created as you indicate. If you want validation to occur at any point then you need to supply a message set.
If you only want validation to occur once (e.g. on input) then set validation to occur there and not when the stream is parsed into the Environment.
| mqxplorer wrote: |
| I am calling a web service which we call it a routing rules service implemented in .NET and this service returns a list of consumer specific queues based on some business rules implmented in the .NET web service. Once I get the list of queues from the web service, I need to distribute the original message to different queues |
An alternative is to use an aggregation where one leg calls the web service and returns this list of queues, a second leg simply contains the original message. In this simple scenario I'm unconvinced this is a better or more performant way of achiving your requirement but I mention it for the record.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Tue Jan 17, 2012 3:28 pm Post subject: |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| Vitor wrote: |
| mqxplorer wrote: |
| I do not want the message to be parsed twice as per my code.... I am revisting my code for performance tuning and I see the code I wrote adds performnce overhead... |
If the input node specifies BLOB as you indicate above (and which I missed previously ) then it's not being parsed by the input node.
If you want the actual XML stored in the Environment under the XMLNSC domain (which is desirable to preserve attributes and so forth) then it needs to be created as you indicate. If you want validation to occur at any point then you need to supply a message set.
If you only want validation to occur once (e.g. on input) then set validation to occur there and not when the stream is parsed into the Environment.
| mqxplorer wrote: |
| I am calling a web service which we call it a routing rules service implemented in .NET and this service returns a list of consumer specific queues based on some business rules implmented in the .NET web service. Once I get the list of queues from the web service, I need to distribute the original message to different queues |
An alternative is to use an aggregation where one leg calls the web service and returns this list of queues, a second leg simply contains the original message. In this simple scenario I'm unconvinced this is a better or more performant way of achiving your requirement but I mention it for the record. |
I think, there is some disconnect in what I wrote and what you understood...
Let me restate what I am doing and what the problem is...
I am getting a message on to the input node which has the message domain set to XMLNSC and message set name also specified by selecting from the drop down menu. I have the validation set to 'ConetntAndValue'. When I first wrote the code like below as per my initial post..
| Code: |
CREATE FIRSTCHILD OF Environment DOMAIN 'XMLNSC' TYPE XMLNSC.Folder NAME 'InMsg';
SET Environment.InMsg = InputRoot.XMLNSC; |
I got the below error when I have the validation set to 'ContentAndValue' on the input node and no error if it is set to 'none'.
| Code: |
Number:INTEGER:5027
Text:CHARACTER:Validation is enabled but no message set has been specified |
Based on some research in the forum I wrote the code as below which is a work around which works forn for me...
| Code: |
DECLARE propRef REFERENCE TO InputRoot.Properties;
DECLARE inCCSID INT propRef.CodedCharSetId;
DECLARE inEncoding INT propRef.Encoding;
CREATE FIELD Environment.Variables.InMsg;
DECLARE InMsgPtr REFERENCE TO Environment.Variables.InMsg;
DECLARE msgBitStream BLOB ASBITSTREAM(InputRoot.XMLNSC, inEncoding, inCCSID);
CREATE FIRSTCHILD OF InMsgPtr DOMAIN 'XMLNSC' PARSE(msgBitStream, inEncoding, inCCSID, 'MS_XYZ'); |
The above code works fine for me, however, as I am parsing the incoming message twice - one on the input node and the other in the code with CRAETE and PARSE. This is definitely a performance overhead. I want to get rid of this double parsing the message. I have tried different options but I could not really find a way to achieve this.
I hope, I am little clear in my explanation.
Thanks
mqxplorer |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Jan 18, 2012 2:46 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
I got the below error when I have the validation set to 'ContentAndValue' on the input node and no error if it is set to 'none':
"Validation is enabled but no message set has been specified".
|
When you copy the entire message into the environment ( or perhaps at some later point in the flow ) you are triggering a parse of the message. But the XMLNSC parser that is attached to Environment.InMsg does not have the MessageSet property set up, so it complains that it cannot do the parsing.
Solution: Use the PARSE clause to set up the message set property in the Environment tree.
You have already implemented that solution. You are not parsing the message twice - message broker 'knows' that the message tree has not been modified, so it probably has not done any parsing at all - not even in the input node. The ASBITSTREAM simply returns the input BLOB, and the CREATE statement simply copies the same bit stream to the parser in the Environment tree.
Later on, your message flow uses those parsers and triggers some actual parsing - but you don't know, and don't actually need to know when that happens. |
|
| Back to top |
|
|
| mqxplorer |
| Posted: Wed Jan 18, 2012 8:04 am Post subject: |
|
|
Master
Joined: 22 Jun 2009
Posts: 206
|
| kimbert wrote: |
When you copy the entire message into the environment ( or perhaps at some later point in the flow ) you are triggering a parse of the message. But the XMLNSC parser that is attached to Environment.InMsg does not have the MessageSet property set up, so it complains that it cannot do the parsing.
Solution: Use the PARSE clause to set up the message set property in the Environment tree.
You have already implemented that solution. You are not parsing the message twice - message broker 'knows' that the message tree has not been modified, so it probably has not done any parsing at all - not even in the input node. The ASBITSTREAM simply returns the input BLOB, and the CREATE statement simply copies the same bit stream to the parser in the Environment tree.
Later on, your message flow uses those parsers and triggers some actual parsing - but you don't know, and don't actually need to know when that happens. |
Thanks for the clarification. The parsing on the Input node is happening as I set the Parser Options - parse timing to Immediate. The reason for this goes back to my other thread http://www.mqseries.net/phpBB2/viewtopic.php?t=55364&highlight= I started sometime back and raised a PMR for which I got the response from IBM as below.
| Code: |
The customer has fallen over a known problem with parser exceptions
being trapped whilst generating monitoring events.
However currently there is no fix available for this.
I have been in contact with our development team and I have found out
that there is still no solution available for this issue.
There are ideas on how to fix it, but none that have yet satisfied all
requirements and so they are still working through this,
I am now working with them to try and speed up the process of arriving
at a solution.
The nature of the scenario, where invoking monitoring can cause a number
of different exception types, in multiple different places in the flow,
means there is no straight forward fix available.
|
The last time I heard from IBM was sometime in August last year.
As my flow is a distribution flow with not much message manipulation happening and parser exceptions are being tarpepd whilst generating monitoring events, I did not want to have the fragmented message send to consumers, I set the Parse timing to 'Imemdiate'. The main reason for continuing the thread after a week I started it and found the solution to save the message in Environment tree (as I implemented) is - I was thinking that as the message is completely parsed on the input node and parsing is happening in the CREATE and PARSE caluse again (which as you explained is not happening).....thought it is a major operformnce issue.
Based on your explanation, I understood that the parsing does not happen with my implemented code. I am going to leave the parse timing set to Immediate till I get the fix from IBM and go with the imlemented code to save the message in the environemnt tree.
Thanks
mqxplorer |
|
| Back to top |
|
|
|
|
