| Author |
Message
|
| er_pankajgupta84 |
 Posted: Tue Feb 16, 2010 6:06 pm Post subject: MRM over XMLNS/XMLNSC parsers for XML messages. |
 |
|
Master
Joined: 14 Nov 2008
Posts: 203
Location: charlotte,NC, USA
|
Hi,
This thing might be discussed here before as well but I am looking for some in depth answer.
What are the benefits does a MRM parser provide when we parse a XML message over parsing the same message using xmlns/xmlnsc parser.
1. Is it only in term of performance?
2. Is some sort of validation can only be done using MRM parser?
3. We don't specify a messageType when we chose XMLNS/XMLNSC parser. We only select message set. Does it makes any difference in terms of validation.
Now I will explain my problem -
I have a XML message which has some fields and one of that field has some invalid characters that comes from legacy. For eg: End of file charcater (0x06). This charcater appear as a rectangular box in editor.
I do not need the parse the data within that field (that has invalid characters) but i do need validations for all other fields.
How can i achieve this using various parser.
I tried to put that field in CDATA block but it still is generating any invalid character exception.
I tried with putting both parsers (MRM and then XMLNS) in a row with validation turned on MRM but still same exception.
One way is to encode the data using Base64 and then decode it after validation. But I cannot choose that way due to some reasons.
Any help is appreciated. Let me know if you need more info. |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Wed Feb 17, 2010 12:09 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
The answer to your initial question really depends on what the XML messages is going to be transformed into.
If it is going to a Cobol CopyBook/Flat File or a TDS (Tagged Delimited) then the MRM parser is the way to go.
If you are going to use the XML as XML then XMLNSC (with a few restrictions so you use the XMLNS) parser is the way to go.
All the development is being done on the XMLNSC parser not on the MRM one.
I'll leave it to Kimbert to expand on the rest of your question 
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Feb 17, 2010 3:11 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| What are the benefits does a MRM parser provide when we parse a XML message |
There are almost no benefits at all. MRM XML performance is several times slower than XMLNSC. You have less control over the output format. The only small advantage of MRM XML is that you can get the XML declaration ( the <?xml ... tag at the start ) automatically inserted for you. But it's not exactly difficult to do the same thing with XMLNSC.
| Quote: |
| 2. Is some sort of validation can only be done using MRM parser? |
If you are on v6.1, XMLNSC can do proper XML validation. If you are on v6.0, MRM XML can offer a pretty good approximation to XML validation.
| Quote: |
| 3. We don't specify a messageType when we chose XMLNS/XMLNSC parser. We only select message set. Does it makes any difference in terms of validation. |
No. The XMLNSC parser works out the correct message definition to use by looking at the root tag of the incoming XML message. If you are concerned that somebody might send a document which has the wrong root tag ( and that root tag might be one of the global elements in the message set ) then you could always check for the correct root tag name in your message flow. But that's a really rare requirement.
| Quote: |
| I have a XML message which has some fields and one of that field has some invalid characters that comes from legacy. |
That's not an XML message. It's a message which is trying to be XML, but failing.
Any properly written XML parser *must* reject such a message. Message broker is not being 'strict' here.
| Quote: |
One way is to encode the data using Base64 and then decode it after validation. But I cannot choose that way due to some reasons. |
That is the correct solution. I suggest that you revisit your 'reasons'.
You should use XMLNSC for all new message flows ( for XML parsing ) if you are on v6.1 or later. MRM is for non-XML.
If on v6.0, use MRM XML only if you *require* validation against an xsd. I don't think you do. |
|
| Back to top |
|
|
| er_pankajgupta84 |
| Posted: Wed Feb 17, 2010 7:16 am Post subject: |
|
|
Master
Joined: 14 Nov 2008
Posts: 203
Location: charlotte,NC, USA
|
Thanks for this valuable information.
I cannot use Base64 encoding and decoding because it would result in modifying many common services like Logging, Exception Handling etc.
| Quote: |
Quote:
I have a XML message which has some fields and one of that field has some invalid characters that comes from legacy.
That's not an XML message. It's a message which is trying to be XML, but failing.
Any properly written XML parser *must* reject such a message. Message broker is not being 'strict' here. |
You are right as legacy is sending a flat file and we are creating an XML for Auditing the message. That Audit message has entire payload,received from legacy, as one of its field.
My question is if we use CDATA for copying the legacy payload to the field of the XML message then it should work. Is there any restriction on CDATA?
Can we do it by making that field as OPAQUE in the RCD node. It seems that opaque elements works when validation is turned off. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Feb 17, 2010 7:21 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| er_pankajgupta84 wrote: |
| Is there any restriction on CDATA? |
CDATA elements must contain valid XML characters.
| er_pankajgupta84 wrote: |
| Can we do it by making that field as OPAQUE in the RCD node. It seems that opaque elements works when validation is turned off. |
Easier to try than to wait for a response. |
|
| Back to top |
|
|
| er_pankajgupta84 |
| Posted: Wed Feb 17, 2010 8:07 am Post subject: |
|
|
Master
Joined: 14 Nov 2008
Posts: 203
Location: charlotte,NC, USA
|
Using Opaque element does not work. As I need to use that field's value later in my JCN and if I make it opaque then it throws an exception from JCN saying that element does not exist.
Can anyone suggest something other than encoding and decoding. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 17, 2010 9:01 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| er_pankajgupta84 wrote: |
| Can anyone suggest something other than encoding and decoding. |
You can't escape the fact (poor pun intended) that this string contains invalid XML characters. Hence nothing you can do, apart from encoding / decoding, is going to get those characters into an XML document in any valid way.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Feb 17, 2010 12:09 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Vitor is 100% correct. If you persist in trying to find an 'easy' way you will simply create a maintenance problem for the future.
It doesn't sound like a difficult problem to solve. Here's a suggestion which is simple, and might work for you.
- base64 encode all strings which might contain illegal XML characters.
- add an xsi:type attribute to the XML tag which contains the encoded content. ( xsi:type="base64Binary" )
-In your logging/auditing framework, check the xsi:type attribute. If it is set to 'base64Binary' then decode the tag's contents before proceeding.
btw, mqjeff is correct about CDATA. Please read the infocenter page on CDATA which explains that point very clearly. |
|
| Back to top |
|
|
| er_pankajgupta84 |
| Posted: Wed Feb 17, 2010 12:48 pm Post subject: |
|
|
Master
Joined: 14 Nov 2008
Posts: 203
Location: charlotte,NC, USA
|
I completely agree with all of the points mentioned here.
Let me tell you my situation. I have to fix some low priority issues without any significant code change. I understand the problems of maintainability and flexibility.
I need to fix some production deployed code. So I might not be in a position to change several things for low priority issues.
Now, in our problem i think i can live with making the element opaque but that works without validation. I donot have any length restriction on my message set, just the cardinality is mentioned.
If i parse the message completely then i think parser will check for cardinality even if the validation is turned off. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Feb 17, 2010 3:14 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| legacy is sending a flat file and we are creating an XML for Auditing the message. That Audit message has entire payload,received from legacy, as one of its field. |
That's not a low-priority issue. It's a design flaw in your auditing solution.
I don't think this is 'low-priority' at all. The person who identified it as 'low-priority' might not realize why it's important. As a responsible engineer you should be providing that feedback.
| Quote: |
| i think i can live with making the element opaque but that works without validation |
True.
| Quote: |
| If i parse the message completely then i think parser will check for cardinality even if the validation is turned off. |
What makes you think that? |
|
| Back to top |
|
|
| er_pankajgupta84 |
| Posted: Wed Feb 17, 2010 5:26 pm Post subject: |
|
|
Master
Joined: 14 Nov 2008
Posts: 203
Location: charlotte,NC, USA
|
Kimbert
Priority is a pretty relative term. I mentioned it as low-priority just to address problems that have relatively more priority.
The thing is all the common services are working fine when we are getting valid data. The problem comes when we try to address the invalid data.
So I resolved few issue related to the problem by following suggestion mentioned in the thread.
Now I have this wired problem.
I have a XML which i need to serialize and paste as a string to one of the field in other XML.
For example:
First XML is :
<Hi>
<Name>Pankaj</Name>
</Hi>
Second XML should be like:
<Message>
<Header>
<Country>US</Country>
<CURR>Dollar</CURR>
</Header>
<Payload>
<PayloadString><Hi><Name>Pankaj</Name></Hi></PayloadString>
</Payload>
<Message>
Here PayloadString should have the first serialized XML.
I am using JCN and tobitstream function did the job of serializing the xml for me.
In debug mode the message looks fine. But when this message goes to a Queue then in the queue "<Hi><Name>Pankaj</Name></Hi>" becomes
<Hi><Name>Pankaj</Name></Hi>
I tried by using CDATA field as well but with CDATA output looks like
<![CDATA[<Hi><Name>Pankaj</Name></Hi>]]
The point is it might be the correct representation of data but in my case Another flow will pick it up from Queue and insert the content in DB. I want to insert "<Hi><Name>Pankaj</Name></Hi>" in DB.
I am not sure if i am using CDATA correctly or not. But I believe if in Queue I receive "<Hi><Name>Pankaj</Name></Hi>" then I am good.
Any idea? |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 17, 2010 6:20 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| er_pankajgupta84 wrote: |
| I am not sure if i am using CDATA correctly or not. |
I think you need a good book on XML.
| er_pankajgupta84 wrote: |
| But I believe if in Queue I receive "<Hi><Name>Pankaj</Name></Hi>" then I am good. |
Pedantically I'm forced to point out that all the characters in that string are valid XML so it doesn't display the problem we've been discussing.
If you have resolved the invalid character problem, please post your solution code for the benefit of others.
As to your other issue, an XML parser will treat < and < as equivalent as per the XML standard. So what you're getting in the queue is exactly what you're expecting. Any problem would only occur if you treat the string as a string rather than re-parsing it, i.e. how you're doing it.
For the record, that CDATA section is another XML equivalent way of representing the document & you are using it correctly.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| er_pankajgupta84 |
| Posted: Wed Feb 17, 2010 6:47 pm Post subject: |
|
|
Master
Joined: 14 Nov 2008
Posts: 203
Location: charlotte,NC, USA
|
Solution for the first problem:
Make the element (PayloadString) as opaque on the RCD node that is parsing the data using XMLNSC parser.
Basically i was receiving invalid character in my input message (Could be XML or flat file) and then i am copying that input message in the field "PayloadString" of an XML message(Audit Message). If the input message is XML then I am serializing it before copying it to the "PayloadString" field.
Now the second problem is of < & gt;. I don't want them to appear in place of < or >.
For testing purpose, I tried parsing my XML(Audit Message) without putting any opaque element but it still did not convert < > back to < or >
I cannot use replace function of java to replace lt gt as the input message itself contains embedded XML.
| Quote: |
I think you need a good book on XML.
|
I would appreciate if you can suggest some short article. The problem here is how to create CDATA section using JCN correctly. If you have some article then let me know. Just for your information the section that i want to put in CDATA doesnot have any invalid character. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 17, 2010 8:13 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| er_pankajgupta84 wrote: |
| For testing purpose, I tried parsing my XML(Audit Message) without putting any opaque element but it still did not convert < > back to < or > |
They wouldn't - no XML parser will see a difference.
| er_pankajgupta84 wrote: |
| I cannot use replace function of java to replace lt gt as the input message itself contains embedded XML. |
I do feel you're digging yourself a hole here. As has been commented earlier, this design has a few issues. This is one of them.
| er_pankajgupta84 wrote: |
| I would appreciate if you can suggest some short article. |
I don't think a short article is really going to cover XML to the depth you're trying to plumb it, but there's any number of good web sites Mr Google will find for you. Failing that, Amazon has a good range of guides.
| er_pankajgupta84 wrote: |
| The problem here is how to create CDATA section using JCN correctly. |
The problem here is I don't think CDATA does what you think it does, or what you need it to do.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| er_pankajgupta84 |
| Posted: Thu Feb 18, 2010 8:54 am Post subject: |
|
|
Master
Joined: 14 Nov 2008
Posts: 203
Location: charlotte,NC, USA
|
If I remove the opaque parsing for that field then even if i receive "<" and ">" for < and > then the parser convert it to < and >.
I don't know why it wasn't working earlier.
Thanks for all the help |
|
| Back to top |
|
|
|
|
