| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Parsing issue during migration from v2.0.1 to v2.1. |
« View previous topic :: View next topic » |
| Author |
Message
|
| jgooch |
 Posted: Fri Mar 07, 2003 7:49 am Post subject: Parsing issue during migration from v2.0.1 to v2.1. |
 |
|
Acolyte
Joined: 29 May 2002
Posts: 63
Location: UK
|
Hi,
We are currently working on the migration of MQSI from v2.0.1 to v2.1.
We have a flow which inserts details from an XML record to a Sybase database. The message is then reparsed to a BLOB, and a null is appended. The message is then reparsed to an null-terminated MRM and the Sybase database record is updated with a field from the MRM (which actually represents the entire XML document).
This flow works fine in v2.0.1 but not in v2.1.
The problem is caused by the reparsing from XML to BLOB.
In v2.0.1, apostrophes within XML are held in memory as "'" and are transferred to BLOB as hex "2661706f733b". In v2.1, the apostrophe is held as "'" and is therefore transferred to BLOB as hex "27".
In v2.1, this means that when the message is reparsed as MRM, it contains an apostrophe (rather than the XML entity reference). Therefore the database insert fails (as MQSI wraps the insert up in apostrophes and thinks that the apostrophe within the data marks the end of the insertion.
It seems to me that the way in which messages are reparsed has changed significantly between the versions. v2.1 seems to reparse the data as originally presented from the MQInput node whereas v2.0.1 reparses the data as presented from the previous node in the flow.
From the usertrace, you can see that the BLOB representations are different lengths. I've highlighted the suspect part:-
v2.0.1 =
3c6d6573736167653e3c64657363726970746f723e3c6d657373616765547970653e4a4552454d592e544553543c2f6d657373616765547970653e3c6572726f72436f64653e31323334353c2f6572726f72436f64653e3c6572726f724465736372697074696f6e3e536f6d657468696e67207265616c6c7920626164206861732068617070656e65643c2f6572726f724465736372697074696f6e3e3c6661696c75726554696d657374616d703e323030322d30362d30322031343a30303a30312e3030303c2f6661696c75726554696d657374616d703e3c696e70757451756575653e424c41434b484f4c453c2f696e70757451756575653e3c756e6971756549643e582661706f733b3431344435313230344435373445353434343330333632303230323032303230344335343745334233333734313230302661706f733b3c2f756e6971756549643e3c6d65737361676549443e36363731383c2f6d65737361676549443e3c7265737562436f756e7465723e303c2f7265737562436f756e7465723e3c2f64657363726970746f723e3c62617463683e3c6261746368436f756e743e323c2f6261746368436f756e743e3c626174636849643e4d574e54444a47323030323031323331363132313332353c2f626174636849643e3c2f62617463683e3c2f6d6573736167653e00
v2.1 =
3c6d6573736167653e3c64657363726970746f723e3c6d657373616765547970653e4a4552454d592e544553543c2f6d657373616765547970653e3c6572726f72436f64653e31323334353c2f6572726f72436f64653e3c6572726f724465736372697074696f6e3e536f6d657468696e67207265616c6c7920626164206861732068617070656e65643c2f6572726f724465736372697074696f6e3e3c6661696c75726554696d657374616d703e323030322d30362d30322031343a30303a30312e3030303c2f6661696c75726554696d657374616d703e3c696e70757451756575653e424c41434b484f4c453c2f696e70757451756575653e3c756e6971756549643e5827343134443531323034443537344535343434333033363230323032303230323034433534374533423333373431323030273c2f756e6971756549643e3c6d65737361676549443e36363731353c2f6d65737361676549443e3c7265737562436f756e7465723e303c2f7265737562436f756e7465723e3c2f64657363726970746f723e3c62617463683e3c6261746368436f756e743e323c2f6261746368436f756e743e3c626174636849643e4d574e54444a47323030323031323331363132313332353c2f626174636849643e3c2f62617463683e3c2f6d6573736167653e00
Has anyone else experienced similar issues? Is it a bug or a feature?
Thanks,
J. |
|
| Back to top |
|
 |
| jgooch |
| Posted: Mon Mar 17, 2003 3:44 am Post subject: Solved (well, nearly). |
|
|
Acolyte
Joined: 29 May 2002
Posts: 63
Location: UK
|
IBM advised me that the manner in which the XML parser works has changed. It's a bit of a long reply from them, but worth reading:-
| Quote: |
The customers observation is correct in that the behaviour of the parsers has changed under some circumstances in WMQI V2.1. These changes are summarised in the WMQI V2.1 CSD03 version of the Working With Messages book. This text can be found in 'Chapter 2. Parsers' in the 'Using the supplied input nodes and parsers' sub-section.
.
For performance reasons in WMQI V2.1, the built-in parsers have the following attributes :
.
1) A message bitstream is not parsed until an actual reference is made to a message tree field within the message body. This means if a message goes through a message flow without being referenced, an invalid bitstream can successfully pass right the flow.
.
2) If a message tree field is referenced, then the bitstream is only parsed up to that field. This is known as partial parsing and means that if only the first few fields in a message are referenced, the parser does not need to create the whole of the message tree in memory. As per (1), if a bitstream is not valid at a point where the parser has not parsed this segment, then the error will not be detected.
.
3) The parsers maintain a record of whether the message tree has been changed throughout the message flow. Irrespective of partial or full parsing, the message tree records whether the user has manipulated/changed the fields within the message tree. At points in the flow, the owning parser may be requested to generate a bitstream based on the current message tree. This could be as a result of an MQOutput node, the ASBITSTREAM function or a ResetContentDescriptor node. If the parser knows that the user has not changed the message tree in anyway, then for performance reasons, it does not create a new message bitstream and uses the one it received on Input. Therefore, the output message created will be the same as input, as though the parser had never processed it.
.
4) Extending on (3) the XML parser has the capability to detect changes at the tag/field level. The message tree can be altered during the message flow, and at some point the parser may be requested to generate a new bitstream. It will detected that the message tree has changed and then start rendering a new output bitstream based on the current message tree. However, if the XML parser finds that a tag/field was created by the input message and has been unchanged, then it will re-use the segment from the input bitstream instead of generating the <fieldName>fieldValue</fieldName> bitstream from the messag tree.
.
All these modifications were made in WMQI V2.1 to enhance the performance of the parsers, and therefore increasing the throughput of message flows. In MQSI V2.01, a full parse was driven regardless, and a new bitstream was generated regardless of what has occured in the message flow.
.
Now we apply this to the customers scenario .....
.
Firstly, although the &pos; entity reference exists in XML, most XML parsers can tolerate the direct use of an apostrophe character in the message. Entity references exist for those characters that could be treated as mark-up characters in the XML language. However, the positioning of the character in the XML message can determine whether the character is part of the mark-up or the message data. For example, the forward slash character is used as part of the markup language in end tags, but always has to be partnered by a less than symbol. So if a forward slash appears without a direct preceeding less than, then the forward slash is not considered part of the markup language. However, some XML parsers, may not have the capabilities to make such determinations. Therefore, in a application integration environment it is safest to use the &pos; entity reference to represent the apostrophe sign.
.
The WMQI XML parser can process both the apostrophe character and the &pos; entity reference such that an apostrophe character will be created for both types in the logical message tree.
.
If the user creates a field in the message tree that contains the apostrophe character, then if the XML parser renders a bitstream from this message tree, then it will create &pos; in the output message.
.
However, based on (1) to (4) above, if the user passes in an XML message that contains the apostrophe character, then when the XML parser re-uses segments (all the whole) of the input message bitstream, the apostrophe character will be written in the output message bitstream generated.
.
Looking at the customers input message, they are passing in an XML message containing an apostrophe character. Therefore, when the XML parser generates a bitstream to pass to the BLOB parser in the ResetContentDescriptor, it passes the one from the input message to the flow. Therefore the users apostrophe character is passed to the BLOB domain as a single character, and not the &pos; representation. |
So, our concern moves onto:-
a) how to (ahem) "fudge" a change in the XML message such that entity reference substitutions are made wherever needed. Alternatively, we re-write the flow entirely (in which case, we'll still have to cross the "data with apostophes" bridge since it'll cause problems when you run the SQL insert statement);
b) which other flows' behaviour (we have 300) will have changed in 2.1? I guess we're going to have to retest all of them for comfort.
J. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
