| Author |
Message
|
| shalabh1976 |
 Posted: Fri Dec 23, 2005 3:25 am Post subject: 0D getting converted to 0A |
 |
|
Partisan
Joined: 18 Jul 2002
Posts: 381
Location: Gurgaon, India
|
Hi,
This is with reference to my post http://www.mqseries.net/phpBB2/viewtopic.php?t=26105
What is happening is that X'0D' is getting converted to X'0A' if X'0D' is part of any XML data. I have not seen this kind of behavior before.
What I have done here is to assign the result of a select clause to an environment variable which has been defined as :
CREATE FIRSTCHILD of Environment DOMAIN 'XML' Name 'Variables'.
Then I do a further select on this Environment Variable tree and then assign the result to an XML tag on the output.
When I check the value in the Environment using a File trace X'0D' remains as X'0D' but once the assignment is done to the XML tag the value changes. Any ideas ?
_________________
Shalabh
IBM Cert. WMB V6.0
IBM Cert. MQ V5.3 App. Prog.
IBM Cert. DB2 9 DB Associate |
|
| Back to top |
|
 |
| elvis_gn |
| Posted: Wed Dec 28, 2005 10:17 pm Post subject: |
|
|
Padawan
Joined: 08 Oct 2004
Posts: 1905
Location: Dubai
|
Hi shalabh1976,
You are using BLOB ?
What exactly is the value in the Environment, is it only X'0D' ....? or is X'0D' part of something bigger, I am hoping that you are outputting it in XML domain....What is the value there, i mean in XML ?
This could be a problem with the CCSID or encoding ?
Will need some more info, can you show us the trace and the outputted value..
Regards. |
|
| Back to top |
|
|
| shalabh1976 |
| Posted: Mon Jan 02, 2006 9:57 pm Post subject: |
|
|
Partisan
Joined: 18 Jul 2002
Posts: 381
Location: Gurgaon, India
|
Here is the input file:
| Code: |
<?xml version="1.0" encoding="iso-8859-1"?>
<Capability>
<Parameters/>
<DataSet Name="Export" RecordCount="1">
<Row>
<Field Name="ID">100889</Field>
<DataSet Name="CustomerInvoice" RecordCount="1">
<Row>
<Field Name="ShippingMode"/>
<DataSet Name="CustomerInvoiceLineItem" RecordCount="1">
<Row>
<Field Name="TrailingText">Knd Konto Beställare: B-O Andersson
VÃ¥r ref: Lena Jervill
05-06-07
13 st Pastasallad m lax inkl bröd och vatten
lev Isbergs g 2, Malmö
Knd Konto </Field>
</Row>
</DataSet>
</Row>
</DataSet>
</Row>
</DataSet>
</Capability>
|
Here is the code:
| Code: |
DECLARE IN_MajorLoop, IN_MinorLoop, IN_MajorLoopCnt, IN_MinorLoopCnt, IN_TransactionCount INTEGER;
SET IN_TransactionCount = 1;
--Create reference to Input
DECLARE InputRef REFERENCE TO InputRoot.XML.Capability.DataSet.Row.DataSet;
--Create reference to Output
CREATE FIELD OutputRoot.XML.M_INTFC_PROJ_MSG.MsgData;
DECLARE OutputRef REFERENCE TO OutputRoot.XML.M_INTFC_PROJ_MSG.MsgData;
Create FIRSTCHILD of Environment DOMAIN 'XML' Name 'PC001Variables';--Required to hold XML tree
--Find out Major Loop ( No. of rows in the input for our calculation)
SET IN_MajorLoop = CARDINALITY(InputRef.Row[]);
--Start Loop
SET IN_MajorLoopCnt = 1;
WHILE IN_MajorLoopCnt <= IN_MajorLoop DO
--Find out Minor Loop ( No. of rows for our secondary calculation)
SET Environment.PC001Variables[IN_MajorLoopCnt].Refere = (SELECT A.DataSet FROM InputRoot.XML.Capability.DataSet.Row.DataSet.Row[IN_MajorLoopCnt] AS A WHERE A.DataSet.(XML.Attribute)Name = 'CustomerInvoiceLineItem');
DECLARE LoopRef REFERENCE TO Environment.PC001Variables[IN_MajorLoopCnt].Refere.DataSet;
SET Environment.PC001Variables.values[IN_MajorLoopCnt] = CARDINALITY(LoopRef.Row[]);
SET IN_MinorLoop = CARDINALITY(LoopRef.Row[]);
--Start Inner Loop
SET IN_MinorLoopCnt = 1;
WHILE IN_MinorLoopCnt <= IN_MinorLoop DO
CREATE FIELD OutputRef.Transaction[IN_TransactionCount];
DECLARE TransRef REFERENCE TO OutputRef.Transaction[IN_TransactionCount];
SET Environment.PC001Variables.RES_USER5 = THE (SELECT ITEM FIELDVALUE(Y) FROM LoopRef.Row[IN_MinorLoopCnt].Field[] AS Y WHERE Y.(XML.Attribute)Name = 'TrailingText');
--SET TransRef.INTFC_PROJ_RES.RES_USER5 = THE (SELECT ITEM FIELDVALUE(Y) FROM LoopRef.Row[IN_MinorLoopCnt].Field[] AS Y WHERE Y.(XML.Attribute)Name = 'TrailingText');
SET TransRef.INTFC_PROJ_RES.RES_USER5 = Environment.PC001Variables.RES_USER5 ;
SET IN_TransactionCount = IN_TransactionCount + 1;
SET IN_MinorLoopCnt = IN_MinorLoopCnt + 1;
END WHILE;
SET Environment.PC001Variables.A[IN_MajorLoopCnt] = IN_MinorLoop;
SET IN_MajorLoopCnt = IN_MajorLoopCnt + 1;
END WHILE;
|
Here is the output:
| Code: |
<M_INTFC_PROJ_MSG>
<MsgData>
<Transaction>
<INTFC_PROJ_RES>
<RES_USER5>Knd Konto Beställare: B-O Andersson Vår ref: Lena Jervill 05-06-07 13 st Pastasallad m lax inkl bröd och vatten lev Isbergs g 2, Malmö Knd Konto </RES_USER5>
</INTFC_PROJ_RES>
</Transaction>
</MsgData>
</M_INTFC_PROJ_MSG> |
As you will see if the output is seen in Hex that the X'0d' in the XML field whose attribute is TrailingText is converted to X'0a' and that is why all the line feeds are missing.
_________________
Shalabh
IBM Cert. WMB V6.0
IBM Cert. MQ V5.3 App. Prog.
IBM Cert. DB2 9 DB Associate |
|
| Back to top |
|
|
| elvis_gn |
| Posted: Mon Jan 02, 2006 10:58 pm Post subject: |
|
|
Padawan
Joined: 08 Oct 2004
Posts: 1905
Location: Dubai
|
Hi shalabh1976,
Can you first check that the input, once in the flow has the value X'0D' and not X'0A'.....
Also, got to help--> Contents and search "line feed".
You can define White Space to preserve the <LF> in the input message.....But you will have to define a simple type first.
This should solve it, if not then this can also be an encoding problem.
Regards. |
|
| Back to top |
|
|
| shalabh1976 |
| Posted: Mon Jan 02, 2006 11:09 pm Post subject: |
|
|
Partisan
Joined: 18 Jul 2002
Posts: 381
Location: Gurgaon, India
|
I added a file trace to capture the value in the environment variable:
| Quote: |
(0x1000010)PC001Variables = (
(0x1000000)Refere = (
(0x1000000)DataSet = (
(0x3000000)Name = 'CustomerInvoiceLineItem'
(0x3000000)RecordCount = '1'
(0x2000000) = '
'
(0x1000000)Row = (
(0x2000000) = '
'
(0x1000000)Field = (
(0x3000000)Name = 'TrailingText'
(0x2000000) = 'Knd KontoáBestllare: B-O Andersson
Vr ref: Lena Jervill
05-06-07
13 st Pastasallad m lax inkl br÷d och vatten
lev Isbergs g 2, Malm÷
Knd Kontoá'
) |
As you can see the X'0d' is retained in the Env Var but is lost when assigned to the Output XML variable.
I am looking at other options you have mentioned.
_________________
Shalabh
IBM Cert. WMB V6.0
IBM Cert. MQ V5.3 App. Prog.
IBM Cert. DB2 9 DB Associate |
|
| Back to top |
|
|
| elvis_gn |
| Posted: Mon Jan 02, 2006 11:46 pm Post subject: |
|
|
Padawan
Joined: 08 Oct 2004
Posts: 1905
Location: Dubai
|
Hi shalabh1976,
The input is fine...did you get a trace of the output also....
If you are simply setting the value to output, the LF should not get affected until it is outputted...Hence we should be able to see the same trace in the trace node after your compute.
When it is sent from the output node, there is can be converting the LF to CR due to encoding/CCSID.
If you are using an ouptut message set, try the whiteSpace preserve as in the previous post.
Regards. |
|
| Back to top |
|
|
| shalabh1976 |
| Posted: Tue Jan 03, 2006 12:30 am Post subject: |
|
|
Partisan
Joined: 18 Jul 2002
Posts: 381
Location: Gurgaon, India
|
Here is the trace for the output:
| Quote: |
(0x1000010)XML = (
(0x1000000)M_INTFC_PROJ_MSG = (
(0x1000000)MsgData = (
(0x1000000)Transaction = (
(0x1000000)INTFC_PROJ_RES = (
(0x1000000)RES_USER5 = (
(0x2000000) = 'Knd KontoáBestllare: B-O Andersson
Vr ref: Lena Jervill
05-06-07
13 st Pastasallad m lax inkl br÷d och vatten
lev Isbergs g 2, Malm÷
Knd Kontoá'
)
)
)
)
)
)
) |
Surprise Surprise.............
In the trace the X'0d' is showing correctly.
When I either browse or get the message using RFHUTIL, MQ Explorer, API Exerciser the X'0d' shows as X'0a'.
I am not using a message set in the output.
_________________
Shalabh
IBM Cert. WMB V6.0
IBM Cert. MQ V5.3 App. Prog.
IBM Cert. DB2 9 DB Associate |
|
| Back to top |
|
|
| fschofer |
| Posted: Tue Jan 03, 2006 12:45 am Post subject: |
|
|
Knight
Joined: 02 Jul 2001
Posts: 524
Location: Mainz, Germany
|
Hi,
| Quote: |
| What is happening is that X'0D' is getting converted to X'0A' if X'0D' is part of any XML data |
this is the normal behavior of xml parsers.
| Quote: |
| An XML processor must normalize line-ends to LF before passing them to the application (2.11). |
=> http://www.w3.org/TR/xml-infoset/
Greetings
Frank |
|
| Back to top |
|
|
| elvis_gn |
| Posted: Tue Jan 03, 2006 12:59 am Post subject: |
|
|
Padawan
Joined: 08 Oct 2004
Posts: 1905
Location: Dubai
|
Hi shalabh1976,
When I looked at a message in the RFHUTIL I find that the field with LF appears as below
| Quote: |
| <Field Name="TrailingText">Knd Konto Beställare: B-O Andersson..Vår ref: Lena Jervill..05-06-07 ..13 st Pastasallad m lax inkl bröd och vatten ..lev Isbergs g 2, Malmö ..Knd Konto </Field> |
Also if I check the CR/LF option too, it looks the same, its only in Char option that the LF is understood...
Is this what you see too ? I think it is working in such a case 
Regards. |
|
| Back to top |
|
|
| shalabh1976 |
| Posted: Tue Jan 03, 2006 1:06 am Post subject: |
|
|
Partisan
Joined: 18 Jul 2002
Posts: 381
Location: Gurgaon, India
|
elvis:
If you see the data in RFHUTIL with the Data Format as Both you will find that ".." is showing as 0A0A instead of 0D0A(You can see this if you use RFHUTIL to view the input message)
fschofer:
Is there any way to prevent the WMQI XML parser from doing this normalization ?
_________________
Shalabh
IBM Cert. WMB V6.0
IBM Cert. MQ V5.3 App. Prog.
IBM Cert. DB2 9 DB Associate |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jan 03, 2006 1:16 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
shalabh1976,
No, there is no way to prevent normalization of line breaks, and neither should there be. WMQI is no different from any other correctly-behaving XML parser. You should refer to the XML specification: http://www.w3.org/TR/REC-xml/#sec-white-space.
| Quote: |
XML parsed entities are often stored in computer files which, for editing convenience, are organized into lines. These lines are typically separated by some combination of the characters CARRIAGE RETURN (#xD) and LINE FEED (#xA).
To simplify the tasks of applications, the XML processor MUST behave as if it normalized all line breaks in external parsed entities (including the document entity) on input, before parsing, by translating both the two-character sequence #xD #xA and any #xD that is not followed by #xA to a single #xA character.
|
|
|
| Back to top |
|
|
| shalabh1976 |
| Posted: Tue Jan 03, 2006 1:46 am Post subject: |
|
|
Partisan
Joined: 18 Jul 2002
Posts: 381
Location: Gurgaon, India
|
What this means that WMQI cannot handle this kind of input. I'll let the client know about this.
_________________
Shalabh
IBM Cert. WMB V6.0
IBM Cert. MQ V5.3 App. Prog.
IBM Cert. DB2 9 DB Associate |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Jan 03, 2006 4:14 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
That's technically correct, but rather unfair to WMQI. If your cliient had bought any other XML-processing application, it would do exactly the same.
I presume your client has an application which needs line breaks to be represented as 0x0D0A. I think your only option is to encode the text field as a CDATA section.
Alternatively, you could try using the AsIsElementContent type. I found this post which shows how http://www.mqseries.net/phpBB2/viewtopic.php?t=1436&highlight=xml+0x0a |
|
| Back to top |
|
|
| shalabh1976 |
| Posted: Tue Jan 03, 2006 8:36 pm Post subject: |
|
|
Partisan
Joined: 18 Jul 2002
Posts: 381
Location: Gurgaon, India
|
Neither CDATA nor AsisElementContent made any difference. The output is same in all cases.
That the output with CDATA does not change is already mentioned in the referenced post.
It does not however mention that using AsisElementContent makes no difference either.
_________________
Shalabh
IBM Cert. WMB V6.0
IBM Cert. MQ V5.3 App. Prog.
IBM Cert. DB2 9 DB Associate |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Jan 04, 2006 1:44 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
|
| Back to top |
|
|
|
|
