| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Problem Setting EBCDIC Encoded Text to CDATA Section |
« View previous topic :: View next topic » |
| Author |
Message
|
| dmx0t1 |
 Posted: Tue Feb 10, 2009 8:53 am Post subject: Problem Setting EBCDIC Encoded Text to CDATA Section |
 |
|
Apprentice
Joined: 23 Nov 2008
Posts: 27
|
Hi,
I have a scenario where I am required to put EBCDIC encoded text in the CDATA section under the xml message tree. I have the following code in my compute node:
DECLARE ebcdicHex BLOB X'C9C2E3E2D3C1F0F040F0F3F2F4F1F9F9F70AF1F0F2F5D3C4D3C3F1F6F5F5F0F90000000000000000002E';
SET OutputRoot.MQMD.CodedCharSetId = 1208;
SET OutputRoot.XMLNSC.Envelope.Body.(XMLNSC.CDataField)Message =
CAST(ebcdicHex AS CHARACTER CCSID 937);
The above code works fine and I get the expected output as follows:
| Code: |
<Envelope>
<Body>
<Message><![CDATA[IBTSLA00 03241997?1025LDLC165509]]></Message>
</Body>
</Envelope>
|
But if my message were altered to be like the following (note the difference marked red in comparison to the previous ebcdicHex variable:
DECLARE ebcdicHex BLOB X'C9C2E3E2D3C1F0F040F0F3F2F4F1F9F9F70EF1F0F2F5D3C4D3C3F1F6F5F5F0F90000000000000000002E';
I get the following exception:
| Code: |
RecoverableException
File:CHARACTER:F:\build\S610_P\src\CommonServices\ImbConverter.cpp
Line:INTEGER:516
Function:CHARACTER:ImbConverterCPP::internalToUnicode
Type:CHARACTER:
Name:CHARACTER:
Label:CHARACTER:
Catalog:CHARACTER:BIPv610
Severity:INTEGER:3
Number:INTEGER:2135
Text:CHARACTER:Unconvertable character
Insert
Type:INTEGER:5
Text:CHARACTER:f0
Insert
Type:INTEGER:5
Text:CHARACTER:c9c2e3e2d3c1f0f040f0f3f2f4f1f9f9f70ef1f0f2f5d3c4d3c3f1f6f5f5f0f90000000000000000002e
Insert
Type:INTEGER:2
|
Is there any way I can make this work? Your help is deeply appreciated 
Much Thanx in Advance~ |
|
| Back to top |
|
 |
| kimbert |
| Posted: Tue Feb 10, 2009 1:38 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Message broker obviously thinks that your second BLOB is not a valid stream of characters in code page 937. If you disagree, please explain why. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Feb 10, 2009 2:29 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Why are you sending the hex encoded text in CCSID 937 when your output in CCSID 1208?
Speacking about text if the recipient expects to get a String using
String(mybytes[]) it would work very poorly.
You have to understand that all characters are not equal under different CCSIDs. You are not transforming your blob by sending a hex interpretation of the bytes. you are sending the bytes as HEX... (BLOB). Now you could go through a whole routine to produce the equivalent characters (0->F) and send those and have the destination interpret them as well.
If you are sending your message as a text message expect them (BLOB) to be affected. And note that your ebcdic hex variable is not text. Hence the bunch of low values in there (0000000).
So 1st you need to represent your ebcdic hex as text representation of hex values for the bytes. Then you need that value to be in the CCSID of the destination, and finally you need the destination to restore the hex representation to the hex value....
Easier to send the message as bytes message with the Hex values unchanged.... or 64 bit encoded (mime)....
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Tue Feb 10, 2009 10:31 pm Post subject: Re: Problem Setting EBCDIC Encoded Text to CDATA Section |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
CCSID 937 is a mix of single-byte EBCDIC and double-byte Traditional Chinese, using X'0E' (Shift Out) and X'0F' (Shift In) to switch between the two.
Is that really the correct CCSID?
| dmx0t1 wrote: |
... I am required to put EBCDIC encoded text in the CDATA section under the xml message tree. ...:
DECLARE ebcdicHex BLOB X'C9C2E3E2D3C1F0F040F0F3F2F4F1F9F9F70AF1F0F2F5D3C4D3C3F1F6F5F5F0F90000000000000000002E';
SET OutputRoot.MQMD.CodedCharSetId = 1208;
SET OutputRoot.XMLNSC.Envelope.Body.(XMLNSC.CDataField)Message = CAST(ebcdicHex AS CHARACTER CCSID 937);
The above code works fine and I get the expected output as follows:
| Code: |
<Envelope>
<Body>
<Message><![CDATA[IBTSLA00 03241997?1025LDLC165509]]></Message>
</Body>
</Envelope> |
|
Works fine?  X'0A' should not look like a question mark.
X'0A', X'00', and X'2E' are EBCDIC control characters; they get converted to Unicode control characters. The XML standard doesn't allow them in CDATA segments.
Is that what was expected?
| dmx0t1 wrote: |
But if my message were altered to be like the following (note the difference marked red in comparison to the previous ebcdicHex variable:
DECLARE ebcdicHex BLOB X'C9C2E3E2D3C1F0F040F0F3F2F4F1F9F9F70EF1F0F2F5D3C4D3C3F1F6F5F5F0F90000000000000000002E';
I get the following exception:
| Code: |
RecoverableException
...
Text:CHARACTER:Unconvertable character
... |
|
X'0E' is the "Shift Out" character; it is expected to be followed by double-byte Chinese characters.
But there are no defined characters for X'F1F0', X'F2F5', X'F1F6', X'F5F5', X'F0F9', X'0000', and X'002E',
... which will cause an 'Unconvertable character' Exception. |
|
| Back to top |
|
|
| dmx0t1 |
| Posted: Wed Feb 11, 2009 6:19 pm Post subject: |
|
|
Apprentice
Joined: 23 Nov 2008
Posts: 27
|
| kimbert wrote: |
| Message broker obviously thinks that your second BLOB is not a valid stream of characters in code page 937. If you disagree, please explain why. |
Well, I was able to serialize the MRM tree via the ASBITSTREAM call, so I expected to get it back by deserializing it via the casting the blob to character...(correct me if i am wrong) |
|
| Back to top |
|
|
| dmx0t1 |
| Posted: Wed Feb 11, 2009 6:45 pm Post subject: |
|
|
Apprentice
Joined: 23 Nov 2008
Posts: 27
|
| fjb_saper wrote: |
| Why are you sending the hex encoded text in CCSID 937 when your output in CCSID 1208? |
Because I need to log all messages in/out the mainframe, and my logging application is written in Java and expects
a predefined XML structure with the UTF-8 encoding for parsing and insert the message in DB.
| fjb_saper wrote: |
You have to understand that all characters are not equal under different CCSIDs.
|
Yea, I understand, that X'0E' is actually the hex control character mainframe expects, I suppose that control character can't be casted properly into ascii?
| fjb_saper wrote: |
So 1st you need to represent your ebcdic hex as text representation of hex values for the bytes. Then you need that value to be in the CCSID of the destination |
But that control character only happens on the 2nd round trip, which is manipulated by the flow itself, so what I did was have the flow setting that field of MRM to X'0E'...e.g. (SET OuputRoot.MRM.Message.HEADER.CTL = X'0E'; ), which worked out fine, but crashed when I am attempting to log it.. 
Last edited by dmx0t1 on Wed Feb 11, 2009 7:32 pm; edited 2 times in total |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Feb 11, 2009 6:54 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Again to make sure you get everything right regardless you need to either
- log it as bytes / byte[] and then no problem with whatever values the single bytes in the byte[] might have. This is best done with a mime type multipart message or by 64 bit encoding the byte[]
- pass it as a hex representation of the byte[] which means you will get a char string of bytes.length * 2. You can then use CCSID translation and recreate the byte[] in the target system.
- Finally your byte[] does not translate into a string natively looking at the bytes as you have hex low values in there and other stuff like control characters and stuff. So a translation into a unicode CCSID may not be possible.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| dmx0t1 |
| Posted: Wed Feb 11, 2009 7:24 pm Post subject: Re: Problem Setting EBCDIC Encoded Text to CDATA Section |
|
|
Apprentice
Joined: 23 Nov 2008
Posts: 27
|
| rekarm01 wrote: |
| Works fine? X'0A' should not look like a question mark. |
Well, I am not so sure, but at least I didn't get any exception when the control character is X'0A'
| rekarm01 wrote: |
X'0A', X'00', and X'2E' are EBCDIC control characters; they get converted to Unicode control characters. The XML standard doesn't allow them in CDATA segments.
|
You are right, X'0A' and X'0E' are control characters in my message, they are modelled in the MRM.Message.HEADER.CTL field. I guess if XML doesn't allow this, I will log the binary as is then...
| rekarm01 wrote: |
X'0E' is the "Shift Out" character; it is expected to be followed by double-byte Chinese characters.
|
The system requires me using X'0A' for first round trip (set by the input message), and use X'0E' for subsequent round trip (set by the flow)
| rekarm01 wrote: |
But there are no defined characters for X'F1F0', X'F2F5', X'F1F6', X'F5F5', X'F0F9', X'0000', and X'002E',
... which will cause an 'Unconvertable character' Exception.
|
Thanx for the tip  , is there a reference URL I can refer to for further studies? |
|
| Back to top |
|
|
| dmx0t1 |
| Posted: Wed Feb 11, 2009 7:28 pm Post subject: |
|
|
Apprentice
Joined: 23 Nov 2008
Posts: 27
|
| fjb_saper wrote: |
Again to make sure you get everything right regardless you need to either
- pass it as a hex representation of the byte[] which means you will get a char string of bytes.length * 2. You can then use CCSID translation and recreate the byte[] in the target system.
|
Thanx for the tip, fjb_saper  I've opted option 2 |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sun Feb 15, 2009 6:59 pm Post subject: Re: Problem Setting EBCDIC Encoded Text to CDATA Section |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| dmx0t1 wrote: |
| Well, I was able to serialize the MRM tree via the ASBITSTREAM call, so I expected to get it back by deserializing it via the CASTing the BLOB to CHARACTER...(correct me if i am wrong) |
CAST does not "deserialize"; in the example provided, CAST converts serial character data from one CCSID to another.
In order for this to work as expected, both ASBITSTREAM and CAST would need to agree about the form of the underlying bitstream; if the CAST is expecting a bitstream containing only character data, then ASBITSTREAM would need to provide a bitstream containing only character data:
- the ASBITSTREAM's CCSID should match the CAST's CCSID
- all of the MRM field values should have CHARACTER base data type, (e.g. not INTEGER, nor FLOAT, nor even BLOB)
- for CCSIDs with multiple shift states, each MRM field value should start and end in the default shift state, (otherwise concatenating field values might not work)
If all of the above aren't true, then ASBITSTREAM might not be adequate to generate a suitable character string.
MRM.Message.HEADER.CTL appears to be a BLOB, not a CHARACTER. It seems to represent a flag (byte), rather than a specific abstract character, so it should probably not be subject to charset conversion. It's likely that the at least some of the tail end of ebcdicHex (X'000000000000000002E') do not represent characters either.
| dmx0t1 wrote: |
| fjb_saper wrote: |
| pass it as a hex representation of the byte[] which means you will get a char string of bytes.length * 2. You can then use CCSID translation and recreate the byte[] in the target system. |
Thanx for the tip, fjb_saper I've opted option 2 |
See also:
| fjb_saper wrote: |
| Finally your byte[] does not translate into a string natively looking at the bytes as you have hex low values in there and other stuff like control characters and stuff. So a translation into a unicode CCSID may not be possible. |
Even if some of this logic is moved from the message flow to the target system, the bitstream writer and bitstream reader still need to agree about the form of the underlying bitstream.
| dmx0t1 wrote: |
| is there a reference URL I can refer to for further studies? |
For more about what can and can't go in XML CDATA, look here: http://www.w3.org/TR/REC-xml/#sec-cdata-sect
For more about which CCSIDs to use, start here: http://www-01.ibm.com/software/globalization/g11n-res.jsp |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
