| Author |
Message
|
| KIT_INC |
 Posted: Mon Jan 16, 2012 7:26 am Post subject: Broker not doing Data conversion |
 |
|
Knight
Joined: 25 Aug 2006
Posts: 589
|
I am using WMB 61 on AIX and need to convert an XML message which contains some French character from CCSID 1208 to CCSID 819.
The Input message
| Code: |
<Myitem>
<Item itemID="1010412" itemName="Duct Tape 1.5†x 30’" ></Item>
</Myitem> |
I did 2 tests
Test 1.
MQINPUT - COMPUTE - MQOUTPUT (Convert using MQ)
Receive message as BLOB
MQInput node has convert checked and CCSID 819 specified
Compute node is just Copy entire message
SET OutputRoot = InputRoot;
MQPUT using RFHUTIL with CCSID 1208 and MQSTR set
The ouptut looks like
| Code: |
<?xml version="1.0" encoding="utf-8"?>
<Myitem>
<Item itemID="1010412" itemName="Duct Tape 1.5 x 30" ></Item>
</Myitem> |
Test 2
MQINPUT - COMPUTE - MQOUTPUT (Convert using broker)
Receive message as BLOB
MQInput node has convert unchecked
Compute node is has ESQL
SET OutputRoot.MQMD = InputRoot.MQMD;
SET OutputRoot.MQMD.CodedCharSetId = 819;
SET OutputRoot.BLOB.BLOB = InputRoot.BLOB.BLOB;
MQPUT using RFHUTIL with CCSID 1208 and MQSTR set
The output looks like
| Code: |
<Myitem>
<Item itemID="1010412" itemName="Duct Tape 1.5†x 30’" ></Item>
</Myitem> |
It seems that conversion is not happening.
Is there anything more I have to do to get the broker to do the conversion ? |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Mon Jan 16, 2012 7:37 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
First you need to make sure that all the characters in the UTF-8 message can be converted to CCSID 819...
Second you have to realize that when using the BLOB domain no conversion occurs --- ever. If you want to convert, you first need to transform (cast) the BLOB to CHARACTER using the source CCSID and then back to BLOB in the destination CCSID.
Do not use the convert flag on the MQInput node... Let the broker do the conversions for you...
However the real question here is WHY do you want to go from UTF-8 to CCSID 819 i.e. a CCSID that does not support all the characters supported in UTF-8? I would have thought you'd like to keep it all in UTF-8...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| KIT_INC |
| Posted: Mon Jan 16, 2012 8:02 am Post subject: |
|
|
Knight
Joined: 25 Aug 2006
Posts: 589
|
Thanks fjb_saper for reminding me "Second you have to realize that when using the BLOB domain no conversion occurs ". I read about that sometime ago and it was paged out of my memory.
The flow was 1208 all way before and everything is fine
The requiremnt came up when some users trying to view the data with some old tools that only use CCSID 819 and got what they call 'garbage'
So in order to make them happy, so we tried.
We realized that 1208 has characters that do not exist in 819 and we did another tests like this as you suggested
DECLARE INBLOB, OUTBLOB BLOB;
DECLARE OUTCHAR CHAR;
SET INBLOB = InputRoot.BLOB.BLOB;
SET OUTCHAR = CAST(INBLOB AS CHAR CCSID 1208);
SET OUTBLOB = CAST (OUTCHAR AS BLOB CCSID 819);
SET OutputRoot.BLOB.BLOB = OUTBLOB;
and get failure at
SET OUTBLOB = CAST (OUTCHAR AS BLOB CCSID 819);
when the character does not exist in 819. The Input message used here will fail.
However, if we use MQ to convert. It will give us the output as mentioned.
It seems that MQ is more forgiving. We really do not want to change our flow if we can have the broker behave the same way as MQ. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jan 16, 2012 8:53 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
MQ is not more forgiving, it just doesn't care...
It will transform a non existing char into a substitution char but this will still make the XML an invalid XML as the substitution char is not a legal XML char.
The broker just catches this error and alerts you to it.
Keep the message UTF-8 and tell your team with the bad viewing tools to just look at the hex values... The hex values are right! If the tool has no UTF-8 support... not your problem...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Jan 16, 2012 1:00 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
The requiremnt came up when some users trying to view the data with some old tools that only use CCSID 819 and got what they call 'garbage'
So in order to make them happy, so we tried. |
I'm 100% with fjb_saper on this one. Don't downgrade your backbone messaging system. Especially do not downgrade it to use a very limited single-byte code page unless you know for sure that you will never need to support characters outside that code page. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Jan 16, 2012 1:13 pm Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Why not upgrade your tooling to support modern code pages?? Circa 1980s code pages are so vogue these days....
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Tue Jan 17, 2012 1:35 am Post subject: Re: Broker not doing Data conversion |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| KIT_INC wrote: |
I am using WMB 61 on AIX and need to convert an XML message which contains some French character from CCSID 1208 to CCSID 819.
The Input message
| Code: |
<Myitem>
<Item itemID="1010412" itemName="Duct Tape 1.5†x 30’"></Item>
</Myitem> |
|
It looks like the input message is garbled already. Is it supposed to look more like this?
| Code: |
<Myitem>
<Item itemID="1010412" itemName="Duct Tape 1.5” x 30’"></Item>
</Myitem> |
Either way, ccsid 819 doesn't support the following characters: € ™ ” ’ |
|
| Back to top |
|
|
|
|
