| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Issue with Special character after Code page changes |
« View previous topic :: View next topic » |
| Author |
Message
|
| Sathyambdev |
 Posted: Tue Jul 28, 2015 7:25 am Post subject: Issue with Special character after Code page changes |
 |
|
Newbie
Joined: 08 May 2015
Posts: 3
|
I have a IDOC message from SAP which has recently migrated to Unicode with code page 4103. Due to this change, I have also changed the Code page to 4103 in the Inbound SAP Adapter.
Now after this change, we are facing issues with the special characters. It adds space or another character after every special character.
During our analysis, I found something strange. I capture the message in a MQ queue after we receive it from the Adapter. When I store the message to a file using RFHUTIL including all the Headers, I see the issue with special character. But when I save it excluding the headers, it shows up fine.
| Code: |
With Headers it show as below
8400 Décarie H4P 2N2 Montréal BORÅS) BORÅS 33179600 (CDC BORÅS) MINILINK BORÅS |
| Code: |
Message saved excluding Headers
Décarie H4P 2N2 Montréal BORÅS) BORÅS 33179600 BORÅS) MI BORÅS |
Due to this, the next WTX component is not able to map all the segments properly. Is this a normal behavior or something wrong with rfhutil? |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Jul 28, 2015 7:42 am Post subject: Re: Issue with Special character after Code page changes |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Sathyambdev wrote: |
I have a IDOC message from SAP which has recently migrated to Unicode with code page 4103. Due to this change, I have also changed the Code page to 4103 in the Inbound SAP Adapter.
Now after this change, we are facing issues with the special characters. It adds space or another character after every special character. |
Yes. As indicated here, 4103 is the SAP number for a UTF-16 Little Endian character set. So double byte representation is to be expected. While I don't profess to know SAP that well, that link has a "Usual problems with encoding" section that seems relevant.
| Sathyambdev wrote: |
| Is this a normal behavior or something wrong with rfhutil? |
Normal. If you want more flexibility in viewing the message & it's headers, use a different tool.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Sathyambdev |
| Posted: Fri Jul 31, 2015 5:55 am Post subject: |
|
|
Newbie
Joined: 08 May 2015
Posts: 3
|
Vitor,
Thanks for the link.
From the external SAP system, I got the confirmation that their Code page is 4103. But when it lies in the MQ queue, the special character changes to Double byte.
I tried to store the message directly to a File and there I could see the special character looks fine.
So then Is there something to be changed in the queue level or message level?
I tried setting the Properties.CodedCharSetID to 1202 but still no luck.
I also tried to set the MQMD.CodedCharSetId to 1202 and then I got an exception "String too long to convert".
How can I convert this Double character(Décarie) to Single Character bytes(Décarie).? |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 31, 2015 6:23 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Sathyambdev wrote: |
| From the external SAP system, I got the confirmation that their Code page is 4103. But when it lies in the MQ queue, the special character changes to Double byte. |
No it doesn't. Unless I'm reading it wrong, 4103 is a double byte code page so it's not changing to double byte in the queue.
| Sathyambdev wrote: |
| I tried to store the message directly to a File and there I could see the special character looks fine. |
How did you unload the message into a file?
| Sathyambdev wrote: |
| So then Is there something to be changed in the queue level or message level? |
I repeat that I'm not a SAP expert by any means, but queues do not contain any CCSID information and the MQMD needs to represent the payload. So if the MQMD of the message on the queue correctly represents a CCSID that's Little Endian Double Byte Unicode you should be fine. The fact that whatever you used to unload the queue seems to understand that is a good sign.
| Sathyambdev wrote: |
I tried setting the Properties.CodedCharSetID to 1202 but still no luck.
I also tried to set the MQMD.CodedCharSetId to 1202 and then I got an exception "String too long to convert". |
Why 1202? 1203 seems to be a more logical choice. I also, as I indicate above, think the input message is properly described.
| Sathyambdev wrote: |
| How can I convert this Double character(Décarie) to Single Character bytes(Décarie).? |
Well to circle back to your original post, I find it slightly hard to believe that WTX (which eats SWIFT messages for breakfast and thinks bit level field mappings are "fun") can't map segments simply because they're presented in double byte Unicode. Having said that, you need to ensure that they're specifically converted from Unicode into a single byte code page that supports that character; I would suggest either 819 (ISO Latin) or 1010 (ISO French) are candidates, and would bet the file you unloaded the message into is 819 as that's a Windows default.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kimbert |
| Posted: Sat Aug 01, 2015 9:14 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
My advice would be to go back to the basics. Make sure that you understand what the message should look like right down to the individual bytes. Then check every stage in the processing of the message until you find out where the problem is.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
