| Author |
Message
|
| siljcjose |
 Posted: Tue Oct 26, 2010 11:37 pm Post subject: Code page Issue on Unix Message Broker 6.0 French Chars |
 |
|
Apprentice
Joined: 18 Aug 2005
Posts: 27
|
Hi,
I have a flow which calls a webservice for PDF generation. I get the message from a mainframe system CCSID is 819 and Encoding 273. I am getting a message that has French characters and sends the same to webservice.
The Webservice replies with the PDF in a XML tag. I use the CAST function for the field and cast it as BLOB and send to a queue.
When I call the webservice from my local broker ( Windows environment) I get a PDF generated and the French characters looks fine in PDF.
When In do the Same from a Unix Broker , the French charactesr are lost in PDF. Any Idea on what codepage encoding I should use for Unix.
Thanks |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Wed Oct 27, 2010 3:48 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
If I recall correctly CCSID 819 does not include all the French Character set.
the oe (the one character version) is at least one of the things missing.
CCSID 923 has these missing character.
Oh, it also has the Euro sign as well. That is missing from 819
819 == ISO 8859-1
923 == ISO 8859-15
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| siljcjose |
| Posted: Wed Oct 27, 2010 4:12 am Post subject: |
|
|
Apprentice
Joined: 18 Aug 2005
Posts: 27
|
| I tried chandinging to 923 still the same problem. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Oct 27, 2010 4:38 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| siljcjose wrote: |
| I tried chandinging to 923 still the same problem. |
Changing it how? If you're just overriding the CCSID value from 819 to 923 the missing characters are unlikely to appear.
You really need the message to be sent in a code page that supports all the needed characters.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Wed Oct 27, 2010 9:10 am Post subject: Re: Code page Issue on Unix Message Broker 6.0 French Chars |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| siljcjose wrote: |
| I get the message from a mainframe system CCSID is 819 and Encoding 273. I am getting a message that has French characters and sends the same to webservice. |
Which French characters, exactly?
| siljcjose wrote: |
When I call the webservice from my local broker (Windows environment) I get a PDF generated and the French characters looks fine in PDF.
When In do the Same from a Unix Broker, the French charactesr are lost in PDF. |
Try to narrow down where the characters are getting lost. Mainframe output message? Broker request message? Web service reply message? Broker output message?
| siljcjose wrote: |
| Any Idea on what codepage encoding I should use for Unix. |
For which message? |
|
| Back to top |
|
|
| paranoid221 |
| Posted: Wed Oct 27, 2010 9:24 am Post subject: |
|
|
Centurion
Joined: 03 Apr 2006
Posts: 101
Location: USA
|
| Code: |
| I get the message from a mainframe system CCSID is 819 and Encoding 273 |
Are you absolutely sure about those values? 819 is the CCSID for AIX platform. I believe 500 would be CCSID for z/os systems. I'm open to being corrected though
_________________
LIFE is a series of complex calculations, somewhere multiplied by ZERO. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Oct 27, 2010 12:09 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| paranoid221 wrote: |
| Code: |
| I get the message from a mainframe system CCSID is 819 and Encoding 273 |
Are you absolutely sure about those values? 819 is the CCSID for AIX platform. I believe 500 would be CCSID for z/os systems. I'm open to being corrected though |
Make sure you do no conversion, neither on the MF.to.broker channel, nor on the MQInput node...
What encoding do you use for the WEB call?
Put ccsid 1208 into the OutputRoot.Properties before the web call and state the XML encoding to be UTF-8. What do you get in return?
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| siljcjose |
| Posted: Wed Oct 27, 2010 8:00 pm Post subject: |
|
|
Apprentice
Joined: 18 Aug 2005
Posts: 27
|
There is a converion happening in the MQ channel from CCSID 297 to 819. When I inspect the message in the input queue, the French chracters "é à " looks fine. And then a webservice call is made with Code Page 1208. The Webservice call returns invlaid characters in Unix broker.
The most important thing to note is that I get the same set up working when I deploy the same to a windows broker it works . On a UNix broker the characters are lost. SO I am missing something on the Unix broker. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Oct 27, 2010 8:10 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| siljcjose wrote: |
There is a converion happening in the MQ channel from CCSID 297 to 819. When I inspect the message in the input queue, the French chracters "é à " looks fine. And then a webservice call is made with Code Page 1208. The Webservice call returns invlaid characters in Unix broker.
The most important thing to note is that I get the same set up working when I deploy the same to a windows broker it works . On a UNix broker the characters are lost. SO I am missing something on the Unix broker. |
What you are not realizing is the impact of CCSID conversion. If the source character does not exist in the target CCSID it gets replaced by a "dummy character" that will then show up as an invalid character when parsing.
Inspecting the message in the Input queue does not cover it if you use convert in the MQInput node... You should really look at the message CCSID and the hex values of the string. They must match. Disable the conversion checkbox on the MQInput node.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sat Oct 30, 2010 10:03 am Post subject: Re: Code page Issue on Unix Message Broker 6.0 French Chars |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| siljcjose wrote: |
| There is a conversion happening in the MQ channel from CCSID 297 to 819. |
The best practice is to let the receiving applications perform any conversions, and only resort to channel conversion when the receiving application can't.
| siljcjose wrote: |
| When I inspect the message in the input queue, the French characters "é à " looks fine. |
Please provide more details. Inspect, how? What does "looks fine" mean?
| siljcjose wrote: |
| And then a webservice call is made with Code Page 1208. The Webservice call returns invalid characters in Unix broker. |
Please provide more details here, too. What made the web service call? What converted the input message to code page 1208? What do the characters look like in the web service request? Which invalid characters does the web service call return?
| siljcjose wrote: |
| The most important thing to note is that I get the same set up working when I deploy the same to a windows broker it works . On a Unix broker the characters are lost. SO I am missing something on the Unix broker. |
Compare the web service response message in the windows broker with the web service response message in the Unix broker, including the headers. Describe any differences between the two.
| fjb_saper wrote: |
| If the source character does not exist in the target CCSID it gets replaced by a "dummy character" that will then show up as an invalid character when parsing. |
That's generally true, but not usually an issue when converting from ccsid=297 to ccsid=819, as they use the same character set. Converting from ccsid=819 to ccsid=1208 is also loss-less. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Oct 30, 2010 1:09 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| rekarm01 wrote: |
| That's generally true, but not usually an issue when converting from ccsid=297 to ccsid=819, as they use the same character set. Converting from ccsid=819 to ccsid=1208 is also loss-less. |
That may well be right. However I would have expected the original CCSID on windows to be 850... Don't know either that the ccsid 297 mentioned by the OP is the correct one for the content... So when the message is being given in 819 it is most probably already corrupt. The only way to uncover this is to follow the changes of CCSID upstream.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sat Oct 30, 2010 5:03 pm Post subject: Re: Code page Issue on Unix Message Broker 6.0 French Chars |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| fjb_saper wrote: |
| However I would have expected the original CCSID on windows to be 850 ... Don't know either that the ccsid 297 mentioned by the OP is the correct one for the content... So when the message is being given in 819 it is most probably already corrupt. |
Probably. So far, the OP has skipped a few details.
| fjb_saper wrote: |
| The only way to uncover this is to follow the changes of CCSID upstream. |
Agreed. Follow the ccsids, and the (hex) byte values too, step by step. |
|
| Back to top |
|
|
|
|
