| Author |
Message
|
| raghav |
 Posted: Wed Jul 13, 2005 12:40 pm Post subject: Unable to Process Special characters in MQManager CCSID(819) |
 |
|
Novice
Joined: 15 Apr 2005
Posts: 24
|
Hi,
I am able to observe a problem with my MQManager. The CCSID was set to 819 earlier, during this time when a message is sent from other Queue Managers the character € was not processed. So, as per some suggestions i changed the CCSID to 1208. Now i am able to find that there was no error thrown. But still the characters are modified before it is stored into the DB.
For example, a string "ohne Worte EUR € AOU ÄÖÜ aou äöü ßßß" when processed is stored as "ohne Worte EUR Ç AOU ?Í? aou õ÷³ ???" in the DB.
What could be the problem here. Should the CCSID be changed to some other value? Or is there some other problem here?
While getting the message i have the following options set,
MQGetMessageOptions getMesgOptions = new MQGetMessageOptions();
getMesgOptions.matchOptions = MQC.MQMO_NONE;
getMesgOptions.options = MQC.MQGMO_CONVERT
+ MQC.MQGMO_SYNCPOINT
+ MQC.MQGMO_WAIT
+ MQC.MQGMO_FAIL_IF_QUIESCING;
Please help me with your valuable comments.
Thank you. |
|
| Back to top |
|
 |
| Nigelg |
| Posted: Wed Jul 13, 2005 11:33 pm Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
| Code page 819 does not contain the Euro symbol; the equivalent CCSID with the euro is 923. |
|
| Back to top |
|
|
| sebastianhirt |
| Posted: Thu Jul 14, 2005 12:58 am Post subject: |
|
|
Yatiri
Joined: 07 Jun 2004
Posts: 620
Location: Germany
|
And a set of codepages and there layouts.
http://www.microsoft.com/typography/unicode/cscp.htm
You can also try 1252(Latin 1) (in case923 does not give you the wished result).
I find all those different character sets are quiet a pain. I really wish everybody would and could either use UTF8 or UTF16.
cheers
Sebastian |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Jul 14, 2005 3:50 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Or use xml and specify the encoding....
Than your xml reader should be able to make the conversion...
Enjoy  |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Jul 14, 2005 3:56 am Post subject: Re: Unable to Process Special characters in MQManager CCSID( |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| raghav wrote: |
| But still the characters are modified before it is stored into the DB. |
Maybe the DB is doing the conversion. Are you sure it is expecting that the characters will be 1208 (Unicode), and not something else?
What is the DB? What platform is it running on?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| raghav |
| Posted: Thu Jul 14, 2005 6:41 am Post subject: |
|
|
Novice
Joined: 15 Apr 2005
Posts: 24
|
| Database used is DB2 V8...running on the AIX V5 platform... |
|
| Back to top |
|
|
| tillywern |
| Posted: Fri Jul 15, 2005 10:11 am Post subject: Special Characters are crazy |
|
|
Centurion
Joined: 28 Jan 2003
Posts: 109
Location: Colorado
|
My current client has done extensive testing on this problem and I think it can be summarized in the following way.
There is a UTF-8 patch for broker to allow support for this. You will find that in interacting with databases this works fine. Both input and output. However once windows gets involved it gets messy.
As I recall rfhutil can't display it and so it shows up in the hex as 7F.. However saving to a file and display in notepad will work if the correct windows character set is used.
In short. ISO even has a spec that states that you should mark symbols like this up. I think the code for euro is EUR or something.
So to summarize. As far as I know MQ handles it. But applications reading the data provided by MQ may not understand what they are seeing.
If you are looking at database data you might convert to blob and then dump the hex to see if it is really a problem in the databse or in the application accessing it. |
|
| Back to top |
|
|
| raghav |
| Posted: Wed Sep 21, 2005 7:15 am Post subject: The response seems to be not going in UTF-8 encoding.... |
|
|
Novice
Joined: 15 Apr 2005
Posts: 24
|
Hi,
There were lots of advancements with this problem now.
When the special characters were not processed, Firstly we identified that the CCSID of our system was 819 (default) and the CCSID for the external system was 1208.
We modified our CCSID for the QManager to 1208.
And then we observed that the special characters coming in to be fine (like ä, ö, ü etc..). But still "€" was shown as "?". Then we planned to transfer the messages in Binary format. We implemented this change.
Now the external system gets the message in Binary format, but they say that the message doesnot come in with UTF-8 encoding but ISO-8859-1 (Latin-1).
I cannot understand why it should go like that when we properly encode it as UTF-8 during the XML message building.
some thing like,
import org.apache.xml.serialize.OutputFormat;
OutputFormat outputFormat = new OutputFormat("XML", "UTF-8", true);
etc...
Then we thought that there is something else that cud change the encoding. But we are not able to find anything from our analysis. Can some one help to know what cud be the problem here which changes the encoding to something else than UTF-8.
From some documents we came to know that changing a setting in WAS would help with this case, so we changed as follows:
(1) -Dclient.encoding.override=UTF-8
(2) Also we modified the file "ibm-web-ext.xmi" with the options "autoRequestEncoding" and "autoResponseEncoding" as "true".
But this doesn't work.
But again my concern is that the above changes could impact the application (the online part involving the Ear file). But we have an Interface program (with an independant Jar file placed in the environment path) which processed the messages. Even this processing puts messages back into the Queue which is found to be of a different encoding than "UTF-8" on the other end.
Please give your valuable comments. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Sep 21, 2005 3:50 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Like I said make sure you store it in XML format and specify the encoding.
It need not be UTF-8.
You can specify ISO-8859-1. Just make sure that the encoding you specify on the XML supports the special chars. They may come out different: the ß may show up in XML as &sz but that is o.k. A good parser should be able to render the right form.... providing the ccsid your app runs in supports it.
Enjoy |
|
| Back to top |
|
|
|
|
