| Author |
Message
|
| satya2481 |
 Posted: Mon Aug 11, 2008 5:36 am Post subject: WMQ Java application Japanese and Korean characters issue |
 |
|
Disciple
Joined: 26 Apr 2007
Posts: 170
Location: Bengaluru
|
Hi All,
We are facing a problem when displaying the Japenese and Korean characters after reading from the MQ Queue.
The main functionality or flow of the message is like this.
BackEnd System puts a message on to queue --> Broker reads the message from queue and converts its structure sends this message with UTF-8 encoding in the message <?xml version="1.0" encoding="UTF-8"?> and MQMD.CodedCharSetId = 1208 to other queue --> Java application reads the message frmo the queue and displays the information
The message contains - Swedish, Korean and Japanese characters along with the English characters. When the Java application reads the message from the Broker output queue all the message and its content gets displayed correctly except the Japanese and Korean characters. Below is part of the code.
| Code: |
FileInputStream fis = new FileInputStream(file);
/*An InputStreamReader object converts the bytes from UTF-8 into Unicode*/
InputStreamReader isr = new InputStreamReader(fis, "UTF-8");
Reader in = new BufferedReader(isr);
StringBuffer buffer = new StringBuffer();
int ch;
while ((ch = in.read()) > -1) {
buffer.append((char)ch);
}
in.close();
|
Can any one tell how to handle such messages. Anything has to be set in the MQMD header before broker puts the message into queue? Or is there some problem with the Java application?
Thanks in advance
_________________
IBM Certified Solution Developer WebSphere Message Broker V6.0
IBM Certified System Administrator WebSphere MQ V6.0 |
|
| Back to top |
|
 |
| David.Partridge |
| Posted: Mon Aug 11, 2008 11:20 pm Post subject: |
|
|
Master
Joined: 28 Jun 2001
Posts: 249
|
The first and most obvious question is are you certain that the data in the file is in UTF-8 and that the data is correctly encoded?
Secondly, don't you need to do something to tell Java about the encoding of the input file - I seem to remember that unless you tell it the default is to treat all input files on "ascii" type platforms as being in iso8859-1.
Why are you reading the input stream one char at a time? There has to be a better way! Like using BufferedReader.readline() which will return a Java String object. You can always get at the StringBuffer under the String.
OK that's the producer end.
Now to the consumer - what CCSID is your Broker Queue Manager set to? When you get the message from the queue are you telling the QM that you DONT want it converted, or that you want it converted to CCSID 1208 (effectively a no-op as no conversion will be called).
I suspect that the problem is that the broker QM is not CCSID 1208 but is CCSID 819 (e.g.) and that the app is getting the message with data conversion to the QMGR CCSID.
_________________
Cheers,
David C. Partridge |
|
| Back to top |
|
|
| satya2481 |
| Posted: Tue Aug 12, 2008 12:18 am Post subject: |
|
|
Disciple
Joined: 26 Apr 2007
Posts: 170
Location: Bengaluru
|
Hi David,
Thank you for the reply.
I would like to explain in detail what we are doing. There is a legacy system where the data is collected and then it is moved into a file. This file will be sent to our team and we use this message to put into the Broker queue. We are using Rfhutilc to put this message into the Broker message flow input queue. When we open the file in rfhutil the characters looks like
| Code: |
| <title title="í’‹ 페달 ì¼€ì´ë¸”, 장착" lang="KR" /> |
From the lang attribute you can see that this are korean characters.
When I open this test file in internet explorer the characters looks like below
| Code: |
| <title title="풋 페달 케이블, 장착" lang="KR" /> |
The above message used by the Message flow and then sent to one more queue which is a remote queue pointing to a local queue from where a java application reads the message. The java application when reads this message and displays the characters some junk value will be displayed like rectagles.
Within the broker before sending to the output queue we are setting the UTF-8 encoding in the XML message. As well as setting the CCSID as 1208 in the MQMD header. The java application uses UTF-8 as the encoding to convert input stream, like the code shown below.
| Code: |
| InputStreamReader isr = new InputStreamReader(fis, "UTF-8"); |
We are not able to figure out where the problem lies.
| Code: |
| I suspect that the problem is that the broker QM is not CCSID 1208 but is CCSID 819 (e.g.) and that the app is getting the message with data conversion to the QMGR CCSID. |
Your point above means when the java reads the message from the queue then only it has to supply the conversion option? Do you mean above java code does not perform this? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Aug 12, 2008 3:21 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Be aware that RFHUtil is not well suited for CCSID, you might be better off writing a little java utility to post the message from the file.
Make sure if you use RFHUtil to put the message to the queue that the following parameters have been set and you need to reset them EVERY time you load a new file(read from file): Check your MQMD header and set the CCSID to 1208. Look at the hex values of the data and make sure that the content is right for CCSID 1208 in hex display...
Set the input node to request the message in CCSID 1208 by setting the CCSID on the MQInput node. You can use get with convert, but it should be irrelevant at this point...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| satya2481 |
| Posted: Wed Aug 13, 2008 10:49 pm Post subject: |
|
|
Disciple
Joined: 26 Apr 2007
Posts: 170
Location: Bengaluru
|
Thank you for the reply.
We are now able to figure out that there must be some problem with the configuration end in the Java application side.
The Broker output message actually sent to a Remote queue. Which is sent to customer queue managers Local queue. They call it as SIBus (I am not aware of this term) a JMS Object is defined for this queue. A Message Driven Bean (MDB) which is listening on this JMS Queue starts processing this message and sends it to the corresponding Java application. All this is done in App Server.
What we did was, a copy of the message which is sent to Remote queue also sent to Broker local queue. We asked the App Server team to check the message which is locally copied in the Broker queue, this message looks ok for them. Now they are trying to check what went wrong in between.
Thank You |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Aug 14, 2008 1:37 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| satya2481 wrote: |
Thank you for the reply.
We are now able to figure out that there must be some problem with the configuration end in the Java application side.
The Broker output message actually sent to a Remote queue. Which is sent to customer queue managers Local queue. They call it as SIBus (I am not aware of this term) a JMS Object is defined for this queue. A Message Driven Bean (MDB) which is listening on this JMS Queue starts processing this message and sends it to the corresponding Java application. All this is done in App Server.
What we did was, a copy of the message which is sent to Remote queue also sent to Broker local queue. We asked the App Server team to check the message which is locally copied in the Broker queue, this message looks ok for them. Now they are trying to check what went wrong in between.
Thank You |
Assuming your message is now correct in the broker... please ensure that the output message is CCSID 1208 by setting the OutputRoot.Properties for the CodedCharSetId to 1208... or setting the right value in the RFH header...
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
