| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Problem in getting Japanese Chars from MQ message |
« View previous topic :: View next topic » |
| Author |
Message
|
| vineetonline |
 Posted: Mon Feb 22, 2010 6:18 pm Post subject: Problem in getting Japanese Chars from MQ message |
 |
|
Newbie
Joined: 22 Feb 2010
Posts: 5
|
Hello people,
Facing a problem while getting a message with Japanese text in it. Would give you a brief background.
1. Application Server migrated from 9i (jdk 1.3) to 10g (jdk 1.5)
2. AIX Server upgrade from Solaris 8 to Solaris 10.
3. MQ server upgrade from 5 to 7.0.1
Using the below mentioned method to get the MQ message. The japanese characters are got correctly in 9i version but they give junk values in 10g. The CCSID for MQ server is 943 (Shift_JIS).
| Code: |
public String syncMQGet(String pstrGetQueueName, int pintWaitInterval, String pstrCorrelId) {
String strCorrelId = pstrCorrelId;
try {
connectGetQueue(pstrGetQueueName);
oGetMsgOpt = new MQGetMessageOptions();
oGetMsgOpt.options = MQC.MQGMO_FAIL_IF_QUIESCING | MQC.MQGMO_WAIT;
oGetMsgOpt.matchOptions = MQC.MQMO_MATCH_CORREL_ID;
oGetMsgOpt.waitInterval=pintWaitInterval*1000;
oGetMessage = new MQMessage();
oGetMessage.correlationId=strCorrelId.getBytes();;
oGetQueue.get(oGetMessage,oGetMsgOpt);
byte bResponse[] = new byte[oGetMessage.getDataLength()];
oGetMessage.readFully(bResponse);
str_Message = new String(bResponse); // also tried as str_Message = new String(bResponse,"Shift_JIS") without success
bResponse = null;
} catch(MQException m) {
if(m.reasonCode == 2033){
} else {
}
} catch(Exception e) {
}
return str_Message;
}
|
Please suggest. Have a sneaky feeling this has something to do with different jdk versions(I may be very wrong) but cannot pin point the issue.
Thanks.
Last edited by vineetonline on Mon Feb 22, 2010 9:11 pm; edited 1 time in total |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Mon Feb 22, 2010 9:11 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Something bizarre. Where did you get the notion that the correlID was a String???
So you are not doing a get with convert. But neither are you setting the CCSID of the received message on the reply message! And you are passing the bytes from the received message to the sent message.
You could be suffering from a CCSID change. You may be setting your bytes right but changing the CCSID of the message between the get and the put.
Remember a byte[] has only a String meaning if there is a CCSID to go with it.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| vineetonline |
| Posted: Mon Feb 22, 2010 9:33 pm Post subject: |
|
|
Newbie
Joined: 22 Feb 2010
Posts: 5
|
| fjb_saper wrote: |
Something bizarre. Where did you get the notion that the correlID was a String???
|
well the function call passes it as a string. Its a legacy code and has been in production since a long time.
| Quote: |
So you are not doing a get with convert.
|
well i did try with option MQC.MQGMO_CONVERT without success (if thats wat you meant)
| Quote: |
But neither are you setting the CCSID of the received message on the reply message!
And you are passing the bytes from the received message to the sent message.
|
could not get this. Am a noobie to MQ and I know it requires formal training but I am the only option we have got :|
| Quote: |
You could be suffering from a CCSID change. You may be setting your bytes right but changing the CCSID of the message between the get and the put.
|
On connecting to the QueueManager, oQMGR.getCharacterSet() gives 943.
| Quote: |
Remember a byte[] has only a String meaning if there is a CCSID to go with it.
Have fun |
Also the putmessage method is below if it gives any pointers. Am really stuck in the middle of nowhere.
| Code: |
public void putMessage(String pstrQueueMgrName, String pstrQueueName, String pstrMessage, int pintMessageExpiry, String pstrReplytoQueueName, String pstrCorrelId) {
int intMsgLen=0;
int iReport=0;
String strMessageID = "";
String strCorrelId=pstrCorrelId;
try {
oPutMsgOpt = new MQPutMessageOptions();
oPutMessage = new MQMessage();
if (pstrMessage.length()>0) {
intMsgLen=pstrMessage.length();
}
byte buffer[]=new byte[intMsgLen];
buffer=pstrMessage.getBytes();
oPutMessage.correlationId = strCorrelId.getBytes();
oPutMessage.expiry=pintMessageExpiry;
oPutMessage.characterSet= 1208;
oPutMessage.encoding = MQC.MQENC_NATIVE;
oPutMessage.replyToQueueName=pstrReplytoQueueName;
try {
oPutMessage.write(buffer);
} catch(IOException i) {
} try {
oPutQueue.put(oPutMessage,oPutMsgOpt);
} catch(MQException m) {
} try {
oQMGR.commit();
} catch(MQException m) {
} try {
byte bMsgID[];
bMsgID = oPutMessage.messageId ;
strMessageID = getHexMessageID(bMsgID);
} catch(Exception m) {
}
buffer = null;
} catch (Exception e ) {
}
}
|
|
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Feb 22, 2010 9:51 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
From your get code:
| Code: |
byte bResponse[] = new byte[oGetMessage.getDataLength()];
oGetMessage.readFully(bResponse);
str_Message = new String(bResponse); // also tried as str_Message = new String(bResponse,"Shift_JIS") without success |
So you are retrieving a byte[] and transforming it into a String, without indication of CCSID or java CHARSET see java.nio.* ...
This is already a disaster in making because there is no indication that the message you just read the bytes from was in the default CCSID of the platform running the java code.... In order to get the correct code points for your text you will have to specify a code page (java code page) corresponding to the CCSID of the message content to create your String... It would be easier to make sure that the message is defined with the right format (MQFMT_STRING) and ask for the message in CCSID 1208 with an MQGET with convert.
So now you do have a text / String returned by the MQGet that is automatically transformed from UTF-8 to the correct code points in java (unicode).
On the put, if your message is text, make sure you specify MQFMT_STRING as format. Also make sure that the content matches the CCSID of the message. This can be achieved easily be setting the CCSID on the qmgr object (IIRC in java base). In JMS I would set it on the connection factory or Destination.
Have fun, explore the samples
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Mon Feb 22, 2010 11:05 pm Post subject: Re: Problem in getting Japanese Chars from MQ message |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
MQMessage has readString/writeString methods, which convert bytes <--> chars using the given characterSet. There's no reason to deal with the message bytes directly:
| Code: |
// ...
oGetQueue.get(oGetMessage,oGetMsgOpt);
str_Message = oGetMessage.readStringOfByteLength(oGetMessage.getMessageLength());
// ... |
| Code: |
// ...
oPutMessage.characterSet= 1208;
// ...
try {
oPutMessage.writeString(pstrMessage);
}
// ... |
|
|
| Back to top |
|
|
| vineetonline |
| Posted: Tue Feb 23, 2010 12:05 am Post subject: Re: Problem in getting Japanese Chars from MQ message |
|
|
Newbie
Joined: 22 Feb 2010
Posts: 5
|
| rekarm01 wrote: |
MQMessage has readString/writeString methods, which convert bytes <--> chars using the given characterSet. There's no reason to deal with the message bytes directly:
| Code: |
// ...
oGetQueue.get(oGetMessage,oGetMsgOpt);
str_Message = oGetMessage.readStringOfByteLength(oGetMessage.getMessageLength());
// ... |
| Code: |
// ...
oPutMessage.characterSet= 1208;
// ...
try {
oPutMessage.writeString(pstrMessage);
}
// ... |
|
Thanks for the replies.
After changing the code as suggested, i got this exception on the below mentioned readstring :
Exception in syncMQGet() of SyncMQ.java : java.io.EOFException: MQJE086: End of file exception ('MQMessag2.readConvertedString()')
Also, note I used
| Code: |
str_Message = oGetMessage.readString(oGetMessage.getMessageLength());
|
as readStringOfByteLength doesnot exist in the jars which are currently present.
Any pointers? |
|
| Back to top |
|
|
| vineetonline |
| Posted: Tue Feb 23, 2010 1:36 am Post subject: |
|
|
Newbie
Joined: 22 Feb 2010
Posts: 5
|
| rekarm01 wrote: |
MQMessage has readString/writeString methods, which convert bytes <--> chars using the given characterSet. There's no reason to deal with the message bytes directly:
| Code: |
// ...
oGetQueue.get(oGetMessage,oGetMsgOpt);
str_Message = oGetMessage.readStringOfByteLength(oGetMessage.getMessageLength());
// ... |
| Code: |
// ...
oPutMessage.characterSet= 1208;
// ...
try {
oPutMessage.writeString(pstrMessage);
}
// ... |
|
ignore my above reply.. i updated the MQ jar files to the latest ones and used readStringOfByteLength. Still im getting question marks (junk characters) as the output. 
The methods now are:
| Code: |
public String syncMQGet(String pstrGetQueueName, int pintWaitInterval, String pstrCorrelId) {
String strCorrelId = pstrCorrelId;
try {
connectGetQueue(pstrGetQueueName);
oGetMsgOpt = new MQGetMessageOptions();
oGetMsgOpt.options = MQC.MQGMO_FAIL_IF_QUIESCING | MQC.MQGMO_WAIT;
oGetMsgOpt.matchOptions = MQC.MQMO_MATCH_CORREL_ID;
oGetMsgOpt.waitInterval=pintWaitInterval*1000;
oGetMessage = new MQMessage();
oGetMessage.correlationId=strCorrelId.getBytes();;
oGetQueue.get(oGetMessage,oGetMsgOpt);
str_Message = oGetMessage.readStringOfByteLength(oGetMessage.getMessageLength()); // Commented the below four lines to add this line
//byte bResponse[] = new byte[oGetMessage.getDataLength()];
//oGetMessage.readFully(bResponse);
//str_Message = new String(bResponse); // also tried as str_Message = new String(bResponse,"Shift_JIS") without success
//bResponse = null;
} catch(MQException m) {
if(m.reasonCode == 2033){
} else {
}
} catch(Exception e) {
}
return str_Message;
}
|
| Code: |
public void putMessage(String pstrQueueMgrName, String pstrQueueName, String pstrMessage, int pintMessageExpiry, String pstrReplytoQueueName, String pstrCorrelId) {
int intMsgLen=0;
int iReport=0;
String strMessageID = "";
String strCorrelId=pstrCorrelId;
try {
oPutMsgOpt = new MQPutMessageOptions();
oPutMessage = new MQMessage();
if (pstrMessage.length()>0) {
intMsgLen=pstrMessage.length();
}
byte buffer[]=new byte[intMsgLen];
buffer=pstrMessage.getBytes();
oPutMessage.correlationId = strCorrelId.getBytes();
oPutMessage.expiry=pintMessageExpiry;
oPutMessage.characterSet= 1208;
oPutMessage.encoding = MQC.MQENC_NATIVE;
oPutMessage.replyToQueueName=pstrReplytoQueueName;
try {
//oPutMessage.write(buffer); Commented and added the below line
oPutMessage.writeString(pstrMessage);
} catch(IOException i) {
} try {
oPutQueue.put(oPutMessage,oPutMsgOpt);
} catch(MQException m) {
} try {
oQMGR.commit();
} catch(MQException m) {
} try {
byte bMsgID[];
bMsgID = oPutMessage.messageId ;
strMessageID = getHexMessageID(bMsgID);
} catch(Exception m) {
}
buffer = null;
} catch (Exception e ) {
}
}
|
|
|
| Back to top |
|
|
| rekarm01 |
| Posted: Wed Feb 24, 2010 2:12 am Post subject: Re: Problem in getting Japanese Chars from MQ message |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| vineetonline wrote: |
| Still im getting question marks (junk characters) as the output. |
Either the message gets corrupted somewhere along the way, or the display tool is not displaying the output correctly, (or both).
What would be helpful is a bit more detail to describe what happens to the message at each step. Try to narrow down where the message might be getting corrupted.
Before consuming a message from a queue, examine the message headers and data more thoroughly, to confirm whether the message headers match the message data. Tools that can display message data in hexadecimal are useful for this, such as rfhutil, amqsbcg0, etc.
Assuming the message is valid, the syncMQGet() method can either get it off the queue and convert it correctly, or it can throw an Exception. Try adding some more exception handling code; it looks a bit sparse:
| vineetonline wrote: |
| Code: |
catch(MQException m)
{
if(m.reasonCode == 2033)
{}
else
{}
}
catch(Exception e)
{} |
|
For example, printStackTrace() might be useful, to at least report the details of an Exception.
Similarly, if the input String is valid, the putMessage() method can either convert it correctly and put it on a queue, or it can throw an Exception. Try adding some more exception handling code there, too. |
|
| Back to top |
|
|
| vineetonline |
| Posted: Wed Feb 24, 2010 6:10 pm Post subject: Re: Problem in getting Japanese Chars from MQ message |
|
|
Newbie
Joined: 22 Feb 2010
Posts: 5
|
| rekarm01 wrote: |
For example, printStackTrace() might be useful, to at least report the details of an Exception.
|
The exception handling is detailed. I removed some of the code from the exception blocks to make the published code concise. Also, not getting any exceptions.
| Quote: |
Before consuming a message from a queue, examine the message headers and data more thoroughly, to confirm whether the message headers match the message data.
|
Can you please elaborate on how this can be done?
Appreciate the help.
Regards. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Thu Feb 25, 2010 1:59 am Post subject: Re: Problem in getting Japanese Chars from MQ message |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| vineetonline wrote: |
| rekarm01 wrote: |
Before consuming a message from a queue, examine the message headers and data more thoroughly, to confirm whether the message headers match the message data.
|
Can you please elaborate on how this can be done? |
Stop the application that consumes messages from the queue. Browse the message using rfhutil, amqsbcg0, MO71 support pack, or comparable tool. Confirm that the message headers contain the expected values, particularly for the CodedCharSetId and Format fields.
If it is possible to configure the display device using the given ccsid, then display the message contents directly, and confirm that the message displays as expected.
If it is not possible to display the message contents directly, then display the bytes as hexadecimal, and look them up in the appropriate code pages. Confirm that the given ccsid will map the given bytes to the expected characters. It's usually not necessary to manually inspect all the bytes in a message; just enough bytes to confirm the correct ccsid for the characters in question.
The IBM globalization web site provides detailed information about IBM's Character Data Representation architecture, but it does not provide DBCS (double byte character set) code pages.
The Unicode web site provides code charts for looking up Unicode characters.
The ICU Project web site provides code pages for other ccsids, such as ibm-943.
If the message on the queue is bad, then the problem lies upstream. Otherwise, the problem lies downstream. Use a similar process to pinpoint where the corruption occurs, and try to identify the cause. |
|
| Back to top |
|
|
| bharathvn |
| Posted: Thu Mar 25, 2010 8:44 pm Post subject: |
|
|
Newbie
Joined: 25 Mar 2010
Posts: 6
|
I was facing a similer issue with my code.
I added the below line during intialization. and it worked!
| Code: |
| MQEnvironment.CCSID=1208; |
The platform that my queue manager is installed on is RedHat Linux.
let me know if this works.
Cheers
Bharath |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
