| Author |
Message
|
| Mangesh1187 |
 Posted: Thu Feb 20, 2014 12:31 am Post subject: Chinease charecters not able to find it in queue |
 |
|
Centurion
Joined: 23 Mar 2013
Posts: 116
|
Hi All,
I am currently facing issue with Chinease charecter in MQ.
Application, ABC Send a message which containg chinease charecter fileds in the body.
But when I see the message in the target local queue, instead of contact admin charecters I can see some junk charecters instead. Other messages are fine.
The Application ABC, is using JMS and unsing UTF-8 encoding while putting the message.
Can you please provide me some check points/Hint/suggession so that I can start my incistigation. |
|
| Back to top |
|
 |
| exerk |
| Posted: Thu Feb 20, 2014 12:39 am Post subject: Re: Chinease charecters not able to find it in queue |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Mangesh1187 wrote: |
| But when I see the message in the target local queue, instead of contact admin charecters I can see some junk charecters instead. Other messages are fine. |
And just what are you using to examine the message? What is the format of the message, and is the application complaining about 'weird' characters?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| mqsiuser |
| Posted: Thu Feb 20, 2014 1:07 am Post subject: Re: Chinease charecters not able to find it in queue |
|
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
| Mangesh1187 wrote: |
Application, ABC Send a message which containg chinease charecter fileds in the body.
But when I see the message in the target local queue, instead of contact admin charecters I can see some junk charecters instead. Other messages are fine.
The Application ABC, is using JMS and unsing UTF-8 encoding while putting the message. |
Please use the proper CCSID when READING and WRITING msgs
_________________
Just use REFERENCEs |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Feb 20, 2014 2:14 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5849
Location: UK
|
Viewing the messages on a PC - can often display "junk" characters - these are simply characters that are not valid ASCII.
The same "problem" happens with EBCDIC - I am always getting developers telling me the data is corrupt.
It is not corrupt - it is simply unconverted but being viewed on a ASCII windows machine.
You can request conversion in most tools (like MO71 or IH03) and this will make the characters readable - assuming there is a valid correspondance between the original data and ASCII.
However conversion in tools, or in applications, will only work if the original message contains valid data for the codepage set in the MQMD - and if the conversion from that CCSID to your PC's codepage (usually ASCII) is supported.
Unless the tools support chinese character display - you may never see it as being "readable". But you can check the values in hexadecimal.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Feb 20, 2014 5:12 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9399
Location: US: west coast, almost. Otherwise, enroute.
|
Ponder this for a moment: Your PC and monitor can only display so-called displayable characters. Are an ASCII x'20' or EBCDIC x'40' displayable on your pc monitor?
You need to use a utility application that can display the hex equivalent of the character data.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Thu Feb 20, 2014 2:20 pm Post subject: |
|
|
Jedi
Joined: 25 Mar 2003
Posts: 2495
Location: Melbourne, Australia
|
If the messages contain Chinese characters encoded as UTF-8, you need a tool that will browse and display the message and correctly render the UTF-8 codes to character glyphs on your screen, otherwise they will appear as random multi-byte sequences that have their most significant bit set.
_________________
Glenn |
|
| Back to top |
|
|
| Mangesh1187 |
| Posted: Tue Feb 25, 2014 6:35 am Post subject: |
|
|
Centurion
Joined: 23 Mar 2013
Posts: 116
|
Hi All,
The Source application ABC sending Chinese messages encoded in UTF-8.
I tried to read the message from the queue, of target QM, Linux server where I can see '?' symbol instead of Chinese characters. And the PQR application (given below ) is getting encoding related exceptions.
The flow is like.
ABC Application (WAS)> QM1 > QM2 > QM3> MQCLNT > PQR Application (Web methods)
As shown above the message hopps between 3 QMs.
I have checked the CCSIDs for these QMs & MQ Client. Also the LANG properties for respective servers:
QM1: 1208 , QM2: 819 , QM3: 819 , MQCLIENT :819
en_us.utf8 , en_us, en_us, en_us.utf8
I have asked to set CCSID as 1208 in PQR application (Webmethods) also and encoding Value is set as 543. To be noted that the
ABC application the encoding value is 273.
But still the same issue.
My question is ,
1.The en_us LANG value causing the problem ?
2.If the answer of above question is yes, and I planed to change the value to en_us.utf8, will it impact the MQ services ? And i hope server need to be rebooted to get the effect for this env variable ?
3. If message is coming from ABC with CCSID 1208 but it is passing through QMs and MQ Client having CCSID value as 819. Is this affecting original encoded message with CCSID 1208 ? |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Feb 25, 2014 6:57 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Mangesh1187 wrote: |
| 1.The en_us LANG value causing the problem ? |
No.
| Mangesh1187 wrote: |
| 2.If the answer of above question is yes, and I planed to change the value to en_us.utf8, will it impact the MQ services ? And i hope server need to be rebooted to get the effect for this env variable ? |
Irrelevant
| Mangesh1187 wrote: |
| 3. If message is coming from ABC with CCSID 1208 but it is passing through QMs and MQ Client having CCSID value as 819. Is this affecting original encoded message with CCSID 1208 ? |
A MQClient doesn't have a CCSID value. If the queue manager to which the client is connected has a CCSID of 819 and the application doing a get of the message (PQR?) specifically requests that the message be converted into the local code page then the message will go from 1208 -> 819 in your example. If any of the channels between the queue managers have been specifically configured to convert the message into the local code page then likewise. The queue managers perform no conversion by default but only on request.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Mangesh1187 |
| Posted: Tue Feb 25, 2014 7:33 am Post subject: |
|
|
Centurion
Joined: 23 Mar 2013
Posts: 116
|
Hi Vitor,
In Source Application , ABC (WAS) not CCSID has mentioned. Its blank. However the QM CCSID is 1208. Hence the message will come with CCSID as 1208. This is my understanding. Please correct me if wrong.
We have a message trakcer application , form that I can see the ABCs message's CCSID is 1208 & Encoding is 273.
On Target Application, PQR , I have asked them to change the CCID to 1208(Although the QM3 has CCSID as 819) .
So that conversion will be like 1208 > 1208 . But still We got the same issue with the encoding related exception.
During analysis what I found is the encoding value in PQR was set as 543.
I suspect the encoding value of Source app ABC (273) & Target App PQR(543) causing a problem.
Also there are no channels between the queue managers have been specifically configured to convert the message into the local code page. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Feb 25, 2014 7:53 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Mangesh1187 wrote: |
| In Source Application , ABC (WAS) not CCSID has mentioned. Its blank. However the QM CCSID is 1208. Hence the message will come with CCSID as 1208. This is my understanding. Please correct me if wrong. |
Well.....
| Mangesh1187 wrote: |
| We have a message trakcer application , form that I can see the ABCs message's CCSID is 1208 & Encoding is 273. |
If you can see that the individual message's CCSID value in the MQMD is 1208 then you're correct are you not?
| Mangesh1187 wrote: |
| On Target Application, PQR , I have asked them to change the CCID to 1208(Although the QM3 has CCSID as 819) . |
Change it where in the application? How will it help? If they do a get with convert then the message will convert into the queue manager's CCSID (819).
| Mangesh1187 wrote: |
| So that conversion will be like 1208 > 1208 |
No it won't
| Mangesh1187 wrote: |
But still We got the same issue with the encoding related exception.
During analysis what I found is the encoding value in PQR was set as 543. |
Because (I suspect) QM3's encoding value is 543, which would match a CCSID of 819.
| Mangesh1187 wrote: |
| I suspect the encoding value of Source app ABC (273) & Target App PQR(543) causing a problem. |
I don't. Encoding controls how packed decimal and other binary fields are encoded, not how characters are converted.
Also there are no channels between the queue managers have been specifically configured to convert the message into the local code page.[/quote]
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Feb 25, 2014 8:03 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9399
Location: US: west coast, almost. Otherwise, enroute.
|
| Mangesh1187 wrote: |
We have a message trakcer application , form that I can see the ABCs message's CCSID is 1208 & Encoding is 273.
On Target Application, PQR , I have asked them to change the CCID to 1208(Although the QM3 has CCSID as 819) .
So that conversion will be like 1208 > 1208 . But still We got the same issue with the encoding related exception.
During analysis what I found is the encoding value in PQR was set as 543.
I suspect the encoding value of Source app ABC (273) & Target App PQR(543) causing a problem. |
I'm a bit confused. CCSID and encoding fields in the MQMD describe different things.
MQENC_NATIVE on UNIX systems = 273 X'00000111
Encoding usually means how the creating platform deals with numeric data in your application data payload. Is the creating platform UNIX? On Intel hardware? Are you looking at the MQENC_ field in the MQMD?
Where do you see the encoding related exception? Creating application or consuming app? Which platform? UNIX? Intel? Please post the exact error message here.
Is the junk data you see supposed to be numbers? Or character (non-numeric) data?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Mangesh1187 |
| Posted: Tue Feb 25, 2014 8:10 am Post subject: |
|
|
Centurion
Joined: 23 Mar 2013
Posts: 116
|
| Quote: |
Mangesh1187 wrote:
On Target Application, PQR , I have asked them to change the CCID to 1208(Although the QM3 has CCSID as 819) .
Change it where in the application? |
Yes. In the PQR (Webemthods) settings where the QM configuration along with channel and CCSID fields.
| Quote: |
How will it help? If they do a get with convert then the message will convert into the queue manager's CCSID (819).
|
But how to resolve this Dilemma then. Do I need to change the CCSID of QM3 or any other parameters are there to handle this situation?
Changing CCSID of the QM3 is not feasible option it seems.
Any thoughts. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Feb 25, 2014 10:10 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Mangesh1187 wrote: |
| But how to resolve this Dilemma then. |
The application needs to stop asking the qmgr to convert it when they read it off so the application gets the original 1208 not the converted 819.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Mangesh1187 |
| Posted: Tue Feb 25, 2014 10:42 am Post subject: |
|
|
Centurion
Joined: 23 Mar 2013
Posts: 116
|
| Quote: |
| The application needs to stop asking the qmgr to convert it when they read it off so the application gets the original 1208 not the converted 819. |
Does it means application needs to read the message without specifying any CCSID value ? Am I correct..... |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Feb 25, 2014 10:49 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9399
Location: US: west coast, almost. Otherwise, enroute.
|
Neither producing nor consuming app needs to override the ccsid if the qmgr or MQclient.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
