ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » MQ and CCSID channel exit

Post new topic  Reply to topic Goto page 1, 2  Next
 MQ and CCSID channel exit « View previous topic :: View next topic » 
Author Message
zrux
PostPosted: Tue Apr 30, 2019 4:21 am    Post subject: MQ and CCSID channel exit Reply with quote

Apprentice

Joined: 21 May 2006
Posts: 37
Location: UK

Hello all

Getting the following errors for CCISD

------
We cannot change the CCSID on either side at a QM level
that is too painful. Is there a sample channel exit program which can do this conversion please.



AMQ6047: Conversion not supported.
EXPLANATION:
WebSphere MQ is unable to convert string data tagged in CCSID 819 to data in
CCSID 1381.
ACTION:
Check the WebSphere MQ Application Programming Reference Appendix and the
appropriate National Language Support publications to see if the CCSIDs are
supported by your system.
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Apr 30, 2019 4:59 am    Post subject: Re: MQ and CCSID channel exit Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

zrux wrote:
Is there a sample channel exit program which can do this conversion please.


The most likely reason for this error is that the code pages the queue manager needs to do the conversion are not available in the OS. If this is the case, any channel exit would likely struggle for the same reason.

To perform the conversion, the channel exit would need to be provided with the necessary code pages and I put it to you that, in this situation, it would be easier and safer to load the code pages into the OS and let the queue manager do it.

Moving into the realm of personal opinion, I wouldn't do conversion in a channel exit. That's a lot of computation to put into an exit and it's going to adversely affect throughput. I'm also suspicious of exits that transform payload because you're getting dangerously close to business logic in a system component.

Full disclosure - I'm suspicious of exits. They're the number one preferred way of crashing MQ as one tiny bug will bring the system processes down. Exits should be carefully written and tested to destruction.

My view can be best summed up as:

Quote:
If the answer is an exit, you're asking the wrong question


I don't think I'd want an exit or MQ doing the conversion from 819 (Latin-1) to 1381 (Simplified Chinese). Sounds like a job for an application to me.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
zrux
PostPosted: Tue Apr 30, 2019 7:33 am    Post subject: Reply with quote

Apprentice

Joined: 21 May 2006
Posts: 37
Location: UK

How do I load the code pages into the OS and let the queue manager do it..we are running a RHEL on our side.

Would this affect any other channels from different systems to stop working?
Back to top
View user's profile Send private message
tczielke
PostPosted: Tue Apr 30, 2019 8:58 am    Post subject: Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

I am thinking you are getting the conversion error because 819 (a Western European CCSID) and 1381 (a Simplified Chinese CCSID) do not support conversion between the two CCSIDs.

For example, how would you data convert "fox" to Simplified Chinese if there is no "f", "o", or "x" in the 1381 character set (which I am assuming there is not)?

I would think you want your source data to be in a Unicode CCSID (e.g. 1208 or UTF-8 ) in order to convert it to 1381. Or is it possible for the application to just work with the data in CCSID 1208?
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Apr 30, 2019 9:19 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

zrux wrote:
How do I load the code pages into the OS and let the queue manager do it..we are running a RHEL on our side.


You ask whoever administers your server do to it; requires root access and a certain level of knowledge.

zrux wrote:
Would this affect any other channels from different systems to stop working?


No, because conversion is at a message level. This channel exit you proposed would need to check each message and that's another good reason not to do this.

I also echo comments of my worthy associate, which I touched on above when I said this conversion was a job for an application. CCSID 1381 may be "simplified" Chinese but (as my associate points out) I would be interested to discover there's a character for character equivalence with the Latin alphabet.

This of course would also explain the lack of conversion tables in the OS - there is no conversion between these sets and you need a launguage aware application.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
zrux
PostPosted: Tue Apr 30, 2019 9:24 am    Post subject: Reply with quote

Apprentice

Joined: 21 May 2006
Posts: 37
Location: UK

at the moment the error is seen just when the channel is trying to start, without any data flowing through the channel

I found this link
https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_8.0.0/com.ibm.mq.adm.doc/q021240_.htm

which talks about changing the file ccsid.tbl which exists on /var/mqm/conv/table folder.

What I am not sure is if we are to change the above file will any of the existing connections get affected, as we got 100s of connections from different source and parts of the world.
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Apr 30, 2019 9:25 am    Post subject: Re: MQ and CCSID channel exit Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

zrux wrote:
We cannot change the CCSID on either side at a QM level
that is too painful.


It's probably worth underlining that the CCSID of the QM is the default; the putting application can use any CCSID it likes on any given message. This is why I keep saying an application needs to do this conversion; sending end or receiving end, take your choice.

HOWEVER

It's imperative that the CCSID of the message matches the payload. The application can change the CCSID of the message from 819 to 1381 and this will make your conversion error go away. It will also make the payload unreadable to the receiving application.

The CCSID in the MQMD must match the one used to encode the payload. Just changing the number in the MQMD will not work.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Apr 30, 2019 9:33 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

zrux wrote:
I found this link
https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_8.0.0/com.ibm.mq.adm.doc/q021240_.htm


From this very link:
Quote:
The queue manager cannot automatically convert messages in built-in formats if their CCSIDs represent different national-language groups


I would be interested if you could post a link demonstrating that Latin (819) and Chinese (1381) are considered the same national language group.

zrux wrote:
which talks about changing the file ccsid.tbl which exists on /var/mqm/conv/table folder.


It does. If you have something more in mind than just adding the needed numbers.

zrux wrote:
What I am not sure is if we are to change the above file will any of the existing connections get affected, as we got 100s of connections from different source and parts of the world.


Well that file is used at a queue manager level so if you screw it up (like putting in an invalid combination) the results are not going to be good.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Apr 30, 2019 9:46 am    Post subject: Re: MQ and CCSID channel exit Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

zrux wrote:
We cannot change the CCSID on either side at a QM level
that is too painful.


Just to wrap up, I agree with this completely. The vast number of applications using a queue manager don't specify an explicit CCSID so changing this is a short road to misery & suffering on an epic, epic scale.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
rekarm01
PostPosted: Tue Apr 30, 2019 1:18 pm    Post subject: Re: MQ and CCSID channel exit Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

tczielke wrote:
For example, how would you data convert "fox" to Simplified Chinese if there is no "f", "o", or "x" in the 1381 character set (which I am assuming there is not)?

ccsid=1381 is a mixed single-/double-byte charset, containing a single-byte code page (cpgid=1115), and a double-byte code page (cpgid=1380). The single-byte code page includes the US ASCII character set.

zrux wrote:
... which talks about changing the file ccsid.tbl which exists on /var/mqm/conv/table folder.

This is generally more useful for default data conversions, such as for converting MQ headers between queue managers, where MQ does not support direct conversion.

MQ does not support conversion directly from ccsid=819 to ccsid=1381, but it does support conversion to/from Unicode, (for example: 819 -> 1208 -> 1381). If the source application could send the message out as UTF-8 (ccsid=1208), then the target application could use MQ to convert it. If the message content is restricted to plain ASCII, then the conversion is trivial.
Back to top
View user's profile Send private message
tczielke
PostPosted: Tue Apr 30, 2019 1:37 pm    Post subject: Re: MQ and CCSID channel exit Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

rekarm01 wrote:
tczielke wrote:
For example, how would you data convert "fox" to Simplified Chinese if there is no "f", "o", or "x" in the 1381 character set (which I am assuming there is not)?

ccsid=1381 is a mixed single-/double-byte charset, containing a single-byte code page (cpgid=1115), and a double-byte code page (cpgid=1380). The single-byte code page includes the US ASCII character set.


Indeed! I should have just tested it first:

Code:

> ./mqcpcnvt fox 1208 1381 TCZ.TEST1
mqcpcnvt start
The input data was detected as char string: fox
target queue is TCZ.TEST1
The following bytes were PUT with CCSID 1208
<666F78>
The following bytes were returned with converted GET with CCSID 1381:
<666F78>
This is the returned byte string printed as chars (a non isprint char will be shown as '.').
<fox>
mqcpcnvt end


I did try to find the CDRA to look it up, but I have found that a difficult task recently with Google, at least with the searches I have been doing.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
rekarm01
PostPosted: Tue Apr 30, 2019 2:02 pm    Post subject: Re: MQ and CCSID channel exit Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

tczielke wrote:
I did try to find the CDRA to look it up, but I have found that a difficult task recently with Google, at least with the searches I have been doing.

IBM "sunsetted" their Globalization website, where the CDRA reference used to be.

Quote:
For globalization or terminology information related to IBM products please refer to the product documentation found in the IBM Knowledge Center.

No comment.
Back to top
View user's profile Send private message
tczielke
PostPosted: Tue Apr 30, 2019 5:30 pm    Post subject: Re: MQ and CCSID channel exit Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

rekarm01 wrote:
tczielke wrote:
I did try to find the CDRA to look it up, but I have found that a difficult task recently with Google, at least with the searches I have been doing.

IBM "sunsetted" their Globalization website, where the CDRA reference used to be.

Quote:
For globalization or terminology information related to IBM products please refer to the product documentation found in the IBM Knowledge Center.

No comment.


If the CDRA is truly gone, technicians and insomniacs will never be the same.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
exerk
PostPosted: Wed May 01, 2019 4:36 am    Post subject: Re: MQ and CCSID channel exit Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

tczielke wrote:
Code:
> ./mqcpcnvt fox 1208 1381 TCZ.TEST1

Is that a home-grown thing or a supplied sample? Because if the latter I can't find it anywhere!
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
tczielke
PostPosted: Wed May 01, 2019 7:21 am    Post subject: Re: MQ and CCSID channel exit Reply with quote

Guardian

Joined: 08 Jul 2010
Posts: 939
Location: Illinois, USA

exerk wrote:
tczielke wrote:
Code:
> ./mqcpcnvt fox 1208 1381 TCZ.TEST1

Is that a home-grown thing or a supplied sample? Because if the latter I can't find it anywhere!


You can find the C source for it here -> https://www.capitalware.com/mq_code_c.html

That is the first hit I see for it with Google, too. You do have to C compile it, to get an executable. I just follow the build C doc in the MQ manual to do that.

There is also some doc for mqcpcnvt in some of my MQTC sessions. Basically, it is a tool to help you see how MQ will convert bytes between code pages.
_________________
Working with MQ since 2010.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » General IBM MQ Support » MQ and CCSID channel exit
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.