ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexClusteringConversion in a cluster

Post new topicReply to topic Goto page Previous  1, 2
Conversion in a cluster View previous topic :: View next topic
Author Message
gbaddeley
PostPosted: Wed Apr 06, 2011 2:43 pm Post subject: Re: Conversion in a cluster Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

rekarm01 wrote:
.. CCSID 500 and CCSID 819 use the same character set. There aren't any characters that will fail to translate from one to the other.


Comparing 819 ISO8859-1 Latin1 West Euro ( http://ascii-table.com/codepage.php?819 ) to 500 EBCDIC International ( http://www.tachyonsoft.com/cp00500.htm ) there appears to be a quite a few characters which don't translate from one code page to the other.
_________________
Glenn
Back to top
View user's profile Send private message
rekarm01
PostPosted: Wed Apr 06, 2011 10:40 pm Post subject: Re: Conversion in a cluster Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

rekarm01 wrote:
CCSID 500 and CCSID 819 use the same character set. There aren't any characters that will fail to translate from one to the other.

To be more accurate, that should have been "fail to convert", rather than "fail to translate".

gbaddeley wrote:
Comparing 819 ISO8859-1 Latin1 West Euro ( http://ascii-table.com/codepage.php?819 ) to 500 EBCDIC International ( http://www.tachyonsoft.com/cp00500.htm ) there appears to be a quite a few characters which don't translate from one code page to the other.

Which characters don't convert from one code page to the other?
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Thu Apr 07, 2011 4:22 pm Post subject: Re: Conversion in a cluster Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

rekarm01 wrote:
gbaddeley wrote:
Comparing 819 ISO8859-1 Latin1 West Euro ( http://ascii-table.com/codepage.php?819 ) to 500 EBCDIC International ( http://www.tachyonsoft.com/cp00500.htm ) there appears to be a quite a few characters which don't translate from one code page to the other.

Which characters don't convert from one code page to the other?


Apologies, they are both single byte Latin 1 code pages and have perfect 1-to-1 character mapping in both directions. In my comment, I think I was drawing on my past bad experiences with incompatibilities between the various EBCDIC code pages, and also garden variety ASCII.
_________________
Glenn
Back to top
View user's profile Send private message
lowellmather
PostPosted: Tue Jan 17, 2012 1:58 pm Post subject: CCSID 500 to CCSID 437 - translations failing? Reply with quote

Novice

Joined: 16 Jul 2009
Posts: 14

rekarm01 wrote:
CCSID 500 and CCSID 819 use the same character set. [...] How about converting to/from CCSID 437 instead? That's almost guaranteed to cause problems ...



This is EXACTLY my problem which is confined to trying to use global characters such as Á (an A with an accent mark over top of it). When we write normal English data the translation works fine (and has for many years this configuration has been in place (long before I got here)). We are testing globalization and have run into this glitch.

We have a z/OS remote queue going to a Windows local queue, "Conversion By Sender = Y" on the z/OS sending channel and the z/OS CCSID = 500. The Windows box has CCSID = 437. Both QMgrs are 7.0.

I had the app write to a local queue on z/OS and the special characters appear in the messages there as expected; however, when it's changed to write remote, the special characters reach the remote queue and they have not been translated correctly (I turned triggering off and viewed the message).

These special characters are available in CCSID 437 and CCSID 500 but for some reason the conversion/translation on the channel is not happening correctly.

I read the "Default Data Conversion" section of the "Data Conversion under MQ" white paper https://www-304.ibm.com/support/docview.wss?uid=swg27005729&aid=1) and looked into the ccsid.tbl on the Windows machine but I'm not sure if that's what I need to do. I believe we need to be able to support European languages (Spanish, French, German) but not sure which CCSID should be used on the Windows machine.

Any direction/advice on what to do next or where to further research?

Thank you...
Back to top
View user's profile Send private message
rekarm01
PostPosted: Fri Jan 20, 2012 2:06 am Post subject: Re: CCSID 500 to CCSID 437 - translations failing? Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

lowellmather wrote:
rekarm01 wrote:
CCSID 500 and CCSID 819 use the same character set. [...] How about converting to/from CCSID 437 instead? That's almost guaranteed to cause problems ...

CCSID 500 and CCSID 437 use different character sets. Many of the characters from CCSID 500 won't convert to CCSID 437 (and vice versa).

lowellmather wrote:
This is EXACTLY my problem which is confined to trying to use global characters such as Á (an A with an accent mark over top of it).
...
These special characters are available in CCSID 437 and CCSID 500

These characters are not available in CCSID 437:

Code:
¤ ¦ ¨ © ­ ® ¯ ³ ´ µ ¸ ¹ ¾ À Á Â Ã È Ê Ë Ì Í Î Ï Ð Ò Ó Ô Õ × Ø Ù Ú Û Ý Þ ã ð õ ø ý þ

So they won't convert as expected.

lowellmather wrote:
I believe we need to be able to support European languages (Spanish, French, German) but not sure which CCSID should be used on the Windows machine.

Any direction/advice on what to do next or where to further research?

This thread describes two other CCSIDs which are more compatible with CCSID 500.
Back to top
View user's profile Send private message
lowellmather
PostPosted: Fri Jan 20, 2012 7:43 am Post subject: Re: Conversion in a cluster Reply with quote

Novice

Joined: 16 Jul 2009
Posts: 14

gbaddeley wrote:
rekarm01 wrote:
.. CCSID 500 and CCSID 819 use the same character set. There aren't any characters that will fail to translate from one to the other.


Comparing 819 ISO8859-1 Latin1 West Euro ( http://ascii-table.com/codepage.php?819 ) to 500 EBCDIC International ( http://www.tachyonsoft.com/cp00500.htm ) there appears to be a quite a few characters which don't translate from one code page to the other.


Thank you very much for taking the time to respond! I'm not sure if 819 is one of the codepages to which you were referring, but since I posted last, we did try 819 and it appears to work when the msgs are on the queue but when they are written out to a .dat file, they are not showing up the same (might be an issue with the codepage Notepad is using).

One question though - our application can run on z/OS, Windows and UNIX. When we run on z/OS we route the msgs from an MVS QMgr to a Windows QMgr, as I've described before.

However, when the app runs under Windows, we write directly to a different QMgr (on the same box as the QMgr connected to MVS) which is also using CCSID 437. When we enter the same special characters in the app under Windows and they are written to the Wiindows QMgr, the characters appear correctly (where no conversion taking place). The hex value of the character in question in CCSID 437 is "CB".

My question, out of curiosity, is: if the character exists in 437 as "CB" and in 500 as a hex "73" why would it not convert? Maybe the characters appear the same visually but are not the same programmatically. Just seems strange that that specific character wouldn't convert.

Thanks again for your assistance! You guys are awesome!
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Fri Jan 20, 2012 1:21 pm Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

If your message format is not MQC.MQFMT_STRING there is no conversion!
In order for the conversion to present the least number of problems you may want to have the corresponding code page installed at OS level (yes even on windows).

However I would suggest you do the conversion in the get call and not at channel level.

Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
rekarm01
PostPosted: Sun Jan 22, 2012 9:52 pm Post subject: Re: CCSID 500 to CCSID 437 - translations failing? Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

lowellmather wrote:
we did try 819 and it appears to work when the msgs are on the queue but when they are written out to a .dat file, they are not showing up the same (might be an issue with the codepage Notepad is using).

Tried 819 where? As the Windows QMgr CCSID (which describes the MQMD)? Or as the MQ message header CCSID (which describes the message data)?

lowellmather wrote:
if the character exists in 437 as "CB" and in 500 as a hex "73" why would it not convert?

But the character doesn't exist in CCSID 437; that's why it won't convert. These are two different characters:

Code:
ccsid 500: X'73' = 'Ë' (<U+00CB>, "LATIN CAPITAL LETTER E WITH DIAERESIS")
ccsid 437: X'CB' = '╦' (<U+2566>, "BOX DRAWINGS DOUBLE DOWN AND HORIZONTAL")

Either the source application used a different CCSID to write the message data, or some converting application had to replace unsupported characters along the way. Examine both the message header (CCSID, Format, etc.) and the hex bytes for the data in question, both before and after conversion, to narrow down where the problem occurs. Confirm that the hex bytes map to the expected characters, using the appropriate Windows-supported or IBM code pages.
Back to top
View user's profile Send private message
lowellmather
PostPosted: Mon Jan 23, 2012 7:43 am Post subject: Reply with quote

Novice

Joined: 16 Jul 2009
Posts: 14

As you can tell, I'm a novice with MQ so I appreciate your patience! Hopefully I won't confuse the issue more.

Quote:
"Tried 819 where? As the Windows QMgr CCSID (which describes the MQMD)? Or as the MQ message header CCSID (which describes the message data)? "


We set 819 as the Windows QMgr CCSID. I did not know you could set the MQ message header CCSID (or is that the CCSID that is set on the sending QMgr, i.e, 500)?

Let me restate the values I'm seeing when I browse the queues:

The value on MVS CCSID 500 = hex 73

For the data displayed on the Windows QMgrs, I'm going to include all the values from the start of the displacement where the special characters are used:

The value from MVS converted on channel to CCSID 819 on Win QMgr on the queue =
20 00 CB 4E 41 4E 44 43

The data values when written in Windows to Windows QMgr CCSID 437 on the queue =
20 00 4E CB 41 4E 44 43

These are the hex values I see when I look at the message on the queue.
I tried to attached screenprints but couldn't figure out how to get them in here.

We have a utility that reads the events from the queue and writes them to a .dat file (previously mentioned). It is supposed to use the CCSID assigned to the QMgr so it should use 819 for the Qmgr with the "former" MVS data and 437 with the Windows-only data.

The .dat output from the Windows-only Qmgr (CCSID 437) appears correctly, however, for the MVS to Windows QMgr (CCSID 819) data, despite the value now displaying on the queue correctly as "Ë", the value in the .dat file is "à".

I will research those links you sent.

Thank you for your help!
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Jan 23, 2012 9:07 am Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Note that 819 and 437 have exactly the same values with a transposition:
CB 4E
4E CB

probably due to difference in endianness?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
rekarm01
PostPosted: Thu Jan 26, 2012 3:40 am Post subject: Re: CCSID 500 to CCSID 437 - translations failing? Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

lowellmather wrote:
We set 819 as the Windows QMgr CCSID. I did not know you could set the MQ message header CCSID (or is that the CCSID that is set on the sending QMgr, i.e, 500)?

The QMgr CCSID is an attribute of the queue manager itself. Each message on a queue has its own MQMD, possibly other MQ message headers, and a byte sequence that makes up the message data. The source application provides all of this when it calls MQPUT. The fields in the MQ message headers (Format, CCSID, etc.) is the only information that's relevant for interpreting the message data. The QMgr CCSID is not relevant for this purpose.

lowellmather wrote:
Let me restate the values I'm seeing when I browse the queues:

The value on MVS CCSID 500 = hex 73

Any message browsing tool should also be able to parse and display at least the MQMD header contents as well. Without that information, it's difficult to offer any help.

lowellmather wrote:
For the data displayed on the Windows QMgrs, I'm going to include all the values from the start of the displacement where the special characters are used:

Do that for the MVS QMgr too. How does the one byte above correlate to the eight bytes below?

lowellmather wrote:
The value from MVS converted on channel to CCSID 819 on Win QMgr on the queue =
20 00 CB 4E 41 4E 44 43

The data values when written in Windows to Windows QMgr CCSID 437 on the queue =
20 00 4E CB 41 4E 44 43

The null byte (00) and the swapped bytes suggest there's non-character data in the message as well. What is the 'Format'?

lowellmather wrote:
We have a utility that reads the events from the queue and writes them to a .dat file (previously mentioned).

Without a more detailed description of how the utility gets the message, whether it converts the message, or how else it may transform the message data after that, (including any relevant code), it's difficult to offer any help with that either.

lowellmather wrote:
The .dat output from the Windows-only Qmgr (CCSID 437) appears correctly, however ...

What utility is displaying the .dat output? It's possible that the .dat output is ok, but the utility displaying the output makes it appear incorrectly.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Goto page Previous  1, 2 Page 2 of 2

MQSeries.net Forum IndexClusteringConversion in a cluster
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.