| Author |
Message
|
| starki78 |
 Posted: Mon May 23, 2011 12:24 am Post subject: How to convert 1141 EBEDIC to ISO8859-15 (923) |
 |
|
Acolyte
Joined: 24 Sep 2007
Posts: 53
|
Hello
I've learnt that when converting between ASCII and EBEDIC
the right thing to do is to look at the convesation table
shipped with the MQ-Series installation.
Further MQC.MQGMO_CONVERT needs to be used as a Get Option
Now a message is going to be sent from Main-Frame (1141) to Non-Main-Frame. However we see that the German "Umlaute" are garbelled.
However the conversation table shows no direct way to do this conversation.
The logical path would be:
04750111 From 1141 to 273. SBCS using enforced subset matching.
('EBCDIC Austria, Germany, with euro (like 273)' to 'EBCDIC German'),
01110333 From 273 to 819. SBCS using CDRA roundtrip algorithm 2.
('EBCDIC German' to 'ISO8859-1 Western European'),
0333039B From 819 to 923. SBCS using enforced subset matching.
('ISO8859-1 Western European' to 'ISO8859-15 Western Europe with euro (Latin 9)'),
Does this mean that the convesation would need 3 steps and the message needs to be converted 3 times?
Is there any workaround?
Thanks for clarification
Christian |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Mon May 23, 2011 2:26 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| I might take the opportunity to convert the sending app to unicode instead. |
|
| Back to top |
|
|
| starki78 |
| Posted: Mon May 23, 2011 5:36 am Post subject: |
|
|
Acolyte
Joined: 24 Sep 2007
Posts: 53
|
thanks for the response.
Are you saying that you recommend to convert first
to 1208 (UTF) and then it is possible to convert to the desired CCSID?
If this is the case is there any chance to convert to 1208 on the Client-Side
and then make a second conversion on the client side to 923?
thanks! |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon May 23, 2011 6:11 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
No, I'm suggesting changing the application that creates the data in the first place to stop creating it in 1141 EBEDIC and instead create it in UTF-8 or UTF-16.
Even COBOL applications these days should be able to do this without too much trouble, presuming you're at reasonably recent levels of zOS or VSE or whatever. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Mon May 23, 2011 3:33 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
Is the msg being put with the default qmgr ccsid (1141 EBCDIC-German-EuroSign) ? This code page might differ from the one the application program is using internally, eg. 273 EBCDIC-German, 500 EBCDIC-International.
Exactly which vowel character is being mistranslated? Upper or lower case?
_________________
Glenn |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Mon May 23, 2011 5:13 pm Post subject: Re: How to convert 1141 EBCDIC to ISO8859-15 (923) |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
This probably belongs under General WebSphere MQ Support.
| starki78 wrote: |
| I've learned that when converting between ASCII and EBCDIC, the right thing to do is to look at the convesation table shipped with the MQ-Series installation. |
Which conversion (?) table would that be? According to the documentation, WMQ supports converting ccsid 1141 directly to either ccsids 273, 819, or 923.
But the different ccsids use different character sets, so they're not entirely compatible; it's simply not possible to convert some of the characters from ccsid 1141 to some of the other ccsids, without some degree of data loss.
For example, ccsids 273 and 819 don't have a EURO SIGN ('€', U+20AC), and ccsid 923 doesn't have a DIAERESIS ('¨', U+00A8), CEDILLA ('¸', U+00B8), or some other characters.
To avoid data loss, only use the subset of characters common to all the ccsids of interest.
| starki78 wrote: |
| Now a message is going to be sent from Main-Frame (1141) to Non-Main-Frame. However we see that the German "Umlaute" are garbled. |
ccsid 923 doesn't have an umlaut character ("DIAERESIS"). So, either don't use the umlaut character, or don't use ccsid 923.
IBM can use some form of character substitution when handling character set mismatches, so as to minimize data loss:- round-trip integrity: substitute mismatched characters with possibly unrelated characters, such that converting source->target->source restores the original data, without loss
- character replacement: substitute mismatched characters with similar characters; the original characters are lost
- enforced subset matching: substitute mismatched characters with a special substitution character, indicating that a substition occurred; the original characters are lost
The specific technique depends on the specific combination of source and target ccsids.
This can produce unexpected results for multiple conversions: given three ccsids, A, B, and C, even if converting A->B is lossless, converting A->B->C might not produce the same results as converting A->C.
| mqjeff wrote: |
| I might take the opportunity to convert the sending app to unicode instead. |
... and perhaps the receiving app, too.
[Edit: misspelled "DIAERESIS"]
Last edited by rekarm01 on Tue May 24, 2011 1:30 pm; edited 1 time in total |
|
| Back to top |
|
|
| skoobee |
| Posted: Tue May 24, 2011 1:14 am Post subject: |
|
|
Acolyte
Joined: 26 Nov 2010
Posts: 52
|
[quote]ccsid 923 doesn't have an umlaut character ("DIARESIS"). So, either don't use the umlaut character, or don't use ccsid 923[/quote]
I think the OP is referring to the set of chars with umlauts - Umlaute is a noun (capitalised), in the plural. These chars do exist in 923 as follows:
ä 132
ö 148
ü 129
Ä 142
Ö 153
Ü 154
ß 225
Also, it is not reasonable to tell somebody sending msgs in German not to use chars with umlauts! |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue May 24, 2011 2:10 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| skoobee wrote: |
| Also, it is not reasonable to tell somebody sending msgs in German not to use chars with umlauts! |
That's why I suggested unicode. And whilst it would be nice to switch both the sending and receiving apps, it's not strictly necessary. And the conversion FROM UTF-8 or UTF-16 to ISO8859-15 (923) should be straightforward and occur in one shot. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Tue May 24, 2011 8:05 pm Post subject: Re: How to convert 1141 EBCDIC to ISO8859-15 (923) |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| skoobee wrote: |
These chars do exist in 923 as follows:
ä 132
ö 148
ü 129
Ä 142
Ö 153
Ü 154
ß 225 |
Every one of those numbers is wrong.
And ß doesn't have an umlaut.
| skoobee wrote: |
| Also, it is not reasonable to tell somebody sending msgs in German not to use chars with umlauts! |
Who suggested that? 
ccsid 923 contains a dozen 'vowel+umlaut' characters, that both Germans and non-Germans alike are free to use, as they see fit. If one of these characters is causing conversion issues, then the problem description needs more work.
But, unlike any of the other ccsids mentioned in this thread, ccsid 923 does not contain the unadorned umlaut character (U+00A8). So, unlike any of the dozen other 'vowel+umlaut' characters, this character won't convert to ccsid 923.
It seemed more likely that that was the problem.
| mqjeff wrote: |
| And whilst it would be nice to switch both the sending and receiving apps, it's not strictly necessary. And the conversion FROM UTF-8 or UTF-16 to ISO8859-15 (923) should be straightforward and occur in one shot. |
... unless the Unicode message contains one or more characters that ISO8859-15 doesn't. |
|
| Back to top |
|
|
| skoobee |
| Posted: Tue May 24, 2011 10:39 pm Post subject: |
|
|
Acolyte
Joined: 26 Nov 2010
Posts: 52
|
[quote]Every one of those numbers is wrong[/quote]
Not a very helpful comment, even if it is right (as it is). The values above are from 437, not 923.
The real values are
ä xE4
ö xF6
ü xFC
Ä xB4
Ö xD6
Ü xDC
ß xDF
Also, it is indeed true, and rather obvious, that the sz-ligature does not have an umlaut, but it is included in this list of non-English German chars for completeness. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Wed May 25, 2011 12:40 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
|
| Back to top |
|
|
|
|
