| Author |
Message
|
| hellobond070 |
 Posted: Wed Dec 09, 2009 11:00 am Post subject: Problem with Conversion from EBCDIC to ASCII |
 |
|
Centurion
Joined: 18 Nov 2009
Posts: 118
|
I have a message coming from a mainframe system and is attached to FileOutput node on Windows broker.
So the message is put from mainframe to Windows machine.
Incoming message deatils in MQMD :
ccsid : 437
Encoding : 546
The characters are getting corrupted when written as a file on windows platform.
Kindly advise. |
|
| Back to top |
|
 |
| rekarm01 |
| Posted: Wed Dec 09, 2009 11:37 am Post subject: Re: Problem with Conversion from EBCDIC to ASCII |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
Either the ccsid does not match the message data, or something converted the message to a ccsid that can not represent the message data.
For a more specific answer, provide more details.
| hellobond070 wrote: |
Incoming message details in MQMD :
ccsid : 437
Encoding : 546 |
437 is not EBCDIC.
Last edited by rekarm01 on Wed Dec 09, 2009 12:48 pm; edited 1 time in total |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Dec 09, 2009 12:45 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Look at the message before it is processed using a tool like MO71, browse it with and without conversion, check the actual data characters (in hex) compared to the MQMD.ccsid codepage. Does it have MQMD.format = MQSTR?
What is being written on Windows - is it EBCDIC data (not in fact corrupt)? Could be it was not converted at all.
Try setting convert on the MQinput node properties. |
|
| Back to top |
|
|
| hellobond070 |
| Posted: Thu Dec 10, 2009 2:03 am Post subject: |
|
|
Centurion
Joined: 18 Nov 2009
Posts: 118
|
Hi zpat and rekarm01,
The incoming data is already in the broker and hence 437 I guess.
But this is what I see in the visual debug mode of the broker 6.1.
Regarding the conversion of the corrupt characters. Basically through fileoutput node the data gets written in a file on windows platform.
When I open it using ultraedit, the characters are corrupted. But when I convert it using EBCDIC to ASCII feature present in ultraedit, the characters come back to life correctly.
Please help. Also I need to know whether applying ccsid whether the below esql is correct :
Set OutputRoot = CAST(InputRoot as CHAR CCSID XXX) where XXX can be any suitable ccsid.
Please adivse. |
|
| Back to top |
|
|
| hellobond070 |
| Posted: Thu Dec 10, 2009 2:36 am Post subject: |
|
|
Centurion
Joined: 18 Nov 2009
Posts: 118
|
Hi zpat and rekarm01,
The incoming data is already in the broker and hence 437 I guess.
But this is what I see in the visual debug mode of the broker 6.1.
Regarding the conversion of the corrupt characters. Basically through fileoutput node the data gets written in a file on windows platform.
When I open it using ultraedit, the characters are corrupted. But when I convert it using EBCDIC to ASCII feature present in ultraedit, the characters come back to life correctly.
Please help. Also I need to know whether applying ccsid whether the below esql is correct :
Set OutputRoot = CAST(InputRoot as CHAR CCSID XXX) where XXX can be any suitable ccsid.
Please adivse. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Dec 10, 2009 2:37 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| hellobond070 wrote: |
When I open it using ultraedit, the characters are corrupted. But when I convert it using EBCDIC to ASCII feature present in ultraedit, the characters come back to life correctly.
|
That is your problem. The data is there but you are reading it using the wrong transation codes. What you are seeing in UltraEdit is correct. When you convert from ASCII to EBCIDIC all you are doing is telling UltraEdit to use a different set of translation codes
Use something like RFHUTIL from the ih03 support pack. Read the message from the flow input queue (After stopping the flow). Look at the MQMD CCSID.
Also take a look at the parameters on the WMQ Channel that receives the data from the CICS system.
Finally look at the MQInput node parameters. Look for things relating to Convert.
Ideally, you don't want WMQ or Broker to conver the message into the local CCSID. Once you have read it into the FLOW the data is in its original format and can be parsed by the flow.
Post your findings here if it is still unclear
It could be that the original message was actually sent with the wrong CCSID. There have been recent posts here with exactly that problem. In that case, you have to go back to the MF developers and kindly point out the error of their ways.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Dec 10, 2009 3:48 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
For a message to be convertable it must:
1. Have MQMD.Format = MQSTR
2. Have MQMD.CCSID = the code page of the message data eg 500 for EBCDIC
3. Have Message data of this code page (eg EBCDIC)
Examine and convert the message with browsing tools like RFHUTILC or MO71 first, only when it works as expected try the broker.
If you are processing the message as a BLOB it won't be converted by the broker, you can either set convert on the MQINPUT node properties or using CAST in ESQL. |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Dec 10, 2009 4:56 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
Also I need to know whether applying ccsid whether the below esql is correct :
Set OutputRoot = CAST(InputRoot as CHAR CCSID XXX) where XXX can be any suitable ccsid. |
No, it's not correct. InputRoot is not a BLOB ( in fact, it is a Name element, and does not have a value at all ). If you want to get the input message as a BLOB, you should either set the input node domain to 'BLOB' or use ASBITSTREAM.
But don't try to adjust the CCSID this way - read the other replies on this thread. |
|
| Back to top |
|
|
| steve_metal |
| Posted: Thu Jul 28, 2011 1:00 am Post subject: |
|
|
Novice
Joined: 22 Sep 2009
Posts: 17
|
Sorry to re-open the forum again .
Facing a similar problem .
Recieving a message from Mainframe . The messaged saved off the queue has CCSID 285 (same message is picked by the flow). In RFHUTIL if the char format is chosen as EBCIDIC it has proper values , in ASCII mode the values are junk representations.
Transforming the message in ESQL to an XML (having few mappings) . No bitstream or data conversion anywhere . When writing the data out in the file output node , data getting written as EBCDIC representation . (If the file is opened and viewed as ASCII it looks fine ) .
Is there something that I am missing to do .
Thanks and Regards
Steve |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Jul 28, 2011 1:46 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| steve_metal wrote: |
Sorry to re-open the forum again .
Facing a similar problem .
Recieving a message from Mainframe . The messaged saved off the queue has CCSID 285 (same message is picked by the flow). In RFHUTIL if the char format is chosen as EBCIDIC it has proper values , in ASCII mode the values are junk representations.
Transforming the message in ESQL to an XML (having few mappings) . No bitstream or data conversion anywhere . When writing the data out in the file output node , data getting written as EBCDIC representation . (If the file is opened and viewed as ASCII it looks fine ) .
Is there something that I am missing to do .
Thanks and Regards
Steve |
Could it just be possible that you are writing the message as CCSID=285?
Can you change the OutputRoot.Properties.CodedCharsetId to something that is not EBCIDIC such as 1208?
There is an implicit CCSID conversion when broker reads the message. As has been said many time (mostly by Kimbert) broker uses unicode internally thus there has been some transformation on the way in. The next transformation is on the way out. You need to tell broker how you want the data presented.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Jul 28, 2011 2:33 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Is the MQMD.Format = MQSTR? |
|
| Back to top |
|
|
| steve_metal |
| Posted: Thu Jul 28, 2011 2:50 am Post subject: |
|
|
Novice
Joined: 22 Sep 2009
Posts: 17
|
Thanks smdavies ... .
Worked pretty .. Just set the properties.ccsid to 1208 and the data is written out fine.
The CCSID in the properties was getting set to '37' out of nowhere , not sure where it comes from , once that is reset to 1208 to represent a UTF-8 XML file ,It displays it properly . |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sun Jul 31, 2011 1:34 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| steve_metal wrote: |
| The CCSID in the properties was getting set to '37' out of nowhere, not sure where it comes from |
Perhaps it comes from the MQInput node 'Convert' check box. If WMB converts already, it's usually not necessary for WMQ to convert too. |
|
| Back to top |
|
|
| zpat |
| Posted: Sun Jul 31, 2011 1:50 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| WMQ will get the first go at conversion. |
|
| Back to top |
|
|
| steve_metal |
| Posted: Mon Aug 01, 2011 10:29 pm Post subject: |
|
|
Novice
Joined: 22 Sep 2009
Posts: 17
|
| My MQ input node does not have a convert on . And MQ should not convert a message unless I tell it to right ? |
|
| Back to top |
|
|
|
|
