| Author |
Message
|
| sumithar |
 Posted: Fri Sep 21, 2012 10:08 am Post subject: GET converts EBCDIC to ASCII? |
 |
|
Apprentice
Joined: 13 Jun 2008
Posts: 47
|
We have a process whereby messages are written to a queue on the IBM Mainframe by CICS transactions as well as transactions running on a UNISYS mainframe.
I can browse the messages fine in MQ Explorer on my PC as well as view them using the File Manager product on zOS and in the latter case if I turn on HEX I can see they are all EBCDIC characters.
I have a batch program on IBM that GETs the messages from this queue and writes them out to a file. When I look at the file what I'm finding is that the messages that the program GETs from the queue that were PUT there by Unisys are written out into the file as ASCII.
But the messages that were PUT by CICS are written out as EBCDIC as they should be.
Is there some option in the PUT that the UNISYS program is using that could potentially cause this to happen? I know they are using format MQSTR. When I look at the messages in MQ explorer, the CCSID is 819 for the Unisys messages and 500 for the CICS messages. I know the former means ASCII and the latter EBCDIC. Does this matter?
There is another column called Encoding which has values 273 and 785 respectively.
FWIW this program is in production and working fine with both CICS and Unisys written messages and all the aforesaid information about CCSID and encoding are the same there as well.
Any thoughts?
Thanks! |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Fri Sep 21, 2012 10:17 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Does the app that MQGETs the messages specify MQGMO_CONVERT option? GMO_CONVERT will cause the message to be converted from ASCII to EBCDIC.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| sumithar |
| Posted: Fri Sep 21, 2012 10:26 am Post subject: |
|
|
Apprentice
Joined: 13 Jun 2008
Posts: 47
|
| bruce2359 wrote: |
| Does the app that MQGETs the messages specify MQGMO_CONVERT option? GMO_CONVERT will cause the message to be converted from ASCII to EBCDIC. |
Yes, sir. The options before the GET are as follows
COMPUTE MQGMO-OPTIONS = MQGMO-NONE +
MQGMO-NO-WAIT +
MQGMO-SYNCPOINT +
MQGMO-CONVERT +
MQGMO-FAIL-IF-QUIESCING
Thanks |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Sep 21, 2012 12:42 pm Post subject: Re: GET converts EBCDIC to ASCII? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sumithar wrote: |
| I know they are using format MQSTR. |
Do you know this because they've told you, or because you can see it in the MQMD of the message?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sumithar |
| Posted: Fri Sep 21, 2012 1:04 pm Post subject: Re: GET converts EBCDIC to ASCII? |
|
|
Apprentice
Joined: 13 Jun 2008
Posts: 47
|
| Vitor wrote: |
| sumithar wrote: |
| I know they are using format MQSTR. |
Do you know this because they've told you, or because you can see it in the MQMD of the message? |
No, not because they said so. I am seeing it in MQ Explorer. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Fri Sep 21, 2012 1:27 pm Post subject: Re: GET converts EBCDIC to ASCII? |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| sumithar wrote: |
The options before the GET are as follows:
| Code: |
COMPUTE MQGMO-OPTIONS = MQGMO-NONE +
MQGMO-NO-WAIT +
MQGMO-SYNCPOINT +
MQGMO-CONVERT +
MQGMO-FAIL-IF-QUIESCING |
|
The MQGMO-NONE is harmless, but unnecessary.
Does the application also set the desired target ccsid in its MQMD argument before each MQGET call? |
|
| Back to top |
|
|
| sumithar |
| Posted: Sat Sep 22, 2012 2:30 pm Post subject: Re: GET converts EBCDIC to ASCII? |
|
|
Apprentice
Joined: 13 Jun 2008
Posts: 47
|
| rekarm01 wrote: |
| sumithar wrote: |
The options before the GET are as follows:
| Code: |
COMPUTE MQGMO-OPTIONS = MQGMO-NONE +
MQGMO-NO-WAIT +
MQGMO-SYNCPOINT +
MQGMO-CONVERT +
MQGMO-FAIL-IF-QUIESCING |
|
The MQGMO-NONE is harmless, but unnecessary.
Does the application also set the desired target ccsid in its MQMD argument before each MQGET call? |
To be perfectly frank I don't know how one would set the CCSID before the MQGET. Is it using the MQMD-CODEDCHARSETID variable? I don't do anything with that in the program. The only options I set before the MQGET are what I listed above. So I think the answer is no! |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sat Sep 22, 2012 6:07 pm Post subject: Re: GET converts EBCDIC to ASCII? |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| sumithar wrote: |
| To be perfectly frank I don't know how one would set the CCSID before the MQGET. Is it using the MQMD-CODEDCHARSETID variable? |
Yes. The MQMD in the queued message provides the source ccsid, and the MQMD in the MQGET MsgDesc parameter provides the target ccsid. Both are necessary to convert a message from the one ccsid to the other.
The MQGET MsgDesc (MQMD) parameter is an input/output parameter; MQGET overwrites the parameter for each call. If the application reuses the parameter, it needs to make sure that it's set correctly for each call. Consult the InfoCenter APG and APR for more details. |
|
| Back to top |
|
|
| sumithar |
| Posted: Mon Sep 24, 2012 1:48 pm Post subject: Re: GET converts EBCDIC to ASCII? |
|
|
Apprentice
Joined: 13 Jun 2008
Posts: 47
|
| rekarm01 wrote: |
| sumithar wrote: |
| To be perfectly frank I don't know how one would set the CCSID before the MQGET. Is it using the MQMD-CODEDCHARSETID variable? |
Yes. The MQMD in the queued message provides the source ccsid, and the MQMD in the MQGET MsgDesc parameter provides the target ccsid. Both are necessary to convert a message from the one ccsid to the other.
The MQGET MsgDesc (MQMD) parameter is an input/output parameter; MQGET overwrites the parameter for each call. If the application reuses the parameter, it needs to make sure that it's set correctly for each call. Consult the InfoCenter APG and APR for more details. |
Before doing the MQGET I manually changed the value of the field MQMD-CODEDCHARSETID to 500 (EBCDIC) from the value of 819 (ascii) that the message has in its MQMD.
But despite this it still GETs ASCII data which is what it writes out as well.
I'll take a closer look at the links you have posted.
Thanks! |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Wed Sep 26, 2012 12:05 am Post subject: Re: GET converts EBCDIC to ASCII? |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| sumithar wrote: |
I can browse the messages fine in MQ Explorer on my PC as well as view them using the File Manager product on zOS and in the latter case if I turn on HEX I can see they are all EBCDIC characters.
... When I look at the messages in MQ explorer, the CCSID is 819 for the Unisys messages and 500 for the CICS messages. I know the former means ASCII and the latter EBCDIC. Does this matter? |
If the Unisys message content is EBCDIC, but its message headers claim it's ASCII, then yes, that matters. The MQPUT application must provide a source ccsid in the message header that correctly describes the message content (hex), when it puts a message on a queue. For example:- ccsid=500, content=EBCDIC (Intl.)
- ccsid=819, content=ASCII (ISO 8859-1)
If the MQPUT application(s) provides the correct source ccsid for a message, then the MQGET application can provide a target ccsid in the MQGET MsgDesc (MQMD) parameter, and set the MQGMO convert option, in order to convert the message.
If the MQPUT application(s) don't provide the correct source ccsid, then the MQGET application can't rely on the ccsid, and shouldn't try to convert it. It has to guess how to interpret the message. It could, for example, just assume that all the messages are EBCDIC, and write them out that way. But it's better to fix the MQPUT application(s). |
|
| Back to top |
|
|
| sumithar |
| Posted: Mon Oct 08, 2012 9:51 am Post subject: |
|
|
Apprentice
Joined: 13 Jun 2008
Posts: 47
|
OK, so I got to the bottom of this. What happens is that if the message read from the queue is greater than the specified length in the variable buf-len then somehow the MQGMO-CONVERT option gets "corrupted", i.e. the conversion doesn't happen anymore.
| Code: |
CALL MQGET USING WS-MQ-HCONN
WS-MQ-HOBJ
MQMD
MQGMO
[b]WS-MQ-BUF-LEN[/b]
WS-RRBF362-LAYOUT
WS-MQ-DATA-LEN
WS-MQ-CC
WS-MQ-RC |
So the original messages were put in the queue as ASCII and if there is no issue with the length (i.e. actual message length is <= to the WS-MQ-BUF-LEN) then the conversion to EBCDIC happens.
Why the Unisys side puts messages in the queue that are longer than the WS-MQ-BUF-LEN is a different discussion! |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Mon Oct 08, 2012 4:56 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
Thanks for the followup post.
| sumithar wrote: |
| What happens is that if the message read from the queue is greater than the specified length in the variable buf-len then somehow the MQGMO-CONVERT option gets "corrupted", i.e. the conversion doesn't happen anymore. |
This illustrates why it's a good idea to always check the Completion Code and Reason Code, after calling any of the MQ functions. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Oct 08, 2012 7:30 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
There a variety of reason codes for each mqi call.
Mq returned a failed reason code alerting the programmer that the message was to big for the buffer. I'm guessing that the app didn't test for this type of failure.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Oct 09, 2012 5:20 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| sumithar wrote: |
OK, so I got to the bottom of this. What happens is that if the message read from the queue is greater than the specified length in the variable buf-len then somehow the MQGMO-CONVERT option gets "corrupted", i.e. the conversion doesn't happen anymore.
|
Nothing got corrupted.
Unless the accept-truncated-message GMO is specified, the get call fails when MSG length exceeds buffer size. If the call fails, no MSG is delivered, therefore no message can be converted.
.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Oct 09, 2012 5:39 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
To put bruce's answer in a slightly different form.
The message didn't get corrupted.
It was invalid after the MQGet becuase the MQGet failed.
So you then tried to access invalid data, which would obviously not return the expected results. |
|
| Back to top |
|
|
|
|
