| Author |
Message
|
| Karan |
 Posted: Sun Oct 21, 2012 11:07 pm Post subject: Convert Option MQInputNode |
 |
|
Apprentice
Joined: 21 Oct 2009
Posts: 29
|
Hi,
I have ticked the convert option on the MQInput node.
(ccsid and encoding below are 0)
I have a message with an mqrfh2 header (and usr folder)
Im expecting the node to convert the input message to ccsid and encoding of Qmgr
What I observe is -
Properties ccsid and encoding are same as input msg (eg 437,546)
MQMD ccsid and ecoding are converted to that of qmgr (eg 819,273)
RFH2 ccsid and encoding are same as input msg (437,546)
Shouldnt all of them change to that of the Qmgr ?
Also if I pass EBCDIC message, will a conversion happen for the payload at the input node ?
Based on what Iv observed above , conversion may not happen. (If conversion does happen it should set Properties , mqrfh2 ccsid/encoding to that of qmgr) |
|
| Back to top |
|
 |
| Esa |
| Posted: Sun Oct 21, 2012 11:33 pm Post subject: Re: Convert Option MQInputNode |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| Karan wrote: |
Im expecting the node to convert the input message to ccsid and encoding of Qmgr
|
Your expectation is incorrect. The broker converts all messages to unicode for internal processing, unless you have configured it to use some other codepage. Regardless of if you have selected the convert option or not. In general, you select the convert option only if you want the get operation to run some exit defined in MQ
The CCSID and Encoding fields in Properties, MQMD and other header folders are used for recording the codepages and encodings of the input message and its headers. They also tell the MQOutput node how to serialeze the message and the headers. So if you wire an MQOutput node after the MQInput and don't modify the headers, the message in the output queue will be converted back to exactly the same codepages as the input message. |
|
| Back to top |
|
|
| Karan |
| Posted: Sun Oct 21, 2012 11:46 pm Post subject: |
|
|
Apprentice
Joined: 21 Oct 2009
Posts: 29
|
I did read somewhere here that ccsid and encoding of 0 will use default values.
Documentation says
"Use the CCSID and encoding default values, because they take their values from the queue manager's encoding and CCSID settings" (default values being 0)
Hence thats what I expect the MQInput node to do.
And it does seem to be kind of doing that (atleast if I try to read a message without rfh2 , both properties and mqmd have been set to that of qmgr)
But with rfh2 Im not that sure |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Oct 22, 2012 12:00 am Post subject: Re: Convert Option MQInputNode |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
This:-
| Karan wrote: |
Properties ccsid and encoding are same as input msg (eg 437,546)
MQMD ccsid and ecoding are converted to that of qmgr (eg 819,273)
RFH2 ccsid and encoding are same as input msg (437,546)
|
and this
| Karan wrote: |
Also if I pass EBCDIC message, will a conversion happen for the payload at the input node ?
|
don't seem right.
CCSID's 437 and 819 are for data in ISO Latin type format and most certainly not EBCIDIC. If I saw EBCIDIC data being sent to me with a CCSID of 437 or 819 I'd start talking to the sending app devs before trying to correct their mistakes in broker. Ideally, they should send their data with a CCSId that actually reflects the EBCIDIC data in the payload.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Oct 22, 2012 12:41 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
The MQInput node convert option will perform the same conversion as the option MQGMO_CONVERT does, because it just sets that option for the MQGET.
If you don't use it, then WMB will perform the data conversion from the message data to Unicode.
However I have seen cases where selecting it has fixed a conversion problem (maybe MQ's conversion options are more comprehensive than WMB's).
| Quote: |
If you select this check box, WebSphere MQ converts data in the message to be received, in conformance with the CodedCharSetId and Encoding values set in the MQMD. Select Convert if you want WebSphere MQ to perform data conversion on the message when it is retrieved from the queue. This option is useful if you are handling messages in the BLOB domain, or are using a user-defined parser. Do not select this option if you are parsing messages with the XML or MRM domains, because the parser does the conversion.
WebSphere MQ converts the incoming message to the encoding and coded character set that is specified in the MQMD that the input node supplies on the MQGET call to retrieve the message from the input queue. The broker uses the MQGMO_CONVERT option on the MQGET call; typical rules for WebSphere MQ conversion apply. The values that you specify in the Convert encoding and Convert coded character set ID options are used in the MsgDesc Encoding and CCSID fields in the MQGET call. WebSphere MQ can convert the incoming message only if the MQMD Format field is a built-in WebSphere MQ value that identifies character data, or if a data conversion exit exists in WebSphere MQ.
This property maps to the MQGMO_CONVERT option of the MQGMO of the MQI.
Clear the check box if you do not want WebSphere MQ to convert the message.
If you select this check box, you can also specify values for the Convert encoding and Convert coded character set ID properties.
For more information about WebSphere MQ data conversion, and why you might choose to use this option, see the Application Programming Guide section of the WebSphere MQ Version 7 Information Center online.
|
Last edited by zpat on Mon Oct 22, 2012 12:49 am; edited 1 time in total |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Oct 22, 2012 12:46 am Post subject: Re: Convert Option MQInputNode |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| smdavies99 wrote: |
This:-
Ideally, they should send their data with a CCSId that actually reflects the EBCIDIC data in the payload. |
It's essential - otherwise the resulting conversion (or lack of) might be amusing to see - but not much use. |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Oct 22, 2012 12:50 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| However I have seen cases where selecting it has fixed a conversion problem (maybe MQ's conversion options are more comprehensive than WMB's). |
I've seen lots of cases where fiddling with conversion options in MQ/WMB has fixed a problem. That doesn't mean that the resulting solution was the best solution.
For the record, the conversion capabilities of MQ and WMB are close to identical - they both support almost any character encoding that the wit of man has devised.
The most important question here is the one raised by smdavies99 - is the input message telling lies about its character encoding? |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Oct 22, 2012 1:01 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Out of interest - does using the MQinput convert option cause a double conversion, i.e. once to the QM codepage (typically ASCII) (by MQ) and then again from that to Unicode (by WMB)? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Oct 22, 2012 1:17 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Everything populated in the logical message tree, to the best of my knowledge, is populated in Unicode - CCSID 1200ish.
To the best of my knowledge, when you set Convert on MQInput, it will tell MQ to convert *to* unicode - i.e. set the ccsid on the mqmd passed into 1200ish.
So I don't think using both does a "double" convert, per se. |
|
| Back to top |
|
|
| Karan |
| Posted: Mon Oct 22, 2012 1:31 am Post subject: |
|
|
Apprentice
Joined: 21 Oct 2009
Posts: 29
|
The confusion here is
Case 1 -->
Pass a message using rfhutil - by default ccsid of 437 is chosen.
I write this to a queue from which broker picks up using the convert option(ccsid 0 encoding 0 )
Properties ccsid and encoding are same as input msg (eg 437,546)
MQMD ccsid and ecoding are converted to that of qmgr (eg 819,273)
RFH2 ccsid and encoding are same as input msg (437,546)
Case 2 -->
Pass a EBCDIC message using rfhutil with ccsid 500 (rfh2 ccsid also 500)
When I do the same thing as above I can see that
ccsid , encoding of rfh2 , mqmd and properties all 3 are 819 , 273 (same as qmgr)
Case 2 is what I expected. But whys it not that way in case 1 ? |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Mon Oct 22, 2012 2:22 am Post subject: Re: Convert Option MQInputNode |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| Karan wrote: |
| I'm expecting the node to convert the input message to ccsid and encoding of Qmgr |
WMQ does not necessarily convert the entire message:
| Quote: |
| 6. For messages consisting of one or more MQ header structures followed by user data, one or more of the header structures might be converted, while the remainder of the message is not. However, ... the CodedCharSetId and Encoding fields in each header structure always correctly indicate the character set and encoding of the data that follows the header structure. |
So, if the message has multiple headers, don't expect WMQ to convert all of it.
| Karan wrote: |
What I observe is -
Properties ccsid and encoding are same as input msg (eg 437,546)
MQMD ccsid and encoding are converted to that of qmgr (eg 819,273)
RFH2 ccsid and encoding are same as input msg (437,546) |
Don't confuse converting an element with modifyng its value. WMQ converts the character and numeric data in the fixed portion of the MQRFH2 header itself, and then modifies the MQMD ccsid and Encoding to describe the resulting MQRFH2 header. If WMQ converts the message data that follows the MQRFH2 header, then it would modify the MQRFH2 ccsid and Encoding to describe the resulting message data. (The Properties ccsid and Encoding should always describe the message data, regardless of the number of message headers.)
| Karan wrote: |
| Case 2 is what I expected. But whys it not that way in case 1? |
It is possible that the WMQ built-in conversion exits won't perform an optional conversion unless it's safe to do so. ccsids 500 and 819 use the same character set, there's no risk of data loss when converting from one to the other; the same is not true for ccsids 437 and 819. Or, it's possible that WMQ may decide that a conversion between two ASCII-based ccsids is unnecessary, while conversion between ASCII and EBCDIC is not. Or maybe it's for some other reason ...
WMB allows for much greater control over what gets converted and what doesn't; that's one reason why it's usually better to let WMB convert the message, rather than WMQ. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Mon Oct 22, 2012 2:43 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| zpat wrote: |
| Out of interest - does using the MQinput convert option cause a double conversion, i.e. once to the QM codepage (typically ASCII) (by MQ) and then again from that to Unicode (by WMB)? |
Yes. If the MQInput convert option is checked, then WMQ attempts to convert the input message to the ccsid and encoding specified in the MQInput node properties, (where '0' represents the qmgr default values). WMQ conversion takes place before the MQInput node receives the resulting bitstream.
The WMB parsers may later convert the bitstream when parsing the input message (input bitstream -> message tree), and yet again when when writing the output message (message tree -> output bitstream). |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Mon Oct 22, 2012 10:01 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| zpat wrote: |
| However I have seen cases where selecting it has fixed a conversion problem (maybe MQ's conversion options are more comprehensive than WMB's). |
One difference is how WMQ and WMB identifies character/numeric data in the message data. WMQ uses the MQ header Format to specify either a built-in or user-defined conversion exit to convert the message data. WMB uses the Message Domain to specify a parser to convert the message data, (which may also require a message model or message set).
For example:- If Format = MQFMT_STRING and Message Domain = BLOB, then WMQ could convert the message data, but WMB would not, since the BLOB parser does not identify any character/numeric data.
- But if Format = MQFMT_NONE and Message Domain = DFDL, then WMB could convert the message data, but WMQ would not, since the MQFMT_NONE format does not identify any character/numeric data.
|
|
| Back to top |
|
|
|
|
