| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| JMS, VB and CCSID |
« View previous topic :: View next topic » |
| Author |
Message
|
| PeterPotkay |
 Posted: Sat Feb 20, 2010 10:17 am Post subject: JMS, VB and CCSID |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
MQ Client application is running on Solaris, programming in JMS. An RFH2 header is present in the messages it produces. The MQMD Format field is set to MQRFH2.
Original Queue Manager is Windows, its CCSID is set to 437.
New Queue Manager is Linux x86-64, its CCSID is set to 1208.
When the client application connects to the Windows QM, the MQMD CCSID is set to 437. 
When the client application connects to the Windows QM, the MQMD CCSID is set to 1208.
No JMS side changes when switching between the 2 QMs, yet the message's CCSID changes.
The consuming application is another MQ Client running on its own Windows machine. Running VB6 I think. It consumes the 437 messages just fine off the Windows QM. But when it consumes the 1208 messages off the Linux QM, its dies with MQRC 2119.
These 2 MQ Client apps were cobbled together years ago via the Windows QM, but they need to move off of it and onto the Linux QM, but the conversion errors have roadblocked us.
When that JMS app puts the messages, why is the MQMD CCSID not 819, which is the CCSID of the client server?
When I look at the messages via MO71 everything looks identical other than the MQMD CCSID. I can view both messages just fine, so why is the VB app getting a 2119?
Using the MO71 Client API Exerciser, I put some test messages to both QMs and they always show up as CCSID 437, which is my Windows XP's CCSID. That's expected. I don't see any value I can specify on the MQPUT for the CCSID that causes the message to inherit the QM's CCSID. What JMS magic is in play here that is causing the messages to be tagged with the QM's CCSID and not the CCSID of the client server the JMS app is running on?
I think this worked all these years because the JMS app was incorrectly getting the CCSID set to 437 and that's what the VB app was running as. For some reason the VB app is choking when the messages are tagged 1208. I wonder if it would choke on them if they were tagged as 819, which is what I think they should have been all along.
| Quote: |
MQRC_NOT_CONVERTED
Explanation:
An MQGET call was issued with the MQGMO_CONVERT option specified in the GetMsgOpts parameter, but an error occurred during conversion of the data in the message. The message data is returned unconverted, the values of the CodedCharSetId and Encoding fields in the MsgDesc parameter are set to those of the message returned, and the call completes with MQCC_WARNING.
If the message consists of several parts, each of which is described by its own CodedCharSetId and Encoding fields (for example, a message with format name MQFMT_DEAD_LETTER_HEADER), some parts may be converted and other parts not converted. However, the values returned in the various CodedCharSetId and Encoding fields always correctly describe the relevant message data.
This error may also indicate that a parameter to the data-conversion service is not supported.
Completion Code:
MQCC_WARNING
Programmer Response:
Check that the message data is correctly described by the Format, CodedCharSetId and Encoding parameters that were specified when the message was put. Also check that these values, and the CodedCharSetId and Encoding specified in the MsgDesc parameter on the MQGET call, are supported for queue-manager conversion. If the required conversion is not supported, conversion must be carried out by the application.
|
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Sat Feb 20, 2010 7:00 pm Post subject: Re: JMS, VB and CCSID |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| PeterPotkay wrote: |
When that JMS app puts the messages, why is the MQMD CCSID not 819, which is the CCSID of the client server? |
Because the message takes the CCSID of the Connection factory. That parameter can be setup in JNDI. The message can also have the CCSID of the destination (queue) which can also be set on the JNDI of the destination. 
| PeterPotkay wrote: |
When I look at the messages via MO71 everything looks identical other than the MQMD CCSID. I can view both messages just fine, so why is the VB app getting a 2119?
Using the MO71 Client API Exerciser, I put some test messages to both QMs and they always show up as CCSID 437, which is my Windows XP's CCSID. That's expected. I don't see any value I can specify on the MQPUT for the CCSID that causes the message to inherit the QM's CCSID. What JMS magic is in play here that is causing the messages to be tagged with the QM's CCSID and not the CCSID of the client server the JMS app is running on?
I think this worked all these years because the JMS app was incorrectly getting the CCSID set to 437 and that's what the VB app was running as. For some reason the VB app is choking when the messages are tagged 1208. I wonder if it would choke on them if they were tagged as 819, which is what I think they should have been all along.
| Quote: |
MQRC_NOT_CONVERTED
Explanation:
An MQGET call was issued with the MQGMO_CONVERT option specified in the GetMsgOpts parameter, but an error occurred during conversion of the data in the message. The message data is returned unconverted, the values of the CodedCharSetId and Encoding fields in the MsgDesc parameter are set to those of the message returned, and the call completes with MQCC_WARNING.
If the message consists of several parts, each of which is described by its own CodedCharSetId and Encoding fields (for example, a message with format name MQFMT_DEAD_LETTER_HEADER), some parts may be converted and other parts not converted. However, the values returned in the various CodedCharSetId and Encoding fields always correctly describe the relevant message data.
This error may also indicate that a parameter to the data-conversion service is not supported.
Completion Code:
MQCC_WARNING
Programmer Response:
Check that the message data is correctly described by the Format, CodedCharSetId and Encoding parameters that were specified when the message was put. Also check that these values, and the CodedCharSetId and Encoding specified in the MsgDesc parameter on the MQGET call, are supported for queue-manager conversion. If the required conversion is not supported, conversion must be carried out by the application.
|
|
Question number 1 Does the message need the RFH header? if not why don't you have it suppressed. JNDI of detestination set to targetClient = MQ (1). If the destination is created from the session with a queue name use the uri form instead and set the attribute queue://qmgr/queue?targetClient=1 ...
What I guess is happening is the the OS (Windows with VB) does not support CCSID 1208 or UTF-8. Make sure the windows environment has been correctly configured to at least understand UTF-8...
Hope this helps some.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun Feb 21, 2010 5:19 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
But FJ, why is the CCSID of the message changing depening on which QM they are connected to? When they, as Solaris, connect to the Windows QM they produce messages with CCSID of 437. When they change the hostname parm of their QCF to go to the Linux QM the messages show up as 1208.
They don't even know the O/S of the QMs. It shouldn't matter so I didn't tell them, so I'm pretty confident they are not changing the CCSID in the QCF to match the QM. I did ask for a copy of their JMSAdmin input and the .bindings file so I can review it.
Is there a CCSID setting in the QCF that would cause the messages to inherit the CCSID of the QM?
Does JMS do "conversion on MQPUT"? In other words if the Solaris CCSID 819 MQ Client does a put with 437 or 1208 in the CCSID of the QCF, will the messages be placed on the queue as 437 or 1208 and tagged correctly as 437 or 1208?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Feb 21, 2010 1:59 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| PeterPotkay wrote: |
But FJ, why is the CCSID of the message changing depening on which QM they are connected to? When they, as Solaris, connect to the Windows QM they produce messages with CCSID of 437. When they change the hostname parm of their QCF to go to the Linux QM the messages show up as 1208.
They don't even know the O/S of the QMs. It shouldn't matter so I didn't tell them, so I'm pretty confident they are not changing the CCSID in the QCF to match the QM. I did ask for a copy of their JMSAdmin input and the .bindings file so I can review it.
Is there a CCSID setting in the QCF that would cause the messages to inherit the CCSID of the QM? |
Like in all things software there is a default. If the CCSID is not specified on the qcf/tcf the message will take by default the CCSID of the qmgr.
If the qcf/tcf has a CCSID specified the message produced will take the CCSID specified on the qcf/tcf. If the DESTINATION has a CCSID specified the message will be put to the destination with the CCSID specified on the destination.
| PeterPotkay wrote: |
| Does JMS do "conversion on MQPUT"? In other words if the Solaris CCSID 819 MQ Client does a put with 437 or 1208 in the CCSID of the QCF, will the messages be placed on the queue as 437 or 1208 and tagged correctly as 437 or 1208? |
Refer to the above. If the message is placed on the queue with a ccsid of 1208 on the QCF (no CCSID specification for the destination) the message will be accepted from java and put to the buffer in UTF-8 and written to the queue / destination with the correct CCSID:1208 ... regardless of the CCSID of the qmgr.
Now remember that the appserver's JNDI can possibly overwrite the settings in the .bindings file to some extent... It is unlikely but I would just double check it to be sure.
Hope this clarifies things...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Feb 21, 2010 2:02 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I've had a qmgr on solaris with CCSID 819 linked to a zOS qmgr. No transformation on the channels.
Text message was put in JMS. Default CCSID on the qcf, overide with ccsid 500 on the Destination (remote queue) setup in .bindings file. When the message arrived on the MF it had CCSID 500.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun Feb 21, 2010 3:30 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| fjb_saper wrote: |
I've had a qmgr on solaris with CCSID 819 linked to a zOS qmgr. No transformation on the channels.
Text message was put in JMS. Default CCSID on the qcf, overide with ccsid 500 on the Destination (remote queue) setup in .bindings file. When the message arrived on the MF it had CCSID 500. |
It was tagged as 500 in the MQMD. Was the actual data sitting in EBCDIC at that point? In other words, the Solaris JMS app running as 819 spit out EBCDIC CCSID 500 data because it was instructed to? If true, this means JMS can do a "convert on put" out of the box without customer built conversion code doing the work.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun Feb 21, 2010 3:44 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| fjb_saper wrote: |
| PeterPotkay wrote: |
But FJ, why is the CCSID of the message changing depening on which QM they are connected to? When they, as Solaris, connect to the Windows QM they produce messages with CCSID of 437. When they change the hostname parm of their QCF to go to the Linux QM the messages show up as 1208.
They don't even know the O/S of the QMs. It shouldn't matter so I didn't tell them, so I'm pretty confident they are not changing the CCSID in the QCF to match the QM. I did ask for a copy of their JMSAdmin input and the .bindings file so I can review it.
Is there a CCSID setting in the QCF that would cause the messages to inherit the CCSID of the QM? |
Like in all things software there is a default. If the CCSID is not specified on the qcf/tcf the message will take by default the CCSID of the qmgr.
|
In an MQ Client scenario, if I accept the above as true, that would mean JMS acts different than every other MQ API. All other MQ APIs put the message in the CCSID of the MQ Client platform when "QMGR DEFAULT" is specified for the CCSID on the put. For example, do an amqsputc from your desktop to a z/OS QM. The message arrives as 437, even though the "QMGR DEFAULT" option was specified for the CCSID on the put.
| fjb_saper wrote: |
If the qcf/tcf has a CCSID specified the message produced will take the CCSID specified on the qcf/tcf. If the DESTINATION has a CCSID specified the message will be put to the destination with the CCSID specified on the destination.
|
All MQ APIs will allow this, but having the MQMD say the CCSID is nnn and actually having the data be CCSID nnn to match when its a different CCSID than the MQ Client platform are 2 different things. If I understand you correctly (see previos post), JMS has the ability to convert on put?
I'm still waiting for them to send me their .bindings file. And I'll do some tracing tommorrow with them. I'm still curious how their message's MQMD CCSID changes from 437 to 1208 when they change the QM they connect if they keep the same CCSID value in their QCFs and destinations.
The Windows app is very old. I bet when this thing was first implemented, they did something to inherit the QM's CCSID which was a Windows QM, the mesages came out as 437, and that worked for that Windows app. Now connected to the Linux QM they are inheriting 1208 and that no longer works for that old VB app. If JMS acts different in MQ Client mode when you ask for the deafult QM CCSID (you actually get the QMGR's CCSID!), this would explain it all.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Feb 21, 2010 8:28 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| PeterPotkay wrote: |
| fjb_saper wrote: |
I've had a qmgr on solaris with CCSID 819 linked to a zOS qmgr. No transformation on the channels.
Text message was put in JMS. Default CCSID on the qcf, overide with ccsid 500 on the Destination (remote queue) setup in .bindings file. When the message arrived on the MF it had CCSID 500. |
It was tagged as 500 in the MQMD. Was the actual data sitting in EBCDIC at that point? In other words, the Solaris JMS app running as 819 spit out EBCDIC CCSID 500 data because it was instructed to? If true, this means JMS can do a "convert on put" out of the box without customer built conversion code doing the work. |
You got it. The qmgr translates the TextMessage to the CCSID of the Destination (queue/topic) before doing the put.
_________________
MQ & Broker admin
Last edited by fjb_saper on Sun Feb 21, 2010 8:37 pm; edited 1 time in total |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Feb 21, 2010 8:36 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Quote: |
| In an MQ Client scenario, if I accept the above as true, that would mean JMS acts different than every other MQ API. All other MQ APIs put the message in the CCSID of the MQ Client platform when "QMGR DEFAULT" is specified for the CCSID on the put. For example, do an amqsputc from your desktop to a z/OS QM. The message arrives as 437, even though the "QMGR DEFAULT" option was specified for the CCSID on the put. |
Like .NET, C, and other languages Java does not have a CCSID, but it has inherent conversion to the CCSID of the platform it is running on.
So if you don't specify a CCSID on the qcf / tcf you will have to check what CCSID your message will have. I was thinking the qmgr's but it might be the client platform's... and should be the qmgr's in bindings mode??
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Feb 22, 2010 5:30 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| fjb_saper wrote: |
| PeterPotkay wrote: |
| fjb_saper wrote: |
I've had a qmgr on solaris with CCSID 819 linked to a zOS qmgr. No transformation on the channels.
Text message was put in JMS. Default CCSID on the qcf, overide with ccsid 500 on the Destination (remote queue) setup in .bindings file. When the message arrived on the MF it had CCSID 500. |
It was tagged as 500 in the MQMD. Was the actual data sitting in EBCDIC at that point? In other words, the Solaris JMS app running as 819 spit out EBCDIC CCSID 500 data because it was instructed to? If true, this means JMS can do a "convert on put" out of the box without customer built conversion code doing the work. |
You got it. The qmgr translates the TextMessage to the CCSID of the Destination (queue/topic) before doing the put. |
This might be splitting hairs, but....
Its not the QM that does the convert on put in this case, its the JMS libraries? If it were the QM, then I think all MQ API languages would allow convert on put, but they don't. Only JMS does.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Feb 22, 2010 12:33 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| PeterPotkay wrote: |
This might be splitting hairs, but....
Its not the QM that does the convert on put in this case, its the JMS libraries? If it were the QM, then I think all MQ API languages would allow convert on put, but they don't. Only JMS does. |
Not quite. All conversions are done by the MQ/OS libraries. The request for the conversion however, would be done by the JMS libraries.
So yes it is the QM that does the convert on put. The request for the convert on put is done by the JMS libraries.
Note that contrary to MQ API and Java base, in JMS you very seldom set a CCSID on a message. It is set on the qcf or the destination...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Feb 22, 2010 1:42 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I'm not sure on your police work there, FJ...
I strongly suspect that the CCSID conversion is being done inside the JMS code when the TextMessage implementation serializes the String data into bytes.
I bet an MQ Trace would show you the buffer being passed to the MQPUT/MQPUT1 and this would tell you for sure... |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Feb 22, 2010 2:49 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
I don't know for sure, but I'll side with mqjeff, if only because if the MQ libs did allow for this conversion, than all MQ API programming languages would allow us to convert on put, be we can't. It must be inside the JMS libs. Which is pretty cool.
I wasn't able to work with them today, but I will post the results as soon as I do.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Feb 22, 2010 9:00 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mqjeff wrote: |
I'm not sure on your police work there, FJ...
I strongly suspect that the CCSID conversion is being done inside the JMS code when the TextMessage implementation serializes the String data into bytes.
I bet an MQ Trace would show you the buffer being passed to the MQPUT/MQPUT1 and this would tell you for sure... |
Jeff, My guess is that the buffer has already the target CCSID.
My point was that I figured that the call to the conversion routines was being done by JMS but that the implementation of the conversion routine was the standard used by the QMGR... different way of saying the same thing I guess...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Feb 23, 2010 4:12 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Looking at their .bindings file where they switched to the new Linux QM, I see all the QCFs are 819, and the destinations are 1208. They use the JMSAdmin tool and looking at their input script, they do not set the CCSID, so they are getting the default values.
http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/topic/com.ibm.mq.csqzaw.doc/uj10910_.htm
This says 819 and 1208 are the defaults
OK! That explains what they are getting now. It has nothing to do with the fact the QM is 1208, the destinations are set to 1208.
BUT, why is their Production app spitting out 437 messages? I asked for their production files, but I wonder if its related to the version of JMSAdmin being used. When I look at the MQ 5.3 Java manual
http://publibfp.dhe.ibm.com/epubs/pdf/csqzaw12.pdf
, there is no default value for CCSID. I assume that means its set to 0 by default, and in JMS land that means use the QM CCSID, even for an MQ Client? If so, that explains the 437s when connected to the Windows QM.
Perhaps when they ran JMSAdmin to make the change to aim at my new MQ server, they used a 6.0 version of it and it changed all the default CCSIDs from 0 (which is what they had at JMSAdmin 5.3?) to 819 and 1208, the defaults for JMSAdmin 6.0???
I'll let you know once I have a gander at their production config files. And yes, I already told them they should start setting all their values explicitly. Never rely on defaults, cause apparently they can change!
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2 Next |
Page 1 of 2 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
