ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexIBM MQ Java / JMSIt is impossible to set correct CCSID

Post new topicReply to topic Goto page 1, 2  Next
It is impossible to set correct CCSID View previous topic :: View next topic
Author Message
Jenum
PostPosted: Tue Sep 25, 2018 5:11 am Post subject: It is impossible to set correct CCSID Reply with quote

Novice

Joined: 13 Nov 2012
Posts: 22

Hi all.
I want to send message with CCSID=1251 with some properties:

Code:

queueManager = ...;
int openOptions = MQConstants.MQOO_INPUT_SHARED | MQConstants.MQOO_OUTPUT | MQConstants.MQOO_BROWSE;
MQQueue queue = queueManager.accessQueue(..., openOptions);
MQPutMessageOptions pmo = new MQPutMessageOptions();
pmo.options = CMQC.MQPMO_NO_SYNCPOINT;

MQMessage message = new MQMessage();
message.characterSet = 1251;
message.write(Charsets.convert("Some text", message.characterSet));

message.setStringProperty("first", "1");
message.setStringProperty("second", "2");
message.setStringProperty("third", "3");

queue.put(message, pmo);


But message in queue has CCSID=1208 (in MQMD and MQRFH2 headers).

After some research, i found, that problem in methods MQMessage#writePropertiesRfh2 and MQHeaderList#write (which call in MQDestination#put): first set CCSID=1208 in MQMD and second set similar characterSet in all headers.

I use com.ibm.mq.allclient.jar with version 8.0.0.8.

Is it correct behaviour or some mistake in IBM libraries?
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Sep 25, 2018 5:50 am Post subject: Re: It is impossible to set correct CCSID Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25266
Location: Ohio, USA

Jenum wrote:
Is it correct behaviour or some mistake in IBM libraries?


Sounds correct to me (given the caveat my Java isn't first class).

The CCSID in the MQMD defines the code page for the payload of the message, which from the MQMD's point of view includes the RFH2. The RFH2 CCSID contains additional descriptive information for the actual payload including the CCSID of the data.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Jenum
PostPosted: Tue Sep 25, 2018 5:53 am Post subject: Reply with quote

Novice

Joined: 13 Nov 2012
Posts: 22

Please, find this:
Quote:
(in MQMD and MQRFH2 headers)


There are no problem if just MQMD contains characterSet = 1208, but MQRFH2.characterSet = 1208 too, so original value (1251) is missing.
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Sep 25, 2018 7:51 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25266
Location: Ohio, USA

Jenum wrote:
Please, find this:
Quote:
(in MQMD and MQRFH2 headers)


There are no problem if just MQMD contains characterSet = 1208, but MQRFH2.characterSet = 1208 too, so original value (1251) is missing.


I don't see anywhere in the code you specific indicating that the RFH CCSID should be 1251. You set that for the message, but not in the header.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Jenum
PostPosted: Tue Sep 25, 2018 7:54 am Post subject: Reply with quote

Novice

Joined: 13 Nov 2012
Posts: 22

I don't create MQRFH2-header manually, i just set some properties. Is there any chance to set neccesary characterSet in this way?
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Sep 25, 2018 8:12 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25266
Location: Ohio, USA

Urgh.

How does your queue manager handle properties? Are you sure you're getting an RFH header in the message or are the properties native on the message?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Jenum
PostPosted: Tue Sep 25, 2018 8:25 am Post subject: Reply with quote

Novice

Joined: 13 Nov 2012
Posts: 22

Quote:
How does your queue manager handle properties?

Which property of queue manager you mean?

I try to send message to queue (MQPUT) with non-1208 ccsid body and some properties. For some reason, i can't set message properties by MQRFH2 object, only by setProperty-methods of MQMessage object. But how to set necessary body ccsid in this case?
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Sep 25, 2018 9:22 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25266
Location: Ohio, USA

Jenum wrote:
Quote:
How does your queue manager handle properties?

Which property of queue manager you mean?


I mean how is your queue manager set to handle properties?

If it's not in a compatibility setting, you won't get an RFH2.

Failing that, you need someone who knows more Java than I do.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
tczielke
PostPosted: Tue Sep 25, 2018 10:37 am Post subject: Reply with quote

Shaman

Joined: 08 Jul 2010
Posts: 739
Location: Illinois, USA

I see similar behavior to what Jenum is reporting.

If I use one of my MQ base Java samples to set the MQMessage.characterSet=1251 and then do a PUT without setting any message properties, the message is PUT on the queue with CCSID 1251.

However, if I then add in a MQMessage.setStringProperty call before the PUT and run the sample again, I now get a message with CCSID 1208, even with the MQMessage.characterSet being set to 1251.

To me this looks like a defect. I don't see this behavior documented anywhere in the MQ manual that calling a MQMessage.setStringProperty would alter the CCSID of the message that is PUT to the queue.

Jenum - You may want to open a PMR to have IBM investigate why this is happening.
_________________
MQ administrator since 2010.
Back to top
View user's profile Send private message
rekarm01
PostPosted: Wed Sep 26, 2018 7:10 pm Post subject: Re: It is impossible to set correct CCSID Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1308

Is there any particular reason for using the IBM MQ classes for Java, rather than the IBM MQ classes for JMS?

Jenum wrote:
I want to send message with CCSID=1251

The MQMD class supports the following values for characterSet: CMQC.MQCCSI_Q_MGR, 850, 819, 37, 1200, and 1208. That's not to say that 1251 wouldn't work, only that it's not documented, and might not be supported. Does the same problem occur with one of these other, documented characterSet values?

Jenum wrote:
Code:
MQMessage message = new MQMessage();
message.characterSet = 1251;
message.write(Charsets.convert("Some text", message.characterSet));

Setting message.format could help; that way the other classes know that characterSet is important, and should be preserved when chaining headers:

Code:
MQMessage message = new MQMessage();
message.characterSet = 1251;
message.format = MQConstants.MQFMT_STRING;
message.writeString("Some text");

... and writeString() might be easier to use here; it has conversion built in.
Back to top
View user's profile Send private message
Jenum
PostPosted: Wed Sep 26, 2018 9:31 pm Post subject: Reply with quote

Novice

Joined: 13 Nov 2012
Posts: 22

Quote:

Is there any particular reason for using the IBM MQ classes for Java, rather than the IBM MQ classes for JMS?

Our framework to work with MQ already use classes for Java. Another reason is JMS doesn't support some features, like segmentation.

Quote:
Does the same problem occur with one of these other, documented characterSet values?

Yes, problem occurs even with documented 850 CCSID.

Quote:
Setting message.format could help

Sad, but no.

Quote:
... and writeString() might be easier to use here; it has conversion built in.

No, resulting CCSID will be 1208.

After decompiling MQDestination and MQMessage classes, the reason for such problem in method MQMessage#writePropertiesRfh2(int), which forcely set characterSet of message to 1208. So, tczielke is right: it should be PMR to IBM.
Back to top
View user's profile Send private message
Jenum
PostPosted: Wed Sep 26, 2018 9:44 pm Post subject: Reply with quote

Novice

Joined: 13 Nov 2012
Posts: 22

May be it would be helpfull for someone:
my work around for this problem is using reflection (it is very, very bad solution, but i can't found anything else):

Code:

MQMessage msg = new MQMessage();
msg.characterSet = 1251;
msg.setStringProperty(...);
msg.write(Charsets.convert("...", msg.characterSet));

int ccsid = msg.characterSet;
if (ccsid != 1208) {
   try {
      /* Converting message properties in MQRFH2 header */
      Method m = msg.getClass().getDeclaredMethod("writePropertiesRfh2", int.class);
      m.setAccessible(true);
      m.invoke(msg, 1208);

      /* Deleting all message properties, otherwise it would be converted again in put method */
      Enumeration<String> props = msg.getPropertyNames("%");
      while (props.hasMoreElements()) {
         String prop = props.nextElement();
         msg.deleteProperty(getCanonicalPropertyName(prop));
      }

      /* Setting necessary encoding in last (and single: MQRFH2) header */
      msg.seek(0);
      MQHeaderList hList = new MQHeaderList(msg);
      MQRFH2 rfh2 = (MQRFH2) hList.get(0);
      rfh2.setCodedCharSetId(ccsid);
   } catch (NoSuchMethodException | InvocationTargetException | MQDataException e) {
      throw new RuntimeException(e);
   }
}

queue.put(msg, pmo);

if (ccsid != 1208) {
   msg.seek(0);
   try {
      /* i need to return MQMessage with properties. Good solution would be copy of MQMessage before previously steps, but i don't know, how to do it :( */
      Method m = msg.getClass().getDeclaredMethod("performProcessingAfterGet");
      m.setAccessible(true);
      m.invoke(msg);
   } catch (NoSuchMethodException | InvocationTargetException e) {
      throw new RuntimeException(e);
   }
}
return msg;
Back to top
View user's profile Send private message
tczielke
PostPosted: Thu Sep 27, 2018 5:37 am Post subject: Reply with quote

Shaman

Joined: 08 Jul 2010
Posts: 739
Location: Illinois, USA

To help me understand, why is it an issue if the message is PUT on the queue with a CCSID of 1208? 1208 would be able to encode all the character symbols that would be in 1251.
_________________
MQ administrator since 2010.
Back to top
View user's profile Send private message
Jenum
PostPosted: Thu Sep 27, 2018 5:43 am Post subject: Reply with quote

Novice

Joined: 13 Nov 2012
Posts: 22

tczielke, 1251 - just example, we often work with CCSID 866 which doesn't compatible with 1208.
Back to top
View user's profile Send private message
tczielke
PostPosted: Thu Sep 27, 2018 5:52 am Post subject: Reply with quote

Shaman

Joined: 08 Jul 2010
Posts: 739
Location: Illinois, USA

I am not familiar with working with CCSID 866, but from a character symbol stand point, I would think any character symbol encoded in 866 would also be encoded in 1208 ( UTF-8 ). If you are at MQ v9, MQ supports full unicode, so 1208 would be the full 1-4 byte encoding suite of Unicode. Unicode pretty much covers everything you would probably want to work with from a character symbol stand point.

This is what I am observing with the IBM MQ base Java. If you set the MQMessage.characterSet to a supported CCSID, it will PUT the message on the queue with that CCSID. However, if you start using these message property methods, then it switches to putting the message on the queue in 1208 ( UTF-8 ). However, UTF-8 can pretty much encode any character symbol that another CCSID could, as far as I know.
_________________
MQ administrator since 2010.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum IndexIBM MQ Java / JMSIt is impossible to set correct CCSID
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.