| Author |
Message
|
| harnischp |
 Posted: Fri Sep 21, 2007 3:22 am Post subject: Weblogic JMS extracted MQ correlation id mismatch |
 |
|
Newbie
Joined: 21 Sep 2007
Posts: 4
|
We have setup Weblogic 8.1 Foreign JMS (on Linux) to connect to Websphere MQ using filessytem (.bindings file). The Weblogic connects to a MQ server on UNIX which receives msgs from AS400 MQ. The MDB configured to process the MQ message is able to read the msg data correctly.
We also need to save the Correlation id of the message as a string. This correlation id is set by AS400 MQ when putting the message. On the Weblogic server, the correlation id is extracted using function getJMSCorrelationIDAsBytes() and converted to String. However the correlation id is different from the actual value - instead of "AMQ TNTLB60B <some spl chars>" the value extracted is "ÁÔØ@ãÕãÓÂöðÂ@@@@FìË". This looks like a encoding/ccsid mismatch problem. Note that when i used a MQAPI through java code, it read the value correctly, without setting and specific ccsid.
I also tried to simulate it by putting a message from UNIX MQ and that was read correctly.
AS400 settings:
data encoding (MDENC) is 273
coded character set id (MDCSI) is 0
message type (MDMT) is 8
CodedCharSetId for mq manager is 037
I changed the .bindings file to reflect the same ccsids at queue connection factory object and queue object level
QCF/RefAddr/6/Content=819 was changed to QCF/RefAddr/6/Content=037
and
Q/RefAddr/4/Content=1208 was changed to Q/RefAddr/4/Content=037
However this did not work. So I changed Q/RefAddr/4/Content=1208 was changed to Q/RefAddr/4/Content=0
This also did not work. The extracted correlation id is still "ÁÔØ@ãÕãÓÂöðÂ@@@@FìË" (the last byte changes with every new message)
I feel quite sure that something can be configured in the .bindings file to ensure it works correctly but i am at a loss now on what to change. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Sat Sep 22, 2007 4:38 am Post subject: Re: Weblogic JMS extracted MQ correlation id mismatch |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| harnischp wrote: |
We have setup Weblogic 8.1 Foreign JMS (on Linux) to connect to Websphere MQ using filessytem (.bindings file). The Weblogic connects to a MQ server on UNIX which receives msgs from AS400 MQ. The MDB configured to process the MQ message is able to read the msg data correctly.
We also need to save the Correlation id of the message as a string. This correlation id is set by AS400 MQ when putting the message. On the Weblogic server, the correlation id is extracted using function getJMSCorrelationIDAsBytes() and converted to String. However the correlation id is different from the actual value - instead of "AMQ TNTLB60B <some spl chars>" the value extracted is "ÁÔØ@ãÕãÓÂöðÂ@@@@FìË". This looks like a encoding/ccsid mismatch problem. Note that when i used a MQAPI through java code, it read the value correctly, without setting and specific ccsid. |
Use getJMSCorrelationID. DO NOT USE getJMSCorrelationIDAsBytes as they are not equivalent. If you use the latter you have to treat the result as a byte[] and CANNOT convert it into a String!. getJMSCorrelationID gives you a hex representation of the byte[]. This is far different from String(byte[])!.
To the risk of repeating myself the messageid and correlationid are byte arrays AND NOT Strings. Treating them as Strings will have dire consequences as you have obviously encountered...
| harnischp wrote: |
I also tried to simulate it by putting a message from UNIX MQ and that was read correctly.
AS400 settings:
data encoding (MDENC) is 273
coded character set id (MDCSI) is 0
message type (MDMT) is 8
CodedCharSetId for mq manager is 037
I changed the .bindings file to reflect the same ccsids at queue connection factory object and queue object level
QCF/RefAddr/6/Content=819 was changed to QCF/RefAddr/6/Content=037
and
Q/RefAddr/4/Content=1208 was changed to Q/RefAddr/4/Content=037
However this did not work. So I changed Q/RefAddr/4/Content=1208 was changed to Q/RefAddr/4/Content=0
This also did not work. The extracted correlation id is still "ÁÔØ@ãÕãÓÂöðÂ@@@@FìË" (the last byte changes with every new message)
I feel quite sure that something can be configured in the .bindings file to ensure it works correctly but i am at a loss now on what to change. |
Again this is of your own doing! If you want to treat the correlid as a String at least use the hex representation of the byte array.
All Strings may be byte arrays but all byte arrays are not Strings!.
Learn from your mistakes! 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| harnischp |
| Posted: Tue Oct 02, 2007 4:05 am Post subject: Re: Weblogic JMS extracted MQ correlation id mismatch |
|
|
Newbie
Joined: 21 Sep 2007
Posts: 4
|
| fjb_saper wrote: |
Use getJMSCorrelationID. DO NOT USE getJMSCorrelationIDAsBytes as they are not equivalent. If you use the latter you have to treat the result as a byte[] and CANNOT convert it into a String!. getJMSCorrelationID gives you a hex representation of the byte[]. This is far different from String(byte[])!.
To the risk of repeating myself the messageid and correlationid are byte arrays AND NOT Strings. Treating them as Strings will have dire consequences as you have obviously encountered...
Again this is of your own doing! If you want to treat the correlid as a String at least use the hex representation of the byte array.
All Strings may be byte arrays but all byte arrays are not Strings!.
Learn from your mistakes! |
I did the following steps:
1. I used java client program (MyMQPut.class) to put a message on the Unix MQ server using the following code to make MQ set the correlation id
MQPutMessageOptions pmo = new MQPutMessageOptions();
pmo.options |= MQC.MQPMO_NEW_CORREL_ID;
myQueue.put(myMessage,pmo);
I then use the following code in another java stub program (MyMQGet.class) to retrieve the correlation id
dumpAsStr(myMessage.correlationId);
static void dumpAsStr(byte[] myId) {
System.out.print("CHARX'");
for (int i = 0; i < myId.length; i++) {
char b = (char) (myId[i] & 0xFF);
if (b < 0x10) {
System.out.print("0");
}
System.out.print(b);
}
System.out.println("'");
}
And i get the following output
AMQ SYSTQM3 F╪►?!╫94
2. Next i put the message in the same manner on the Unix MQ server but now i read it using Weblogic JMS & an Mdb.
The code in the MDB to get the correlationid is
dumpAsStr(message.getJMSCorrelationIDAsBytes());
And the code to print it is the same as above and the output is
AMQ SYSTQM3 FØ!×¼c
and in hex X'414D512053595354514D33202020202046D8108F21D7BC63'
3. However now if i put the message from the AS400 MQ program and read it using Weblogic JMS & the same Mdb as above, the output is
ÁÔØ@ãÕãÓÂöðÂ@@@@FÿEÇ s¿
in hex ID:c1d4d840e3d5e3d3c2f6f0c24040404046ff45c72073bf04
4. Now i put the message from the AS400 MQ program, but read it using the java client of step 1, the output is
AMQ TNTLB60B ãT¥ÿ¬Þ¬
Now i am trying to identify if the problem is in Weblogic JMS or the JMS-MQ API or because of the AS400.
I hope this clears up exactly what i am doing. I dont think the issue is with Byte Array and String. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Oct 02, 2007 2:14 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
The issue is totally with byte array and string.
While the byte array does start with some string the CCSID the string is in will vastly vary depending on the CCSID of the source qmgr.
As you have noticed the byte array starts with
'AMQ " + QMGR NAME + hex values (not string anymore)
Of course the String values have different byte values in a different CCSID (ex space = x20 in ASCII but x40 in EBCDIC).
As the correlationId/messageId fields are really byte[] and are treated as such there is NEVER any CCSID conversion being made EVER.
So why in the would you want to be able to interpret them the same?
Check the hex value on the source qmgr system and check the hex value on the destination qmgr system and you will see that they are the same on the same message.
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| harnischp |
| Posted: Wed Oct 03, 2007 2:13 am Post subject: Re: Weblogic JMS extracted MQ correlation id mismatch |
|
|
Newbie
Joined: 21 Sep 2007
Posts: 4
|
| fjb_saper wrote: |
The issue is totally with byte array and string.
While the byte array does start with some string the CCSID the string is in will vastly vary depending on the CCSID of the source qmgr.
As you have noticed the byte array starts with
'AMQ " + QMGR NAME + hex values (not string anymore)
Of course the String values have different byte values in a different CCSID (ex space = x20 in ASCII but x40 in EBCDIC).
As the correlationId/messageId fields are really byte[] and are treated as such there is NEVER any CCSID conversion being made EVER.
So why in the would you want to be able to interpret them the same?
Check the hex value on the source qmgr system and check the hex value on the destination qmgr system and you will see that they are the same on the same message.
Enjoy |
Looks like its going to be more complicated than i thought. I have been focussing on the translation of the initial part of the string only, which i have not been able to do yet. What you have just pointed out means that it will not be as simple as that, my solution would need to consider the bytes after the string as well.
What i am doing is extracting the correlation id from the incoming message and then adding it as part of the data in the response message. The data is a fixed length string in which the correlation id field in the message is 24 chars. Will this allow teh AS400 to get the same correlation id that it populated initially?
Even if i extract the correlation id in Hex, i would need to translate it to the 24byte value before adding it to the record.
I have not been able to extract the 'AMQ ' + QMGR NAME string in Ascii format yet. I was trying to achive this as a first step.  |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Oct 03, 2007 2:17 am Post subject: Re: Weblogic JMS extracted MQ correlation id mismatch |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| harnischp wrote: |
| I have not been able to extract the 'AMQ ' + QMGR NAME string in Ascii format yet. I was trying to achive this as a first step. |
Perhaps slightly off topic here, but why? Any of this? The id is simply an internal identifier and is only useful in it's original form (as fjb_saper correctly rants) as a byte array to identify the message?
Why extract the queue manager name from it? Bearing in mind that's just a default and can be reworked.
There are easier ways of finding the originating queue manager's name!
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Oct 03, 2007 2:32 am Post subject: Re: Weblogic JMS extracted MQ correlation id mismatch |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| harnischp wrote: |
What i am doing is extracting the correlation id from the incoming message and then adding it as part of the data in the response message. The data is a fixed length string in which the correlation id field in the message is 24 chars. Will this allow the AS400 to get the same correlation id that it populated initially?
Even if i extract the correlation id in Hex, i would need to translate it to the 24byte value before adding it to the record.
I have not been able to extract the 'AMQ ' + QMGR NAME string in Ascii format yet. I was trying to achive this as a first step. |
You are barreling down the wrong track at full speed here.
The correlationId is a field on the MQMD. Why add it to the payload?
Let the AS400 extract it from the MQMD of the incoming message and put it on the MQMD of the response message...
If you absolutely need to add it to the payload of a message to preserve a service integrity and need it as string use 48 char => to allow for a hex representation of the byte[24]. (1 byte = from 00 to FF). If you then recreate the byte[] from the hex string (easy to do in Java or any other language) you should recreate exactly the same byte[] as the original.
In JMS the methods would allow you to set the correlationId using the hex representation in form 'ID:xx" where xx stands for the 48 char hex representation of the byte[].
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 03, 2007 3:36 am Post subject: Re: Weblogic JMS extracted MQ correlation id mismatch |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| harnischp wrote: |
| I have not been able to extract the 'AMQ ' + QMGR NAME string in Ascii format yet. I was trying to achive this as a first step. |
And fjb-saper told you in the first place that it would never be an ASCII string.
But what are you trying to FINALLY achieve, that this is supposed to be a "first step"?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| harnischp |
| Posted: Wed Oct 03, 2007 5:13 am Post subject: Re: Weblogic JMS extracted MQ correlation id mismatch |
|
|
Newbie
Joined: 21 Sep 2007
Posts: 4
|
This is what needs to be done:
The AS400 sends a message the Unix MQ server. The correlation id is populated by the MQ server. After processing, if there is an error, a response needs to be sent. The response message payload is a fixed length record. One field in the record is the correlation id sent by the AS400. My application needs to extract it from the incoming msg, save it in a table, and then populate it in the response if it is created. The size of the field in the record is 24 bytes.
When we developed the application, we tested with 2 unix servers and it worked correctly. But the correlation id was being populated by the application rather than MQ. Now we are facing this problem in the test environment of the client where the communication is between AS400 and UNIX mq servers.
I know that the way the correlation id is being used is not correct, but there are some existing programs on the AS400 that require it in this manner.
Note that I am not trying to extract the qmanager name from the correlation id. I was just using that to judge my progress in extracting the correlation id with correct encoding, since it was readable. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 03, 2007 6:22 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
So.. the AS400 sends a request to the Unix machine. This request includes a correlation ID field that is created by MQ.
The Unix machine is supposed to copy the value of this correlID into the response message in the Body.
Is the AS400 going to read the body field as string data, or bytes?
I'm guessing it will be strings.
This means that you need to convert it into ascii characters that, when converted BACK to characters in the CCSID of the AS400 will have the same byte value.
So essentially you need to go Bytes->EBCDIC (or whatever CCSID the AS400 is)->ASCII on the Unix side. Then the channel or the AS400 app will go ASCII->EBCDIC.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Oct 03, 2007 7:49 pm Post subject: Re: Weblogic JMS extracted MQ correlation id mismatch |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| harnischp wrote: |
This is what needs to be done:
The AS400 sends a message the Unix MQ server. The correlation id is populated by the MQ server. After processing, if there is an error, a response needs to be sent. The response message payload is a fixed length record. One field in the record is the correlation id sent by the AS400. My application needs to extract it from the incoming msg, save it in a table, and then populate it in the response if it is created. The size of the field in the record is 24 bytes.
When we developed the application, we tested with 2 unix servers and it worked correctly. But the correlation id was being populated by the application rather than MQ. Now we are facing this problem in the test environment of the client where the communication is between AS400 and UNIX mq servers.
I know that the way the correlation id is being used is not correct, but there are some existing programs on the AS400 that require it in this manner.
Note that I am not trying to extract the qmanager name from the correlation id. I was just using that to judge my progress in extracting the correlation id with correct encoding, since it was readable. |
This is so convoluted and complicated I would mandate for the applications (both of them AIX and AS400) to move to the regular request response pattern. No translations and all fields on the MQMD. Yes a little bit more programming but what a gain in speed and efficiency. On top of it you get an industry wide supported pattern vs your own funky requirements... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
