| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Trouble in reading messages with RFH2 header MQRC 2011 |
« View previous topic :: View next topic » |
| Author |
Message
|
| viks |
 Posted: Wed Jun 12, 2013 3:33 am Post subject: Trouble in reading messages with RFH2 header MQRC 2011 |
 |
|
Newbie
Joined: 12 Jun 2013
Posts: 8
|
I'm having an issue while migrating our MQ environment from a solaris queue manager to a Linux one. We have a perl program which gets the message from a queue and modify it's RFH2 header ( topic ) and post it to an another queue. In this new Linux environment this programs does the same. But issue is while trying to access the message that been posted to the destination queue we are getting MQRC2011. But of course when I read the message using amqsbcg I can see couple of extra HEX characters been added ( C2 )..
here is how I'm now checking this issue.
1) Below program now creates a new message with RFH2 and posts to the Source Queue:
use MQSeries::Message::RFH2;
use MQSeries::QueueManager;
use Data::Dumper;
#
# Create an RFH2 message with default settings
#
my $msg1 = MQSeries::Message::RFH2->new('Encoding' => MQENC_NATIVE, 'Data' => 'message with mydata' );
#
# Get RFH2 data
#
my $qmgr_obj = MQSeries::QueueManager->new(QueueManager => 'QM.US.T.01');
my $oqueue = MQSeries::Queue->
new(QueueManager => $qmgr_obj,
Queue => 'Q.TEST.IN',
Mode => 'output');
$msg1->NameValueData('<psc><Command>Publish</Command><Topic>test/abcd</Topic></psc>');
$oqueue->Put(Message => $msg1);
2) Then I am running my existing program which reads this message and reconstruct the RFH2 as below.
# Convert the RFH object into its raw binary form
sub binary
{
my $rfh = shift;
# Check for empty RFH
return '' unless $rfh->exists;
# Create default header if none exists
unless ($rfh->{StrucId})
{
@$rfh{qw(StrucId Version StrucLength Encoding CodedCharSet Format Flags NameValueCCSID)} =
( 'RFH', 2, 36, MQENC_NATIVE, -2, MQFMT_NONE, 0, 1208);
}
my $N = $rfh->endian;
my $rfh_data = '';
for (@$rfh)
{
my $data = "$_";
$data =~ s/>\s+</></sg;
# Replace "<tag/>" with "<tag></tag>"
$data =~ s/<([a-zA-Z_:]+)\s*\/>/<$1><\/$1>/g;
# Space pad data to a quad byte boundary
$data .= ' ' x (4 - length($data) % 4);
my $length = length $data;
$rfh_data .= pack "${N}A$length", $length, $data;
}
my $length = length $rfh_data;
$rfh->{StrucLength} = $length + 36;
return pack "A4${N}4A8${N}2A$length",
@$rfh{qw(StrucId Version StrucLength Encoding CodedCharSet Format Flags NameValueCCSID)},
$rfh_data;
};
sub endian
{
my $rfh = shift;
# If no message descriptor assume native
return 'L' unless $rfh->{-Rfh}{msgDesc};
# Find the integer encoding (big or little endian)
my $encoding = $rfh->{-Rfh}{msgDesc}{Encoding} & MQENC_INTEGER_MASK;
$encoding == MQENC_INTEGER_NORMAL || $encoding == MQENC_INTEGER_REVERSED
or croak "Invalid integer encoding: $encoding";
# Integer template code for unpack depends on encoding
return ($encoding == MQENC_INTEGER_NORMAL) ? 'N' : 'V';
}
3) Before posting the message to the queue I did a Hex dump of the message which is matching with the format it should be, while a message object dump shows some different character such as ( \x{b8} )
4) After the message is been put while reading with local char set it gives 2011, and by amqsbcg I can see an extra Hex character C2, which is at the same positions as of \x{b8} form the message dump.
Any clue why this happening? Any issues with Encoding ? Your help is appreciated. |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Wed Jun 12, 2013 3:37 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Encoding only applies to numeric values, not to string values.
CodedCharSetId applies to string values.
what do you mean by "message object dump"? Is this output from something like Data::Dumper or etc? Or ?
Can you show the amqsbcg hex output? |
|
| Back to top |
|
|
| viks |
| Posted: Wed Jun 12, 2013 6:38 pm Post subject: |
|
|
Newbie
Joined: 12 Jun 2013
Posts: 8
|
Thanks jeff for your reply..
Yes, you aright I mean the Data::Dumper ,to be clear this is what I'm doing:
#step 1: Data:Dumper of msg before mqput
#step 2: do a Hex dump of msg before posting
| Code: |
| print $self->hexdump($msg); |
#step 3: mqput the msg
| Code: |
| MQPUT($self->{CONN}->{HCONN}, $self->{HOBJ}, \%desc, \%opts, $msg, $cc, $reason); |
#step 4: get the AMQSBCG hex output of the message after mqput
here the hexdump used in Step -2 is a sub to dump the msg in Hex format with int the script,and this is how the logic looks :
| Code: |
sub dump
{ my $self = shift;
my $data = shift;
my $sprint;
for ($data =~ /(.{1,16})/gs)
{ my $bin = join ' ', map { sprintf '%02X', ord } split '', $_;
s/[^\040-\177]/./gs;
$sprint .= sprintf "%-48s %s\n", $bin, $_;
print $_;
}
return $sprint;
} |
And the output of each steps are below:
Step 1:
$VAR1 = "RFH ^Bh\"^B3^C \x{b8}^D\@<psc><Command>Publish</Command><Topic>test/abcd</Topic></psc> message with mydata";
Step 2:
52 46 48 20 02 00 00 00 68 00 00 00 22 02 00 00 RFH ....h..."...
33 03 00 00 20 20 20 20 20 20 20 20 00 00 00 00 3... ....
B8 04 00 00 40 00 00 00 3C 70 73 63 3E 3C 43 6F ....@...<psc><Co
6D 6D 61 6E 64 3E 50 75 62 6C 69 73 68 3C 2F 43 mmand>Publish</C
6F 6D 6D 61 6E 64 3E 3C 54 6F 70 69 63 3E 74 65 ommand><Topic>te
73 74 2F 61 62 63 64 3C 2F 54 6F 70 69 63 3E 3C st/abcd</Topic><
2F 70 73 63 3E 20 20 20 6D 65 73 73 61 67 65 20 /psc> message
77 69 74 68 20 6D 79 64 61 74 61 with mydata
step 4:
AMQSBCG0 - starts here
**********************
MQOPEN - 'Q.TEST.OUT'
MQGET of message number 1
****Message descriptor****
StrucId : 'MD ' Version : 2
Report : 0 MsgType : 8
Expiry : -1 Feedback : 0
Encoding : 546 CodedCharSetId : 819
Format : 'MQHRF2 '
Priority : 0 Persistence : 1
MsgId : X'414D5120514D2E55532E542E30312020704FB35102530720'
CorrelId : X'000000000000000000000000000000000000000000000000'
BackoutCount : 0
ReplyToQ : ' '
ReplyToQMgr : 'QM.US.T.01 '
** Identity Context
UserIdentifier : 'mqmuser '
AccountingToken :
X'0533333437370000000000000000000000000000000000000000000000000006'
ApplIdentityData : ' '
** Origin Context
PutApplType : '6'
PutApplName : 'perl '
PutDate : '20130612' PutTime : '11150494'
ApplOriginData : ' '
GroupId : X'000000000000000000000000000000000000000000000000'
MsgSeqNumber : '1'
Offset : '0'
MsgFlags : '0'
OriginalLength : '-1'
**** Message ****
length - 124 bytes
00000000: 5246 4820 0200 0000 6800 0000 2202 0000 'RFH ....h..."...'
00000010: 3303 0000 2020 2020 2020 2020 0000 0000 '3... ....'
00000020: C2B8 0400 0040 0000 003C 7073 633E 3C43 '¸...@...<psc><C'
00000030: 6F6D 6D61 6E64 3E50 7562 6C69 7368 3C2F 'ommand>Publish</'
00000040: 436F 6D6D 616E 643E 3C54 6F70 6963 3E74 'Command><Topic>t'
00000050: 6573 742F 6162 6364 3C2F 546F 7069 633E 'est/abcd</Topic>'
00000060: 3C2F 7073 633E 2020 206D 6573 7361 6765 '</psc> message'
00000070: 2077 6974 6820 6D79 6461 7461 ' with mydata '
Though not really sure, It seems like the position of character \x{b8} from Data::Dumper(step 1) output and hex value C2 from the AMQSBCG output( line 3 from hex output),
And without that extra C2 hex the whole Hex representation should look valid. Point to note that when I try to take hex output before putting the message ( step 2) I cannot find the hex character C2,and which look valid
Thanks in advance 
-Viks |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Jun 13, 2013 2:42 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
... try setting the ccsid of the message itself to 1208? or 1200?
  |
|
| Back to top |
|
|
| viks |
| Posted: Thu Jun 13, 2013 3:53 am Post subject: |
|
|
Newbie
Joined: 12 Jun 2013
Posts: 8
|
Thanks Jeff,
Do you mean the MQMD CodedCharSetId ? or RFH2 CodedCharSetId ?
I tried for both MQCCSI_DEFAULT, MQCCSI_Q_MGR, MQCCSI_INHERIT, MQCCSI_EMBEDDED . None set that to 1028 ( stays as 819 ) . Fyi, Queue manager is on 819.
then I tried by setting it using a Java client
jMsg.setIntProperty("JMS_IBM_Character_Set", 1208), it changes only the RFH2 CodedCharSetId to 1208 and 2100 respectively.
But after running through the RFH2 build script while putting the message it gave me : ERROR(2142, MQRC_HEADER_ERROR): MQ header structure not valid.
Might be a bit elementary doubt, but Any other way to try ?
Thanks a lot again,
Viks |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Jun 13, 2013 4:08 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I meant the mqmd ccsid. This indicates the CCSID of the next header, in your case the RFH2.
The namevalue ccsid is already 1208, so assuming your strings are in 1208, then you're okay on that point.
It's possible that your strings are in 1200 or 1201 or otherwise something that puts a BOM at the front, and that's what you're seeing. But I didn't double-check the positioning of the C2 well enough to verify that it would be in the right spot to be a BOM. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Thu Jun 13, 2013 9:22 am Post subject: Re: Trouble in reading messages with RFH2 header MQRC 2011 |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| viks wrote: |
| We have a perl program which gets the message from a queue and modify it's RFH2 header ( topic ) and post it to an another queue ... when I read the message using amqsbcg I can see couple of extra HEX characters been added ( C2 ).. |
The problem seems to be that the perl program is corrupting the Format of the MQRFH2 header. The Format specifies which bytes belong to character fields (subject to CCSID), and which bytes belong to numeric fields (subject to Encoding). If the perl program is not preserving the Format, then changing any of the CCSIDs or Encoding won't help.
The MQRFH2 header is a mix of MQCHARn and MQLONG fields. The NameValueCCSID (1208) is an MQLONG field. For little-endian encoding, its byte sequence is (hex: B8 04 00 00). It looks like the perl program is treating it as MQCHAR4 instead, and converting it from ISO8859 to UTF-8 (hex: C2 B8 04 00 00). The perl program might be doing that for all the fields, but the NameValueCCSID is the only one that contains a non-ASCII byte value (B8), so it's the only one that would cause a problem. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Jun 13, 2013 9:41 am Post subject: Re: Trouble in reading messages with RFH2 header MQRC 2011 |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| rekarm01 wrote: |
| viks wrote: |
| We have a perl program which gets the message from a queue and modify it's RFH2 header ( topic ) and post it to an another queue ... when I read the message using amqsbcg I can see couple of extra HEX characters been added ( C2 ).. |
The problem seems to be that the perl program is corrupting the Format of the MQRFH2 header. The Format specifies which bytes belong to character fields (subject to CCSID), and which bytes belong to numeric fields (subject to Encoding). If the perl program is not preserving the Format, then changing any of the CCSIDs or Encoding won't help.
The MQRFH2 header is a mix of MQCHARn and MQLONG fields. The NameValueCCSID (1208) is an MQLONG field. For little-endian encoding, its byte sequence is (hex: B8 04 00 00). It looks like the perl program is treating it as MQCHAR4 instead, and converting it from ISO8859 to UTF-8 (hex: C2 B8 04 00 00). The perl program might be doing that for all the fields, but the NameValueCCSID is the only one that contains a non-ASCII byte value (B , so it's the only one that would cause a problem. |
which suggests you reload your perl MQSeries modules from CPAN to make sure you're at most recent and contact the maintainers. |
|
| Back to top |
|
|
| viks |
| Posted: Fri Jun 14, 2013 3:08 am Post subject: resolved |
|
|
Newbie
Joined: 12 Jun 2013
Posts: 8
|
Thanks rekarm01, Jeff,
Looks like I debugged the issue. , and was able to resolve it, though I'm not sure this is the exact fix or it just a workaround.
Just a substring did the trick.
Let me explain it in detail:
I'have the latest CPAN ( 1.34) installed and perl is 5.12 and it is on Linux. But the code which i'm trying to make it run is currently running on Solaris, with perl 5.6.1 and cpan lib 1.29. And I'm not in a position to rewrite application now, somehow want the existing code to run with minimal changes.
Now we have a function to build the RFH2 folder structure and pad it to make it a valid header as below.
sub binary
{
my $rfh = shift;
# Create default header if none exists
unless ($rfh->{StrucId})
{
@$rfh{qw(StrucId Version StrucLength Encoding CodedCharSet Format Flags NameValueCCSID)} =
( 'RFH', 2, 36, MQENC_NATIVE, -2, MQFMT_NONE, 0, 1208);
}
my $N = $rfh->endian;
my $rfh_data = '';
for (@$rfh)
{
my $data = "$_";
$data =~ s/>\s+</></sg;
# Replace "<tag/>" with "<tag></tag>"
$data =~ s/<([a-zA-Z_:]+)\s*\/>/<$1><\/$1>/g;
# Space pad data to a quad byte boundary
$data .= ' ' x (4 - length($data) % 4);
my $length = length $data;
$rfh_data .= pack "${N}A$length", $length, $data;
}
here the $rfh is an hash where all the folder values are there. Adding this statement after the above BOLD Line did the trick to me. (
$data = substr($data,0);
So I assume that though the $data has the required header values, it somehow doesn't recognize as a simple string when it get converted.
Thanks for the help..
Viks |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Fri Jun 14, 2013 9:20 am Post subject: Re: resolved |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| viks wrote: |
for (@$rfh)
{
my $data = "$_";
$data =~ s/>\s+</></sg;
# Replace "<tag/>" with "<tag></tag>"
$data =~ s/<([a-zA-Z_:]+)\s*\/>/<$1><\/$1>/g;
# Space pad data to a quad byte boundary
$data .= ' ' x (4 - length($data) % 4);
my $length = length $data;
$rfh_data .= pack "${N}A$length", $length, $data;
}
|
First, it's not entirely clear why this for loop needs to iterate over all the RFH elements; wouldn't it be sufficient to just iterate over the NameValueData elements? (On an unrelated note, it doesn't seem that this subroutine would work for multiple NameValueData elements.)
| viks wrote: |
here the $rfh is an hash where all the folder values are there. Adding this statement after the above BOLD Line did the trick to me. (
$data = substr($data,0);
So I assume that though the $data has the required header values, it somehow doesn't recognize as a simple string when it get converted. |
It's also not clear why it would need to recognize NameValueCCSID as a simple string, since it's not a simple string, or how that would fix the problem. It's probably due to some non-obvious Perl quirk, but if it works, at least that's progress ... |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
