Author |
Message
|
Ramphart |
Posted: Thu Apr 07, 2005 4:27 am Post subject: MQMD, MQRFH2, Empty Msg to BLOB |
|
|
 Disciple
Joined: 21 Jul 2004 Posts: 150 Location: South Africa, JHB
|
Hi,
- WMQI 2.1 CSD 07 on Windows 2000.
- I'm try to log an incoming message to a database after parsing it as a BLOB and also log specific fields from the MQMD and the MQRFH2.
[1]Sub FLow that logs the data
InputTerminal => RCD (Message Domain=BLOB, Reset Msg Domain) => Trace (${Root}) => Database Node
[2]Input Message 1
- Non Empty Message
- MQRFH2.usr.Filename ='INV23.0003'
-Results when Parsing this Message
Code: |
(0x1000000)Properties = (
(0x3000000)MessageSet = 'E3BPREC09S001'
(0x3000000)MessageType = 'Message'
(0x3000000)MessageFormat = 'IN.STRK'
(0x3000000)Encoding = 546
(0x3000000)CodedCharSetId = 437
(0x3000000)Transactional = TRUE
(0x3000000)Persistence = FALSE
(0x3000000)CreationTime = GMTTIMESTAMP '2005-04-07 11:29:10.940'
(0x3000000)ExpirationTime = -1
(0x3000000)Priority = 0
(0x3000000)ReplyIdentifier = X'000000000000000000000000000000000000000000000000'
(0x3000000)ReplyProtocol = 'MQ'
(0x3000000)Topic = NULL
)
(0x1000000)MQMD = (
(0x3000000)SourceQueue = 'DIST.CLAYVILLE.IDS.INVENTORY.EXTRACT.QL'
(0x3000000)Transactional = TRUE
(0x3000000)Encoding = 546
(0x3000000)CodedCharSetId = 437
(0x3000000)Format = 'MQHRF2 '
(0x3000000)Version = 2
(0x3000000)Report = 0
(0x3000000)MsgType = 8
(0x3000000)Expiry = -1
(0x3000000)Feedback = 0
(0x3000000)Priority = 0
(0x3000000)Persistence = 0
(0x3000000)MsgId = X'414d5120435650573236302e574d514979fa4c4220009e05'
(0x3000000)CorrelId = X'000000000000000000000000000000000000000000000000'
(0x3000000)BackoutCount = 0
(0x3000000)ReplyToQ = ' '
(0x3000000)ReplyToQMgr = 'CVPW260.WMQI.QMGR '
(0x3000000)UserIdentifier = 'wmqi '
(0x3000000)AccountingToken = X'16010515000000235f636bee94020743170a32ea03000000000000000000000b'
(0x3000000)ApplIdentityData = ' '
(0x3000000)PutApplType = 11
(0x3000000)PutApplName = 's\ih03 - RFHUtil\rfhutil.exe'
(0x3000000)PutDate = DATE '2005-04-07'
(0x3000000)PutTime = GMTTIME '11:43:09.170'
(0x3000000)ApplOriginData = ' '
(0x3000000)GroupId = X'000000000000000000000000000000000000000000000000'
(0x3000000)MsgSeqNumber = 1
(0x3000000)Offset = 0
(0x3000000)MsgFlags = 0
(0x3000000)OriginalLength = -1
)
(0x1000000)MQRFH2 = (
(0x3000000)Version = 2
(0x3000000)Format = ' '
(0x3000000)Encoding = 546
(0x3000000)CodedCharSetId = 437
(0x3000000)Flags = 0
(0x3000000)NameValueCCSID = 1208
(0x1000000)usr = (
(0x1000000)Filename = (
(0x2000000) = 'INV23.0003'
)
)
(0x1000000)mcd = (
(0x1000000)Msd = (
(0x2000000) = 'none'
)
(0x1000000)Fmt = (
(0x2000000) = 'IN.STRK'
)
)
)
(0x1000000)BLOB = (
(0x3000000)BLOB = X'30323036303530323720202 etc etc
)
|
- The MQRFH2 is parsed properly and I have access to it once I enter the Database Node
- The BLOB.BLOB contains the actual data in the message
[2]Input Message 2
- This is an Empty message
- MQRFH2.usr.Filename ='INV23.0003'
-Results when Parsing the EMPTY Message
Code: |
(
(0x1000000)Properties = (
(0x3000000)MessageSet = 'E3BPREC09S001'
(0x3000000)MessageType = 'Message'
(0x3000000)MessageFormat = 'IN.STRK'
(0x3000000)Encoding = 546
(0x3000000)CodedCharSetId = 437
(0x3000000)Transactional = TRUE
(0x3000000)Persistence = FALSE
(0x3000000)CreationTime = GMTTIMESTAMP '2005-04-07 11:44:45.610'
(0x3000000)ExpirationTime = -1
(0x3000000)Priority = 0
(0x3000000)ReplyIdentifier = X'000000000000000000000000000000000000000000000000'
(0x3000000)ReplyProtocol = 'MQ'
(0x3000000)Topic = NULL
)
(0x1000000)MQMD = (
(0x3000000)SourceQueue = 'DIST.CLAYVILLE.IDS.INVENTORY.EXTRACT.QL'
(0x3000000)Transactional = TRUE
(0x3000000)Encoding = 546
(0x3000000)CodedCharSetId = 437
(0x3000000)Version = 2
(0x3000000)Report = 0
(0x3000000)MsgType = 8
(0x3000000)Expiry = -1
(0x3000000)Feedback = 0
(0x3000000)Priority = 0
(0x3000000)Persistence = 0
(0x3000000)MsgId = X'414d5120435650573236302e574d514979fa4c4220009e07'
(0x3000000)CorrelId = X'000000000000000000000000000000000000000000000000'
(0x3000000)BackoutCount = 0
(0x3000000)ReplyToQ = ' '
(0x3000000)ReplyToQMgr = 'CVPW260.WMQI.QMGR '
(0x3000000)UserIdentifier = 'wmqi '
(0x3000000)AccountingToken = X'16010515000000235f636bee94020743170a32ea03000000000000000000000b'
(0x3000000)ApplIdentityData = ' '
(0x3000000)PutApplType = 11
(0x3000000)PutApplName = 's\ih03 - RFHUtil\rfhutil.exe'
(0x3000000)PutDate = DATE '2005-04-07'
(0x3000000)PutTime = GMTTIME '11:44:45.610'
(0x3000000)ApplOriginData = ' '
(0x3000000)GroupId = X'000000000000000000000000000000000000000000000000'
(0x3000000)MsgSeqNumber = 1
(0x3000000)Offset = 0
(0x3000000)MsgFlags = 0
(0x3000000)OriginalLength = -1
)
(0x1000000)BLOB = (
(0x3000000)BLOB = X'5246482002000000540000 etc etc
)
)
|
- The MQRFH2 is NOT present after the RCD BLOB and I thus do not have access to it once I enter the Database Node
- The BLOB.BLOB contains the data that was in the MQRFH2 from the incoming message
Why does this happen and how can I fix it? Is it a bug? _________________ Applications Architect
Last edited by Ramphart on Thu Apr 07, 2005 4:35 am; edited 3 times in total |
|
Back to top |
|
 |
jefflowrey |
Posted: Thu Apr 07, 2005 4:31 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
Your second message does not have an MQMD.Format.
If your second message contains an MQRFH2 header, the MQMD.Format needs to specify that.
This is not a broker "problem", this is an MQ "problem". _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
Ramphart |
Posted: Thu Apr 07, 2005 5:24 am Post subject: |
|
|
 Disciple
Joined: 21 Jul 2004 Posts: 150 Location: South Africa, JHB
|
jefflowrey wrote: |
Your second message does not have an MQMD.Format.
If your second message contains an MQRFH2 header, the MQMD.Format needs to specify that.
This is not a broker "problem", this is an MQ "problem". |
Thanks Jeff. Do you know why MQ do this or what I can do to work around this from a Broker perspective ?
The other thing worth mentioning is that if I put the EMPTY message (with an RFH2) onto a queue and I read it off the Queue with say the RFHutil (support pack ih03) then the RFH2 header is present and the MQMD.Format=MQHRF2.
The MQMD.Format=' ' only when I use a flow (via the broker) to read and RCD the EMPTY message. Could this thus be a bug on the broker side? _________________ Applications Architect
Last edited by Ramphart on Thu Apr 07, 2005 9:27 am; edited 1 time in total |
|
Back to top |
|
 |
jefflowrey |
Posted: Thu Apr 07, 2005 5:33 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
Can you post an amqsbcg output of the message before the broker sees it?
I strongly suspect that either the MQMD is wrong or the MQRFH2 is wrong, and RFHUtil is friendlier than Broker. _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
Ramphart |
Posted: Thu Apr 07, 2005 5:58 am Post subject: |
|
|
 Disciple
Joined: 21 Jul 2004 Posts: 150 Location: South Africa, JHB
|
amqsbcg output - Empty Message
Code: |
AMQSBCG0 - starts here
**********************
MQOPEN - 'AA.OUT'
MQGET of message number 1
****Message descriptor****
StrucId : 'MD ' Version : 2
Report : 0 MsgType : 8
Expiry : -1 Feedback : 0
Encoding : 546 CodedCharSetId : 437
Format : 'MQHRF2 '
Priority : 0 Persistence : 0
MsgId : X'414D5120435650573236302E574D514979FA4C4220009E0F'
CorrelId : X'000000000000000000000000000000000000000000000000'
BackoutCount : 0
ReplyToQ : ' '
ReplyToQMgr : 'CVPW260.WMQI.QMGR '
** Identity Context
UserIdentifier : 'wmqi '
AccountingToken :
X'16010515000000235F636BEE94020743170A32EA03000000000000000000000B'
ApplIdentityData : ' '
** Origin Context
PutApplType : '11'
PutApplName : 's\ih03 - RFHUtil\rfhutil.exe'
PutDate : '20050407' PutTime : '13544343'
ApplOriginData : ' '
GroupId : X'000000000000000000000000000000000000000000000000'
MsgSeqNumber : '1'
Offset : '0'
MsgFlags : '0'
OriginalLength : '-1'
**** Message ****
length - 84 bytes
00000000: 5246 4820 0200 0000 5400 0000 2202 0000 'RFH ....T..."...'
00000010: B501 0000 2020 2020 2020 2020 0000 0000 'µ... ....'
00000020: B804 0000 2C00 0000 3C75 7372 3E3C 4669 '¸...,...<usr><Fi'
00000030: 6C65 6E61 6D65 3E49 4E56 3233 2E30 3030 'lename>INV23.000'
00000040: 333C 2F46 696C 656E 616D 653E 3C2F 7573 '3</Filename></us'
00000050: 723E 2020 'r> '
No more messages
MQCLOSE
MQDISC |
amqsbcg output - NON Empty Message
Code: |
AMQSBCG0 - starts here
**********************
MQOPEN - 'AA.OUT'
MQGET of message number 1
****Message descriptor****
StrucId : 'MD ' Version : 2
Report : 0 MsgType : 8
Expiry : -1 Feedback : 0
Encoding : 546 CodedCharSetId : 437
Format : 'MQHRF2 '
Priority : 0 Persistence : 0
MsgId : X'414D5120435650573236302E574D514979FA4C4220009E0D'
CorrelId : X'000000000000000000000000000000000000000000000000'
BackoutCount : 0
ReplyToQ : ' '
ReplyToQMgr : 'CVPW260.WMQI.QMGR '
** Identity Context
UserIdentifier : 'wmqi '
AccountingToken :
X'16010515000000235F636BEE94020743170A32EA03000000000000000000000B'
ApplIdentityData : ' '
** Origin Context
PutApplType : '11'
PutApplName : 's\ih03 - RFHUtil\rfhutil.exe'
PutDate : '20050407' PutTime : '13535552'
ApplOriginData : ' '
GroupId : X'000000000000000000000000000000000000000000000000'
MsgSeqNumber : '1'
Offset : '0'
MsgFlags : '0'
OriginalLength : '-1'
**** Message ****
length - 302 bytes
00000000: 5246 4820 0200 0000 5400 0000 2202 0000 'RFH ....T..."...'
00000010: B501 0000 2020 2020 2020 2020 0000 0000 'µ... ....'
00000020: B804 0000 2C00 0000 3C75 7372 3E3C 4669 '¸...,...<usr><Fi'
00000030: 6C65 6E61 6D65 3E49 4E56 3233 2E30 3030 'lename>INV23.000'
00000040: 333C 2F46 696C 656E 616D 653E 3C2F 7573 '3</Filename></us'
00000050: 723E 2020 3032 3036 3035 3032 3720 2020 'r> 020605027 '
00000060: 2020 2020 2030 3030 3030 3030 3030 3030 ' 00000000000'
00000070: 3030 3030 3030 3030 3030 3030 3030 3030 '0000000000000000'
00000080: 3030 3030 3030 3030 3030 3030 3030 3030 '0000000000000000'
00000090: 3030 3030 3030 3030 3030 3030 3030 3030 '0000000000000000'
000000A0: 3030 3030 3030 3030 3030 3030 3030 3043 '000000000000000C'
000000B0: 5332 3030 3330 3433 3031 3535 3932 340D 'S20030430155924.'
000000C0: 0A30 3230 3630 3530 3239 2020 2020 2020 '.020605029 '
000000D0: 2020 3030 3030 3030 3030 3030 3030 3030 ' 00000000000000'
000000E0: 3030 3030 3030 3030 3030 3030 3030 3030 '0000000000000000'
000000F0: 3030 3030 3030 3030 3030 3030 3030 3030 '0000000000000000'
00000100: 3030 3030 3030 3030 3030 3030 3030 3030 '0000000000000000'
00000110: 3030 3030 3030 3030 3030 3030 4353 3230 '000000000000CS20'
00000120: 3033 3034 3330 3135 3539 3234 0D0A '030430155924.. '
No more messages
MQCLOSE
MQDISC |
_________________ Applications Architect |
|
Back to top |
|
 |
Ramphart |
Posted: Fri Apr 08, 2005 1:15 am Post subject: |
|
|
 Disciple
Joined: 21 Jul 2004 Posts: 150 Location: South Africa, JHB
|
Hi,
I've found the issue and need some help please. The broker reads the message properly from the Queue and all headers are present even if the message is empty. The problem is with the RCD BLOB node.
My Subflow:
InputTerminal => RCD (Message Domain=BLOB, Reset Msg Domain) => Trace (${Root}) => Database Node
If I trace the EMPTY message before the RCD node then the RFH2 exist. After the RCD to BLOB the Root.BLOB.BLOB contains the RFH2 data and the RFH2 header does not exist.
For a non-Empty message the RFH2 always exist. Before and after the RCD node and the Root.BLOB.BLOB contains the actual message data.
I need to do the RCD to BLOB though since I have a generic message logger and the incoming message can be in any domain (MRM,XML, BLOB, etc).
Any ideas on why the RCD to BLOB on and EMPTY message do this? _________________ Applications Architect |
|
Back to top |
|
 |
JT |
Posted: Fri Apr 08, 2005 6:39 am Post subject: |
|
|
Padawan
Joined: 27 Mar 2003 Posts: 1564 Location: Hartford, CT.
|
I'm sure I'm missing something here, but why use the RCD node at all.
Couldn't you simply CAST the InputBody, regardless of domain type and when the message data is present, to a BLOB variable within the Database node. |
|
Back to top |
|
 |
jefflowrey |
Posted: Fri Apr 08, 2005 7:05 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
Okay, I think I know what's going on.
The RCD assumes that the last child of Root is going to be the Body.
If the input message is empty, then the RFH header is the last child of Root.
So, if you check to see if InputRoot.BLOB is NULL, and set the VALUE of OutputRoot.BLOB.BLOB to NULL if so, then you should be fine for your RCD. _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
Ramphart |
Posted: Mon Apr 11, 2005 3:48 am Post subject: |
|
|
 Disciple
Joined: 21 Jul 2004 Posts: 150 Location: South Africa, JHB
|
jefflowrey wrote: |
So, if you check to see if InputRoot.BLOB is NULL, and set the VALUE of OutputRoot.BLOB.BLOB to NULL if so, then you should be fine for your RCD. |
The problem is that before the RCD node the message can be in any domain [XML, MRM,. BLOB, etc] and since it a generic sub-flow logger I cannot just check for the InputRoot.BLOB.BLOB. After the BLOB I run into the issue that an EMPTY message will end as follows:
Root.BLOB.BLOB = Original Message MQMD if no RFH2 is present
Root.BLOB.BLOB = RFH2 if a RFH2 is present
I've gone this route for now to solve this issue:
SubFlow
----------
InputTerminal => Database Node (I've eliminated the RCD all together)
Database Fields
------------------
MsgID is a CHAR, RFH2USRFILENAME is a CHAR, MSGPAYLOAD is a CLOB
Code: |
DECLARE dataBLOB BLOB;
IF (Body.Encoding is NULL) AND (Body.CodedCharSetId IS NULL) THEN
SET dataBLOB = CAST(BITSTREAM(Body) AS BLOB CCSID Root.MQMD.CodedCharSetId ENCODING Root.MQMD.Encoding);
END IF;
INSERT INTO Database.NULLID.MESSAGELOG
(MsgID, RFH2USRFILENAME, MSGPAYLOAD)
VALUES (SUBSTRING(Cast(Root.MQMD.MsgId AS CHAR) FROM 3 FOR 48),
CAST(COALESCE(Root.MQRFH2.usr.Filename, 'N/A') AS CHAR),
dataBLOB); |
I've chosen two fields (Encoding and CodedCharSetId) that will always be present in both the MQMD and the RFH2. My logic is thus that if none of these fields are present in the Body (InputBody) then I know a non-empty message was read from the queue.
I'm open to any other suggestions. _________________ Applications Architect |
|
Back to top |
|
 |
jefflowrey |
Posted: Mon Apr 11, 2005 4:09 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
Actually, if you know that there will always be an RFH2, then you can check to see if the last child of Root is named 'MQRFH2'. If it is, then you know there isn't a Body.
You could also check to see if it's named 'MQMD' - then you know there isn't an MQRFH2 either.
Or you can use the "same node" operation.. or is that only in the Java API? _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
Ramphart |
Posted: Mon Apr 11, 2005 6:58 am Post subject: |
|
|
 Disciple
Joined: 21 Jul 2004 Posts: 150 Location: South Africa, JHB
|
I'm not aware of a "same node" operation in WMQI 2.1. Further more, messages does not always come with a RFH2 since we getting it from source systems all over the show and hence my predicament. For now I'll have to stick to the solution above to achieve the generic message logging. _________________ Applications Architect |
|
Back to top |
|
 |
JT |
Posted: Mon Apr 11, 2005 7:58 am Post subject: |
|
|
Padawan
Joined: 27 Mar 2003 Posts: 1564 Location: Hartford, CT.
|
You could also check for a valid message domain.....
Code: |
IF POSITION(FIELDNAME(Root.*[<]) IN 'XML,XMLNS,BLOB,MRM,JMSStream,JMSMap') > 0 THEN
SET dataBLOB.......
END IF; |
|
|
Back to top |
|
 |
shra_k |
Posted: Fri Apr 15, 2005 3:24 am Post subject: |
|
|
Apprentice
Joined: 30 Dec 2002 Posts: 37
|
|
Back to top |
|
 |
jefflowrey |
Posted: Fri Apr 15, 2005 4:14 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
Is that an internal IBM site? Or just a network/proxy error on my side? _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
mqmatt |
Posted: Fri Apr 15, 2005 4:49 am Post subject: |
|
|
 Grand Master
Joined: 04 Aug 2004 Posts: 1213 Location: Hursley, UK
|
Quote: |
When an MQSeries message is parsed into the message tree, then a folder is created for the Properties, each of the headers and the final folder is for the message body
(ie BLOB, XML, MRM etc). When the MQSeries message has no user message body, then no message body folder is created. Therefore the current problem occurs.
Unfortunately this problem is NOT unique to the RCD node alone. the broker assumes that the last folder is the Body (which is assumed to exist ALWAYS) and no efforts
are made to ensure the same. Therefore, this exposes the following unexpected behaviour when an empty body message is used:
1) In a compute node, you are allowed to use "Copy All Headers" option using the ESQL. The "Copy All Headers" facility expects the message body folder to be the
last folder, and copies N-1 folders to the output message. Obviously when no user message body exists, then copying N-1 folders results in the last header not being
copied. When a message only contains a single header - the MQMD, then only the properties folder is copied, thus resulting in a completely empty MQSeries message.
2) The other case is the use of the correlation name Body. The Body (or InputBody) correlation name actually refers to the last header instead of the message body.
This is meant to represent whatever body folder is at the end of the message tree. But in fact, it represents InputRoot.[<] (the last folder in the tree). So if there
is no body folder, then Body actually refers to last header in the message tree.
3) The ResetContentDescriptor node does not copy the last header and will serialise the last header as the message body and pass it to the new owning parser. In this case
the last header MQMD is serialised as the message body.
In all these cases the product is actually working as per the current design, although it is not an expected behaviour.
We would like to provide the following workarounds to overcome this problem.
1) Replace the RCD node with a compute node which has ESQL that uses the ASBITSTREAM to serialise the current bitstream and then issue CREATE with PARSE to assign
the new owning parser to the bitstream. You should enclose this in an IF statement that checks if the last folder was a header or not. For example,
if customer knows that they only get XML or MRM messages in their flow, then they could write the ESQL to check the name of the last folder. For example :
CALL CopyMessageHeaders();
DECLARE lastFolderName CHAR FIELDNAME( InputBody)
IF lastFolderName = 'XML' OR lastFolderName = 'MRM' THEN
DECLARE myMsg ASBITSTREAM(InputBody, InputRoot.MQMD.Encoding,InputRoot.MQMD.CodedCharSetId);
CREATE LASTCHILD OF DOMAIN(theNewDomain) OutputRoot PARSE(myMsg, InputRoot.MQMD.Encoding, InputRoot.MQMD.CodedCharSetId);
END IF;
and obviously adding three parms on the PARSE clause, if the MRM or New Era is involved.
2) Put a Filter node before the RCD node that checks if a message body folder exists. This would use ESQL as above, such as :
DECLARE lastFolderName CHAR FIELDNAME( Body)
IF lastFolderName = 'XML' OR lastFolderName = 'MRM' THEN
RETURN TRUE;
ELSE
RETURN FALSE;
END IF;
|
|
|
Back to top |
|
 |
|