| Author |
Message
|
| tbt102 |
 Posted: Sun Jul 27, 2003 4:04 pm Post subject: Reset Content Descriptor not converting data properly |
 |
|
Apprentice
Joined: 21 Apr 2003
Posts: 28
|
Hi All,
I have a RCD that is not converting a BOLB to char properly.
1. there is a message FTPed from the OS/390 in a binary format to an AIX box.
2. the message is put on the flows input queue. The message domain is BLOB
3. the message is sent to a compute node that adds some additional data to the message
4. next the message is sent to the RCD to change it to the message I'm interested in
5. step 4 is sending invalid data to the next compute node for processing. The converted data looks like this "±⌡⌠≥±±≤â‰"
As a test I removed the RCD and tried converting the blob via the following and got the same results:
| Code: |
DECLARE Fld1 CHAR;
DECLARE Fld2 CHAR;
DECLARE blobInputMessage BLOB;
SET blobInputMessage = InputRoot."BLOB"."BLOB";
SET Fld1 = CAST(SUBSTRING(blobInputMessage FROM 1 FOR 11) AS CHARACTER CCSID 819 ENCODING 546);
SET Fld2 = CAST(SUBSTRING(blobInputMessage FROM 12 FOR 40) AS CHARACTER CCSID 819 ENCODING 546); |
Does anyone have an idea why the RCD is not converting?
Thanks in advance for any help. |
|
| Back to top |
|
 |
| kirani |
| Posted: Sun Jul 27, 2003 5:57 pm Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
Let's first try to isolate the problem by removing all compute nodes in the flow to make sure you are receiving the expected data. To do this create/modify your flow as follows,
1. Read your input message as BLOB in MQInput node.
2. In a compute node write similar code that you have posted with small modifications.
| Code: |
SET Environment.Variables.InputMessage = CAST(InputRoot."BLOB"."BLOB" AS CHAR CCSID InputRoot.MQMD.CodedCharSetId ENCODING InputRoot.MQMD.Encoding;
|
3. Attach a Trace node to the output of this compute node and print ${Environment} into it.
Now, verify your output in a trace file.
_________________
Kiran
IBM Cert. Solution Designer & System Administrator - WBIMB V5
IBM Cert. Solutions Expert - WMQI
IBM Cert. Specialist - WMQI, MQSeries
IBM Cert. Developer - MQSeries
|
|
| Back to top |
|
|
| tbt102 |
| Posted: Mon Jul 28, 2003 3:04 am Post subject: |
|
|
Apprentice
Joined: 21 Apr 2003
Posts: 28
|
kirani,
here is the code in the compute node:
| Quote: |
DECLARE Fld1 CHAR;
DECLARE Fld2 CHAR;
DECLARE blobInputMessage BLOB;
SET blobInputMessage = InputRoot."BLOB"."BLOB";
SET Fld1 = CAST(SUBSTRING(blobInputMessage FROM 1 FOR 11) AS CHARACTER CCSID 819 ENCODING 546);
SET Fld2 = CAST(SUBSTRING(blobInputMessage FROM 12 FOR 40) AS CHARACTER CCSID 819 ENCODING 546);
SET Environment.Variables.InputMessage =
CAST(InputRoot."BLOB"."BLOB" AS CHAR CCSID InputRoot.MQMD.CodedCharSetId ENCODING InputRoot.MQMD.Encoding); |
here is a portion of the trace node results:
| Code: |
Executing statement 'SET Fld1 = CAST(SUBSTRING(blobInputMessage FROM 1 FOR 11) AS CHARACTER . . . .
Evaluating expression 'CAST(SUBSTRING(blobInputMessage FROM 1 FOR 11) AS CHARACTER CCSID 819. . . .
Evaluating expression 'SUBSTRING(blobInputMessage FROM 1 FOR 11)' at (15, 17).
Evaluating expression 'blobInputMessage' at (15, 27).
Finished evaluating expression 'SUBSTRING(blobInputMessage FROM 1 FOR 11)' at (15, 17). This resolved to 'SUBSTRING(X'f1f5f4f2f1f1f3f0f2f3f0f1f0f1f1f0f1f0d4f0f8f0d4f1f8f5e7e7d9d7f0f4f4f7f1f0f0f3f5c2c5f0f0f1f0f2c5f1f4f0f0' FROM 1 FOR 11)'. The result was 'X'f1f5f4f2f1f1f3f0f2f3f0''.
Finished evaluating expression 'CAST(SUBSTRING(blobInputMessage FROM 1 FOR 11) AS CHARACTER CCSID 819 ENCODING 546)' at (15, 12). This resolved to 'CAST(X'f1f5f4f2f1f1f3f0f2f3f0' AS CHARACTER CCSID 819 ENCODING 546 )'. The result was ''ñõôòññóðòóð''.
Executing statement 'SET Fld2 = CAST(SUBSTRING(blobInputMessage FROM 12 FOR 40) AS CHARACTER . . . .
Evaluating expression 'CAST(SUBSTRING(blobInputMessage FROM 12 FOR 40) AS CHARACTER CCSID 819. . . .
Evaluating expression 'SUBSTRING(blobInputMessage FROM 12 FOR 40)' at (16, 17).
Evaluating expression 'blobInputMessage' at (16, 27).
Finished evaluating expression 'SUBSTRING(blobInputMessage FROM 12 FOR 40)' at (16, 17). This resolved to 'SUBSTRING(X'f1f5f4f2f1f1f3f0f2f3f0f1f0f1f1f0f1f0d4f0f8f0d4f1f8f5e7e7d9d7f0f4f4f7f1f0f0f3f5c2c5f0f0f1f0f2c5f1f4f0f0' FROM 12 FOR 40)'. The result was 'X'f1f0f1f1f0f1f0d4f0f8f0d4f1f8f5e7e7d9d7f0f4f4f7f1f0f0f3f5c2c5f0f0f1f0f2c5f1f4f0f0''.
Finished evaluating expression 'CAST(SUBSTRING(blobInputMessage FROM 12 FOR 40) AS CHARACTER CCSID 819 ENCODING 546)' at (16, 12). This resolved to 'CAST(X'f1f0f1f1f0f1f0d4f0f8f0d4f1f8f5e7e7d9d7f0f4f4f7f1f0f0f3f5c2c5f0f0f1f0f2c5f1f4f0f0' AS CHARACTER CCSID 819 ENCODING 546 )'. The result was ''ñðññðñðÔðøðÔñøõççÙ×ðôô÷ñððóõÂÅððñðòÅñôðð''.
Executing statement 'SET Environment.Variables.InputMessage = CAST(InputRoot.BLOB.BLOB AS CHARACTER. . . .
Evaluating expression 'CAST(InputRoot.BLOB.BLOB AS CHARACTER CCSID InputRoot.MQMD.CodedCharSetId. . . .
Evaluating expression 'InputRoot.BLOB.BLOB' at (19, 7).
Evaluating expression 'InputRoot.MQMD.CodedCharSetId' at (19, 45).
Evaluating expression 'InputRoot.MQMD.Encoding' at (19, 84).
Finished evaluating expression 'CAST(InputRoot.BLOB.BLOB AS CHARACTER CCSID InputRoot.MQMD.CodedCharSetId ENCODING InputRoot.MQMD.Encoding)' at (19, 2). This resolved to 'CAST(X'f1f5f4f2f1f1f3f0f2f3f0f1f0f1f1f0f1f0d4f0f8f0d4f1f8f5e7e7d9d7f0f4f4f7f1f0f0f3f5c2c5f0f0f1f0f2c5f1f4f0f0' AS CHARACTER CCSID 437 ENCODING 546 )'. The result was ''±±±±±±±°±°±±±''.
Assigning value ''±±±±±±±°±°±±±'' to field / variable 'Environment.Variables.InputMessage'.
Message propagated to 'out' terminal of compute node 'PWF.Compute1'.
Node 'PWF.Trace2': Evaluating expression 'Environment' at (1, 3).
Data '(
(0x1000000)Variables = (
(0x3000000)InputMessage = '±±±±±±±°±°±±±'
)
)
' from trace node 'PWF.Trace2'. |
It looks like the data is coming in fine and the sub string is working. The cast/conversion seems to be going wrong. Also, since the ESQL and the RCD are both producing the same kind of results, I think the problem is not with the flow but something else. I think. |
|
| Back to top |
|
|
| tbt102 |
| Posted: Mon Jul 28, 2003 11:10 am Post subject: |
|
|
Apprentice
Joined: 21 Apr 2003
Posts: 28
|
Hi kirani,
I have code a couple of flows similar to this one. As a test I did the following:
1) Put the message for the flow I'm working on in the MQInput of an old flow. The conversion failed
2) Put the message for the old flow on in the MQInput of the flow I'm working on. The conversion was successful.
The person FTPing the 2 files is sure there is no difference in the way he is sending the files. |
|
| Back to top |
|
|
| kirani |
| Posted: Mon Jul 28, 2003 1:05 pm Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
What are you doing in RCD node? Are you trying to parse the message using some MRM defn? Can I take a look at sample input message? Can I take a look at the code in the first compute node that's modifying your input message?
_________________
Kiran
IBM Cert. Solution Designer & System Administrator - WBIMB V5
IBM Cert. Solutions Expert - WMQI
IBM Cert. Specialist - WMQI, MQSeries
IBM Cert. Developer - MQSeries
|
|
| Back to top |
|
|
| tbt102 |
| Posted: Tue Jul 29, 2003 4:48 am Post subject: |
|
|
Apprentice
Joined: 21 Apr 2003
Posts: 28
|
Hi kirani,
In the RCD I'm parsing the message using a MRM defn (CWF) and want the data converted from binary to text.
Sample message binary format:
ñõôòññóðòóðñðññðñðÔðøðÔñøõççÙ×ðôô÷ñððóõÂÅððñðòÅñôðð
Sample message text format:
154211302301011010M080M185XXRP044710035BE00102E1400
Compute node modifying input message (snip of the esql)
| Code: |
. . . .
SET blobNewInputMessage = blobRecordCount;
WHILE intWorkIndex <= intRecordCount DO
SET blobNewInputMessage = blobNewInputMessage ||
SUBSTRING(InputRoot."BLOB"."BLOB" FROM intStartRecordPos FOR intRecordLenght) || blobRecordCount;
SET intStartRecordPos = intStartRecordPos + intRecordLenght;
SET intWorkIndex = intWorkIndex + 1;
END WHILE;
SET OutputRoot."BLOB"."BLOB" = blobNewInputMessage;
PROPAGATE;
. . . . |
|
|
| Back to top |
|
|
| Craig B |
| Posted: Tue Jul 29, 2003 6:51 am Post subject: |
|
|
Partisan
Joined: 18 Jun 2003
Posts: 316
Location: UK
|
Hi,
initially in your CAST function which converts the BLOB to CHAR you have specified a CCSID parameter of 819 and encoding of 546. Then when you swapped these hardcoded values for InputRoot.MQMD.CodedCharSetId and InputRoot.MQMD.Encoding this produced values of 437 and 546. However, the data you have the in messageBody is EBCDIC data not ASCII data. Therefore when 819 or 437 is used as the CCSID value, you are saying that the BLOB data is in there codepages. Therefore the bytes will be converted from this codepage to Unicode and this is why the BLOB is being 'corrupted'. So for example,where you have x'F1' ('1' in EBCDIC), then when you use 437, it will think the x'F1' is in 437 and perform a mapping of x'F1' to Unicode which is incorrect. When you send your MQSeries message in containing EBCDIC data then you need to specify the MQMD.CodedCharSetId and MQMD.Encoding to appropriate values to match this data.
Also, it is not clear as to why you are trying to cast this BLOB data to CHARACTER since you intend to parse it with the MRM anyway. The MRM parser does not need data to be in a specific code page. It will parse against your dictionary definition, and your codepage specific data will be converted to the Unicode message tree. You can then specify a different OutputRoot.MQMD.CodedCharSetId to produce an output message in a different code page from the message flow.
Usually when the MRM is being used, then you have a mix of Character data and Integer data. When this is the case, then you cannot CAST the BLOB data to CHARACTER, because not all the bytes represent character data. If this is the case, then this usually means that conversion cannot be done on the MQInput node, because the queue manager would need a data conversion exit to correctly convert the character and numeric data separately.
But since you are using the MRM parser, you do not need to manually cast the data, or use the conversion on the MQInput node. The MRM parser should take the codepage into account when parsing the data into the message tree.
Hope this helps
_________________
Regards
Craig |
|
| Back to top |
|
|
| tbt102 |
| Posted: Wed Jul 30, 2003 4:09 am Post subject: |
|
|
Apprentice
Joined: 21 Apr 2003
Posts: 28
|
Hi Craig B,
My apologies to you and all the others. Yes the data is in EBCDIC it is FTP from the OS/390 in binary format but the final result is EBCDIC. Sorry about the confusion.
The cast was intended as an illustration. I do appreciate your comments. They help strip away more MRM mystery for me. Thanks. |
|
| Back to top |
|
|
| tbt102 |
| Posted: Wed Jul 30, 2003 4:12 am Post subject: |
|
|
Apprentice
Joined: 21 Apr 2003
Posts: 28
|
Hi kirani,
The message I'm working with is 51 bytes in length. As a test I had the message length increased to 64 at the OS/390 and FTPed to me. I put the new message through the flow and the conversion was successful.
As another test I had a colleague put both messages through a flow on a different machine. The conversion of the file that is 51 bytes long failed. The conversion of the file that is 64 bytes long was successful. Any thoughts? Thanks. |
|
| Back to top |
|
|
| kirani |
| Posted: Wed Jul 30, 2003 12:52 pm Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
What's the OS on these 2 machines? what utility are you using to put the message on the queue? What's ur input CCSID and Encoding in MQMD header?
_________________
Kiran
IBM Cert. Solution Designer & System Administrator - WBIMB V5
IBM Cert. Solutions Expert - WMQI
IBM Cert. Specialist - WMQI, MQSeries
IBM Cert. Developer - MQSeries
|
|
| Back to top |
|
|
| tbt102 |
| Posted: Wed Jul 30, 2003 12:55 pm Post subject: |
|
|
Apprentice
Joined: 21 Apr 2003
Posts: 28
|
Hi kirani,
As a test I ran the 51 byte message through the following code and the conversion worked
| Code: |
DECLARE Fld as CHAR;
DECLARE Fld as CHAR;
SET Environment.Variables.InputMessage = cast(InputRoot.BLOB.BLOB as char ccsid 500);
SET Fld1 = CAST(SUBSTRING (InputRoot.BLOB.BLOB from 1 for 11) as char ccsid 500);
SET Fld2 = cast(substring (InputRoot.BLOB.BLOB from 12 for 40) as char ccsid 500); |
When I ran the 64 byte message through the conversion faild. Any Thoughts?
Sorry about all the posting. |
|
| Back to top |
|
|
| kirani |
| Posted: Wed Jul 30, 2003 1:11 pm Post subject: |
|
|
Jedi Knight
Joined: 05 Sep 2001
Posts: 3779
Location: Torrance, CA, USA
|
Could you please capture the traces for both the tests and post the output here? You can just put ${Root} in a trace node.
_________________
Kiran
IBM Cert. Solution Designer & System Administrator - WBIMB V5
IBM Cert. Solutions Expert - WMQI
IBM Cert. Specialist - WMQI, MQSeries
IBM Cert. Developer - MQSeries
|
|
| Back to top |
|
|
|
|
