| Author |
Message
|
| gechu |
 Posted: Mon Dec 12, 2011 7:12 am Post subject: add line-feed using message set |
 |
|
Apprentice
Joined: 27 Feb 2008
Posts: 48
|
The message set is built up using MRM TDS:
| Code: |
complexTypeContainer (taged delimited, unordeted set, 0 to many)
complexType1 (fixed length, sequence)
elementA
elementB
complexType2 (fixed length, sequence)
elementC
elementQ
complexType3 (fixed length, sequence)
elementZ
elementK
|
It corresponds well to the input data which looks like this:
| Code: |
| complexType1complexType3complexType1complexType2complexType1complexType3 .. (random order) |
BACKGROUND
In the message flow an input message should result in two output messages. One transformed to a XML formated message, the other should be similar to the input message but with line-breaks after each fixed length field.
I want to solve it by using two physical message models. One which is built to handle what the actual message looks like when I get it from the wire (TDS1),, and another one which contains line-breaks (TDS2).
First I parse it using TDS1 into my logical model. Then change the physical format to TDS2 which should contain <CR><LF>. Problem is that I just can´t get the parser to write these <CR><LF>, which I´ve tried to add in two different ways:
1. at complexTypeContainer level as a "delimiter".
2. at complexTypeX level as a "group terminator".
What am I doing wrong?
I am using version 7001, CMVC Level: S700-FP01.
Thanks, Erik |
|
| Back to top |
|
 |
| kimbert |
| Posted: Mon Dec 12, 2011 7:27 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Your overall approach is completely valid - that is exactly how I would have approached the task. Either Delimiter or Group Terminator should work just fine - choose Group Terminator if you want a trailing line feed, delimiter otherwise.
Possible causes:
a) You didn't actually change the physical format, so the TDS writer is just outputting the original data stream.
b) You didn't specify the line feed correctly in the Group Terminator / Delimiter property.
c) Your input data did not match the message model, but the TDS parser helpfully parsed it as 'self-defining data'. This is rare, and only happens if your data is tagged and simple. However, I don't have full details of your model...
Next steps:
1. Prove that the parsing has worked 100% correctly. Do that by adding a Trace node after the input node and inspecting the message tree carefully.
2. Prove that you really did change OutputRoot.Properties.MessageFormat. Do this by adding a Trace node just before the output node, or by taking a user trace, or both. This will also eliminate c) from our enquiries. |
|
| Back to top |
|
|
| gechu |
| Posted: Thu Dec 15, 2011 6:08 am Post subject: |
|
|
Apprentice
Joined: 27 Feb 2008
Posts: 48
|
Looking in a trace I find this:
| Code: |
| (0x03000000:NameValue):MessageFormat = 'TDS2' (CHARACTER) |
Note: TDS2 is my other physical format which contains line breaks. How do you mean when you write "inspect the message tree carefully"?
When I add line break, either as a "group terminator" or "delimeter" I write it as this:
Is it correct? |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Dec 15, 2011 6:22 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| How do you mean when you write "inspect the message tree carefully"? |
I mean 'do you get *exactly* the message tree that you expected to get?
| Quote: |
Looking in a trace I find this:
|
You did not say where the Trace node was located. I assume that it was positioned just before the MQOutput node?
Next step is to take a debug-level user trace. That will tell you whether
a) the TDS parser actually bothered to do any work or
b) it decided to lazily re-use the input message. |
|
| Back to top |
|
|
| gechu |
| Posted: Fri Dec 16, 2011 3:59 am Post subject: |
|
|
Apprentice
Joined: 27 Feb 2008
Posts: 48
|
You were correct about the location of the trace node.
Now I´ve analyzed the user trace. It complained about the mcd being jms_bytes when I receive the message and treat is as a BLOB.
A side track. I think the mcd=jms_bytes prevented me from setting the MQFORMAT to MQSTRING using a "MQ Header Node". However, if I used ESQL I could set it to MQSTRING and by that way remove the jms_bytes issue. Anyhow, I´m not sure this "jms thing" had anything to do with my problem..
What I do think might have something to do with it, is that the user trace ends with a row telling me that XMLS parser was used:
"BIP6061I: Parser type ''XMLS'' created on behalf of node '.InputNode' to handle portion of incoming message of length '231' bytes beginning at offset '364'. Parser type selected based on value ''XMLS'' from previous parser. "
I find no traces of my TDS2 message format being used. Should I see it somewhere?
Here is a more comprehensive citation from the user trace log:
| Code: |
2011-12-16 12:43:51.989734 3816 UserTrace BIP2632I: Message received and propagated to 'out' terminal of MQ input node '.InputNode'.
2011-12-16 12:43:51.989902 3816 UserTrace BIP6060I: Parser type ''Properties'' created on behalf of node '.InputNode' to handle portion of incoming message of length 0 bytes beginning at offset '0'.
2011-12-16 12:43:51.989933 3816 UserTrace BIP6061I: Parser type ''MQMD'' created on behalf of node '.InputNode' to handle portion of incoming message of length '364' bytes beginning at offset '0'. Parser type selected based on value ''MQHMD'' from previous parser.
2011-12-16 12:43:51.990039 3816 UserTrace BIP6061I: Parser type ''XMLS'' created on behalf of node '.InputNode' to handle portion of incoming message of length '231' bytes beginning at offset '364'. Parser type selected based on value ''XMLS'' from previous parser. |
|
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Dec 16, 2011 4:09 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| Now I´ve analyzed the user trace. It complained about the mcd being jms_bytes when I receive the message and treat is as a BLOB. |
I don't understand most of that. Did your Trace node after the input node show that the *parsing* worked correctly? Because if that's going wrong then everything downstream of the input node will go wrong.
| Quote: |
| the user trace ends with a row telling me that XMLS parser was used: |
The XMLS parser is used internally by message broker. Those lines in the user trace may well be referring to a deployment or some other administration message that was sent to the execution group.
You should do the following:
a) add a Trace node after the input node to check that parsing is working *exactly* as you intend
b) If a) does not show any problems then take another user trace, and read the lines that are added by your flow. Pay special attention to the entries that relate to the writing of the message. |
|
| Back to top |
|
|
| gechu |
| Posted: Mon Dec 19, 2011 12:31 am Post subject: |
|
|
Apprentice
Joined: 27 Feb 2008
Posts: 48
|
Hi again, still no solution in sight. Here are some more contextual info:
1. MQInputNode:
receive it as BLOB
2. ComputeNode:
cast payload from 278 to 1208
change payload CCSID descriptor to 1208
create MQRFH2 header value "MQFMT_STRING"
3. ResetContentDescriptorNode
resets all values in the "basic" tab. I think the parsing goes well. In the end of the user trace I get this line which I think is normal due to the unbound number of rows (in random order) described in my msg-set:
| Quote: |
2011-12-19 08:09:36.434501 3236 UserTrace BIP5612I: ''WRITE'', ending at byte '206541', was matched as the tag for ''/msg_123A/WRITE_filler''.
2011-12-19 08:09:36.434539 3236 UserTrace BIP5607I: While scanning for the end of a field, the end of the bit stream was encountered at byte '206664'. |
send one copy of the message to another ResetContentDescriptorNode, and another copy to a compute node where I do some more stuff and don´t have any problems.
4. AnotherResetContentDescriptorNode
change payload format descriptor to another physical format which contains line breaks.
MQOutput
write to MQ:
| Quote: |
2011-12-19 09:22:00.264001 4440 UserTrace BIP2638I: The MQ output node 'myMQOutNode..' attempted to write a message to queue ''myQueueOut'' connected to queue manager ''''. The MQCC was '0' and the MQRC was '0'.
2011-12-19 09:22:00.264023 4440 UserTrace BIP2622I: Message successfully output by output node 'myMQOutNode..' to queue ''myQueueOut'' on queue manager ''''. |
|
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Dec 19, 2011 3:43 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Your solution looks over-complicated to me. The actions listed under '2. ComputeNode' are not necessary. You don't need to change the CCSID of the input message before you parse it - the MRM parser can parse it in the original CCSID. Maybe you need the output XML to be in UTF-8 ( CCSID 1208 )? If so, just set OutputRoot.Properties.CodedCharSetId='1208' before writing the XML.
So, my advice, based on the requirements that you have given is:
- Remove the compute node
- In the MQInputNode set the Domain to 'MRM' and parse the incoming message in its original CCSID.
- Send the parsed message tree to two branches of your flow. In branch one, copy InputRoot.MRM to OutputRoot.XMLNSC.yourRootTagName and output the message using an MQOutput node. In the other branch, change OutputRoot.Properties.MessageFormat to 'TDS2' using a Compute node or a ResetContentDescriptor Node and output the message using an MQOutput node. |
|
| Back to top |
|
|
| gechu |
| Posted: Tue Dec 20, 2011 1:32 pm Post subject: |
|
|
Apprentice
Joined: 27 Feb 2008
Posts: 48
|
Kimbert, thanks for your patience! The reason for all the steps under "2. ComputeNode" is that the MQMD is falsely set. But it shouldn´t cause any trouble because the payload is handled as a binary until it reaches the broker. Anyhow, I will build a message flow according to you guidelines and report my findings.
I have two questions about modeling a tagged delimited message without unbound occurrences:
1. Is it even possible to model "the end" of this kind of message? If not, I´m thinking this message is unavoidable:
| Quote: |
2011-12-19 08:09:36.434539 3236 UserTrace BIP5607I: While scanning for the end of a field, the end of the bit stream was encountered at byte '206664'.
|
2. In the message set I build a tag identifier for each possible tag (i.e. tag XYZ, YXZ..). If the parser reaches an unidentified tag it will stop parsing. But that behavior isn´t very robust. I´d rather want it to keep searching for matching tags until the end of the file.
A previsously mentioned, the message definitions consists of two different formats: The outside format is taged fixed length, and the inside format is fixed length. I´m thinking the parser can´t match all possible characters for matches, but if I know the length of the "unknown" tag it would only have to check once for every fixed length. Is this a possible solution? If so, can you please give me some hints on how to implement in? |
|
| Back to top |
|
|
| gechu |
| Posted: Mon Jan 02, 2012 12:27 am Post subject: |
|
|
Apprentice
Joined: 27 Feb 2008
Posts: 48
|
Follow up on the subject.
I played around with the message type properties. According to my knowledge I should have been able to use composition "unorderedSet" and "content validation" set to "closed", but then I failed to parse the message. Therefore I think it was something in the way the message was constructed that wasn´t entirely correct.
In the end the "line break issue" wasn´t a very big deal for the receiving application so we decided to only focus on the transformation parts performed in the message broker. Too bad though, being forced to surrender like this hurts my ego 
Anyways, thanks Kimbert for your patience and thoughtful comments! |
|
| Back to top |
|
|
|
|
