| Author |
Message
|
| cloucas |
 Posted: Tue May 23, 2006 2:40 am Post subject: Problem with parsing tagged delimited messages |
 |
|
Apprentice
Joined: 10 Mar 2006
Posts: 37
|
I am using TDS to model our host message structure which is composed of a fixed length header, followed by a body. The body is a complex type that consists of a number of tagged delimited messages. Each of the messages may appear 0 or 1 time in the body, such that:
[System Header][[MessageA][MessageB][MessageC]......]
Furthermore each message consists of a number of delimited fields. These fields are declared as global elements which the messages reference. Note that many messages share the same fields (i.e. a specific field may appear in more than one message):
Message A: [F1][F2][F3][F4]
Message B: [F1][F8][F9][F10]
Message C: [F1][F7][F11]
When I parse these I get the following parsing:
Message A: [F1][F2][F3][F4][F1]
Message B: [F8][F9][F10]....
My guess is that, although [F1] has been defined to exist a maximum of 1 time, the parser keeps reading until it encounters a field that does not exist in the Message A definition and then decides to switch to the next message. I, ofcourse could be off by a mile...
Note also that there is no end delimiter between the messages; each message start can be signalled by [F1].
I am stuck as I don't know how to model this. Any help is greatly appreciated. |
|
| Back to top |
|
 |
| juddg |
| Posted: Tue May 23, 2006 3:45 am Post subject: Clarification |
|
|
Apprentice
Joined: 22 Nov 2004
Posts: 33
|
Hi cloucas,
Please can you clarify the format of the data that your are parsing. Does each message have a tag or does the data look like?
[F1][F2][F3][F4][F1][F8][F9][F10][F1][F7][F11]
It would also be useful if you could describe the TDS model you have created
with the property values that you have set.
Regards,
juddg. |
|
| Back to top |
|
|
| cloucas |
| Posted: Tue May 23, 2006 4:25 am Post subject: Further info on TDS Parser Problem |
|
|
Apprentice
Joined: 10 Mar 2006
Posts: 37
|
Hi again
The messages themselves do not have a tag. However their first field is always [F1]. It pretty much shows like your model.
To help you with the properties of the TDS model:
The parent message properties are:
Logical Properties
Complex Type: Composition is SEQUENCE
Content Validation is CLOSED
Physical properties
Complex type: Data Element Separation: Variable Length Elements
Delimited
Delimiter: <U+0011>
Suppress: End Of Type
The Body Properties are:
Logical Properties
Complex Type: Composition is SEQUENCE
Content Validation is CLOSED
Physical properties
Complex type: Data Element Separation: Variable Length Elements
Delimited
Group Indicator: <U+0011>
Delimiter: <U+0011>
Suppress: End Of Type
The Message Properties are:
Logical Properties
Complex Type: Composition is unordered Set
Content Validation is closed
Physical properties
Complex type: Data Element Separation: Tagged Delimited
Delimiter: <U+0011>
Suppress: End Of Type
Length of Tag: 4
Within messages, each element is perceded by the character (Hex 11) and its tag (a 4-byte character). Because the TDS model specifies the delimiter as the character separating fields I had to also insert (Hex 11) as the group indicator to get it working. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue May 23, 2006 4:56 am Post subject: Re: Further info on TDS Parser Problem |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| cloucas wrote: |
| The messages themselves do not have a tag. However their first field is always [F1]. |
Does this first field have a specific range of values that are allowable for Message A, that are different than what is allowable for Message B? In particular, is this first field always a FIXED value for Message A, and a different fixed value for Message B? If so, it's a tag!
If not, then what characteristics differentiate Message A from Message B, and Message C, and etc? Is it order of fields? Is it a specific combination of fields?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue May 23, 2006 1:04 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
Complex type: Data Element Separation: Variable Length Elements
Delimited |
A small point, but worth noting. You should almost certainly be using 'All Elements Delimited'.
'Variable Length Elements Delimited' is intended for modeling structures which contain a mixture of delimited and fixed-length fields. This is a very common ( and very understandable) mistake.
I'll let juddg deal with the main issue. |
|
| Back to top |
|
|
| cloucas |
| Posted: Tue May 23, 2006 8:13 pm Post subject: |
|
|
Apprentice
Joined: 10 Mar 2006
Posts: 37
|
| Yes, the first field is always fixed and unique for each type of message. It actually contains the type of message. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue May 23, 2006 8:55 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| cloucas wrote: |
| Yes, the first field is always fixed and unique for each type of message. It actually contains the type of message. |
Then, in fact, this is a tag and you can use it as one.
The only side effect of this is that this first field will exist only as a name element and not as a name/value. But the data of the tag will be preserved. And any business logic you need to do based on this value you can do based on the existence of the name element.
So, for example, you would have Message A that would have a tag of the value of F1 for Message A. Then you would tell that you got a Message A because you would see something like InputRoot.MRM.Body.MessageA, and not something like InputRoot.MRM.Body.F1 who's value was the indicator for Message A.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| cloucas |
| Posted: Wed May 24, 2006 1:21 am Post subject: |
|
|
Apprentice
Joined: 10 Mar 2006
Posts: 37
|
Thanks for your response...
So, you mean that I can use the element as a tag? Where would I define it as a tag? |
|
| Back to top |
|
|
| juddg |
| Posted: Wed May 24, 2006 4:07 am Post subject: Clarification |
|
|
Apprentice
Joined: 22 Nov 2004
Posts: 33
|
Hi cloucas,
I think what jefflowrey means is to make the whole of the F1 field the tag for the messages. That is the tag and the value. From your description this value should be unique for each messages. Change the Data Element Separation of the body to be Tagged Delimited and set the Tag Data Separator to be <U+0011>. As jefflowrey says this will mean that F1 does not actually appear an an element in the tree. So for your input MessageA will have children F2,F3 & F4. From ESQL this will be
InputRoot.MRM.Body.MessageA.F2
InputRoot.MRM.Body.MessageA.F3
InputRoot.MRM.Body.MessageA.F4
I hope this helps,
Regards,
juddg. |
|
| Back to top |
|
|
| cloucas |
| Posted: Thu May 25, 2006 12:26 am Post subject: |
|
|
Apprentice
Joined: 10 Mar 2006
Posts: 37
|
Ok thanks for this
I have tried to define the messages as tagged delimited with tags the first field and its data. This however creates a problem as it seems to be conflicting with the Physical Properties of the Message Set. I created a second Physical Format and specified that the tag is 8, but it appears that all elements inherit both formats. The result is conflicts and compilation errors.
How do I handle this? Can I have two physical formats defined, one that applies to the individual messages and one that applies to their enclosing structure (body)? |
|
| Back to top |
|
|
| juddg |
| Posted: Thu May 25, 2006 12:37 am Post subject: |
|
|
Apprentice
Joined: 22 Nov 2004
Posts: 33
|
Hi,
There is no way of restricting a physical format to just part of a message set defintion. When you say:
| Quote: |
| This however creates a problem as it seems to be conflicting with the Physical Properties of the Message Set. |
What error do you get?
Regards,
juddg. |
|
| Back to top |
|
|
| cloucas |
| Posted: Thu May 25, 2006 9:40 pm Post subject: |
|
|
Apprentice
Joined: 10 Mar 2006
Posts: 37
|
Thanks for your reply.
Let me try to make sure I understood your recommendation correctly:
Since the three transaction messages are in fact elements of the Body structure, I went into the Body structure, expanded it, and for each of the three transaction messages I changed the physical properties Tag Value to be "T0017033", "T001GNFM" and "T001GNFP" respectively. Please note that T001 is the actual tag and 7033,GNFM and GNFP are the transaction types.
I first tried with the Body structure defined as Variable Length Elements Delimited and with only the delimiter set to <U+0011>.
When I do this the incoming message parses the 7033 transaction OK (although it shows in the trace the T001 field as part of the transaction message), and starts to parse the GNFM transaction, buts stops halfway through. There is no error in the trace. Note that for the GNFM message T001 tag does not appear in the trace as part of the transaction message.
When I now set the Body properties to be of Tagged Delimited and change the Tag Data Separator to <U+0011>, I get the error below in the trace:
2006-05-26 08:36:41.370399 3720 ParserException BIP5285E: Message Translation Interface Parsing Errors have occurred:
Message Set Name : 'messageSetResponse'
Message Set Level : '1'
Message Format : 'ELSResponseTDS'
Message Type Path : 'GenericResponse'
Review further error messages for an indication to the cause of the errors.
2006-05-26 08:36:41.370555 3720 ParserException BIP5421S: Tagged/Delimited String Format (TDS) parsing error
Current message : 'GenericResponse'
Path to current element : '/GenericResponse'
Offset from start of message : 1823
See following errors for more details.
2006-05-26 08:36:41.370639 3720 ParserException BIP5288E: MRM parsing error. Message bit stream larger than expected.
The bit stream of the message being parsed by the MRM parser is larger than expected for a message of this message type.
The message bit stream has been incorrectly constructed, or the logical model is incorrect.
Ensure that the message is correctly packaged.
Ensure that the message properties are correct.
Thanks for your continuing help. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri May 26, 2006 1:58 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
When I said
| Quote: |
| You should almost certainly be using 'All Elements Delimited' |
I was assuming that your data was not tagged. If your data is tagged, you definitely need to use 'Tagged Delimited'.
The BIP5288 exception means that the parser reached the end of the model before it reached the end of the bitstream - so the model needs to be corrected. See here http://www.mqseries.net/phpBB2/viewtopic.php?t=29550 for some tips on diagnosing/debugging BIP5288. |
|
| Back to top |
|
|
| juddg |
| Posted: Fri May 26, 2006 4:31 am Post subject: Another thing to check |
|
|
Apprentice
Joined: 22 Nov 2004
Posts: 33
|
Hi,
One thing I didn't make clear is that the F1 element should be removed as children of the messages in the model because all of the F1 element will be consumed by the tags of the messages.
Regards,
juddg |
|
| Back to top |
|
|
| jesieltrevisan |
| Posted: Tue Jun 27, 2006 4:48 am Post subject: |
|
|
Novice
Joined: 21 Mar 2006
Posts: 13
|
I have an error too with MessageFormat.
My file out must be a TXT File, with TAG delimited.
Exemplo:
[ Header ] Field01 Field03 Field03 ... etc...
[ Itens1 ] Filed01 Field02 Field03 ... etc...
[ Itens2 ] Filed01 Field02 Field03 ... etc...
[ Itens3 ] Filed01 Field02 Field03 ... etc...
Etc...
Well.. but in my file out, the message was with a space on the second line, before the first Item1.. and i don´t know why.
For exemplo, my messageSet is :
<Message>
<Header>
<Fields>
</Header>
<Item>
<Fields>
</Item>
</Message>
Somebody can help-me ?? I need to insert Into the Intems in Type of <Header> ?
Tks´s
_________________
Analista de Sistemas.
J2EE, WebSphere MQSI Skill´s
----------------------------------------------------
Brazil [<O>] |
|
| Back to top |
|
|
|
|
