| Author |
Message
|
| fabyos |
 Posted: Fri Aug 21, 2009 10:42 am Post subject: Optionals with TDS parsing |
 |
|
Apprentice
Joined: 15 May 2009
Posts: 37
|
Hi,
I'm trying to parse a string with TDS in order to produce a customized tree.
I've read the fixed post with a tutorial about tagged fixed length with TDS but it not worked on my message definition.
I have created a similar message definition with the tutorial and the last element is defined as optional by setting the cardinality 0..1.
Ok, the tree is ok, however when adding more than one optional element or between two elements that is supposed to be, it fails.
Let me show an example, the string comes with an start and end groups.
1) #STARTXXXX
2) #STARTXXXX#ENDYYYY
it work with the tutorial about TDS, passing 1) and 2) tests.
Adding a third element
#STARTXXXX#DATAYYYY#ENDZZZZ
And, the #DATA is optional (0..1) it fails, because it expects the #DATA.
My message set:
| Code: |
+ TdsMessage
+-+ START (1..1, tag: #START, group identifier: #START)
+- SIZE=XXXX (length: 4)
+-+ DATA (0..1, tag: #DATA, group identifier: #DATA)
+- CONTENT=YYYY (length: 4)
+-+ END (1..1, tag: #END, group identifier: #END)
+- CODE=ZZZZ (length: 4)
|
the exception:
| Code: |
(0x01000000):ParserException = (
(0x03000000):File = 'F:\build\S600_P\src\cpi\pwf\nxd\nxdscanner.cpp'
(0x03000000):Line = 103
(0x03000000):Function = 'NXDScanner::extractFixedLengthData'
(0x03000000):Type = ''
(0x03000000):Name = ''
(0x03000000):Label = ''
(0x03000000):Catalog = 'BIPv600'
(0x03000000):Severity = 3
(0x03000000):Number = 5426
(0x03000000):Text = 'Not enough data in bitstream'
)
|
I'm using broker V6, not 6.1 |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Fri Aug 21, 2009 11:23 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| You have set both the Tag and the Group Indicator. |
|
| Back to top |
|
|
| fabyos |
| Posted: Fri Aug 21, 2009 11:32 am Post subject: |
|
|
Apprentice
Joined: 15 May 2009
Posts: 37
|
| mqjeff wrote: |
| You have set both the Tag and the Group Indicator. |
Hum, the fact that I set up both tag and group indicator is because tag doesnt consumes the "tag"... I dont know why, when I do a simple test like this:
#STARTXXXX
| Quote: |
+- START (tag: #START)
+- SIZE XXXX
|
the #START keeps there, and fails the parse... but when I put the group indicator it works as it should...
And using only Group Indicator it fails with optional... My conclusions until now is that the options is only valid with terminal elements... but I'm not sure about it... |
|
| Back to top |
|
|
| fabyos |
| Posted: Fri Aug 21, 2009 12:01 pm Post subject: |
|
|
Apprentice
Joined: 15 May 2009
Posts: 37
|
Just to complement, here is a simple parsing using TDS
#INIXXXX
I'm expecting the following tree:
| Code: |
+ TdsTree
+-+ INI
+- SIZE=XXXX
|
So I set up my Message set with the following configuration: (note that the configuration not showed is because the default settings is used)
| Code: |
+ TdsTree
+-+ INI (1..1, tag: #INI)
+-+ {local complex type}
+- SIZE=XXXX (1..1, length: 4, type: xsd:string)
|
Note. I'm not using the tag parameter on the last child (SIZE), I'm putting it on INI. I'm not showing my full implementation, but i could have more that one child on INI... and my data is not tagged, it is just fixed length.
And parsing it on ESQL code:
| Code: |
DECLARE responseBlob BLOB CAST('#INI1234' AS BLOB CCSID 1208);
CREATE LASTCHILD OF OutputRoot DOMAIN('MRM') PARSE (
responseBlob
ENCODING InputRoot.Properties.Encoding
CCSID 1208
SET 'TdsParsing'
TYPE 'TdsTree'
FORMAT 'TDS1');
|
the exception:
| Code: |
(0x01000000):ParserException = (
(0x03000000):File = 'F:\build\S600_P\src\MTI\MTIforBroker\MtiImbParser2\MtiImbFIHandler.cpp'
(0x03000000):Line = 998
(0x03000000):Function = 'MtiImbFIHandler::endMessageContent'
(0x03000000):Type = ''
(0x03000000):Name = ''
(0x03000000):Label = ''
(0x03000000):Catalog = 'BIPv600'
(0x03000000):Severity = 3
(0x03000000):Number = 5288
(0x03000000):Text = 'MTI. Not all the buffer was used when reading message'
(0x01000000):Insert = (
(0x03000000):Type = 5
(0x03000000):Text = ''
)
)
|
As I said, the tag is not consuming the token, I dont know why.. unless it is this the expectation...
If this is kind of parsing is not possible, the TDS parsing is too much limited... |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Aug 21, 2009 2:40 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| If this is kind of parsing is not possible, the TDS parsing is too much limited... |
Tut. You can't give up that easily 
I suspect that you have failed to set Data Element Separation to 'Tagged Delimited' or 'Tagged Fixed Length'. It defaults to 'Fixed Length'.
You have not taken a debug-level user trace. If you did, it would show you what the TDS parser is up to, and it might even confirm my suspicion about Data Element Separation. I strongly suggest that you get into the habit of using User Trace for debugging message flows. When other techniques fail, or when you are diagnosing parsing errors, it is by far the most powerful diagnostic tool. |
|
| Back to top |
|
|
| fabyos |
| Posted: Mon Aug 24, 2009 6:51 am Post subject: |
|
|
Apprentice
Joined: 15 May 2009
Posts: 37
|
| Does WMB V6.0 have "tagged fixed length"? I think it was implemented on WMB 6.1... |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Aug 24, 2009 6:58 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I think it's best if I don't reply to that.  |
|
| Back to top |
|
|
| fabyos |
| Posted: Mon Aug 24, 2009 1:36 pm Post subject: |
|
|
Apprentice
Joined: 15 May 2009
Posts: 37
|
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Aug 24, 2009 9:54 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| fabyos wrote: |
| How do I trace parse? |
Sigh...
1) Enable user trace on the flow either through the Toolkit or the command line
2) Send a message through the flow
3) Disable user trace
4) read the trace log ( mqsireadlog)
5) format it into human readable form (mqsiformatlog)
It is all explained in the documentation.  (OR take some formal training)
With V6.1 a lot more information about where the parser was having problems is included in the user trace output.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| fabyos |
| Posted: Tue Aug 25, 2009 1:52 pm Post subject: |
|
|
Apprentice
Joined: 15 May 2009
Posts: 37
|
Ok, my journey on TDS parsing keeps going..
Looking at Infocenter at http://publib.boulder.ibm.com/infocenter/ratdevz/v7r1m1/index.jsp?topic=/com.ibm.etools.est.doc/ref/rsfmsg006.html, I created a TDS definition to this string:
| Code: |
| {Tag1:Data1*Tag2:[Data2+Data3]*Tag3:+Data4Data5-} |
(only changed the < and > to + and -)
| Code: |
+ Message2 (All Elements Delimited, Indicator: '{', Terminator: '}', Delimiter: '*')
+-+ Tag1
+-+ {local complex type} (Tagged Fixed Length, Separator: ':')
+-+ Data1 (string, Tag: 'Tag1', Length: 5)
+-+ Tag2
+-+ {local complex type} (Tagged Delimited, Separator: ':')
+-+ Group (Tag: 'Tag2')
+-+ {local complex type} (All Elements Delimited, Indicator: '[', Terminator: ']', Delimiter: '+')
+-+ Data2 (string)
+-+ Data3 (string)
+-+ Tag3
+-+ {local complex type} (Tagged Fixed Length, Separator: ':')
+-+ Group (Tag: 'Tag3')
+-+ {local complex type} (Fixed Length, Indicator: '+', Terminator: '-')
+-+ Data4 (string, length: 5)
+-+ Data5 (string, length: 5)
|
Ok, it parses, but I needed to create a Group element for Tag2 and Tag3 to accomplish the goal...
There is a way to parse without Group?
After this, my main motive to try this is to have a middle element that is optional, in this case, tag2, for example...
So, I want to parse things like this:
| Code: |
{Tag1:Data1*Tag3:+Data4Data5-}
{Tag1:Data1} |
However, it fails... I changed the cardinality for tag2 and tag3 to 0..1, but fails...
I tried the to read the user trace, but not used to it...
| Code: |
2009-08-25 18:04:02.086088 4008 ParserException BIP5285E: Message Translation Interface Parsing Errors have occurred:
Message Set Name : ''MyMessageSet''
Message Set Level : '1'
Message Format : ''TDS1''
Message Type Path : ''Message2''
Review further error messages for an indication to the cause of the errors.
2009-08-25 18:04:02.086152 4008 ParserException BIP5421S: Tagged/Delimited String Format (TDS) parsing error
Current message : ''377^Tag3''
Path to current element : ''/Message2/377^Tag3''
Offset from start of message : 28
See following errors for more details.
2009-08-25 18:04:02.086179 4008 ParserException BIP5604E: Missing markup following a complex child or group
No markup was found following the complex child or group '377^Tag3'. The parent of this child or group has a Data Element Separation of 'All Elements Delimited' or 'Variable Length Elements Delimited'. All complex children within a type or group with a Data Element Separation of 'All Elements Delimited' or 'Variable Length Elements Delimited' must be followed by some markup. That is a 'Repeating Element Delimiter', a 'Delimiter', a 'Group Terminator' or some markup from a higher level in the message model.
If the message model is valid correct the incoming message bitstream and parse again. If the incoming message bitstream is correct modify the message model to conform to the message bitstream. Then redeploy the message model to the broker and parse the message bitstream again. |
Any suggestions? |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Aug 26, 2009 2:44 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Look again at your debug-level user trace. It should contain messages like this:
| Code: |
2009-08-26 04:12:07.790060 11492 UserTrace BIP5613I: ''START'' was matched as the group indicator for ''/TDSPARSE'' at byte '5'.
2009-08-26 04:12:07.791981 11492 UserTrace BIP5612I: ''A:'', ending at byte '7', was matched as the tag for ''/TDSPARSE/StructureA''.
2009-08-26 04:12:07.794346 11492 UserTrace BIP5613I: ''['' was matched as the group indicator for ''/TDSPARSE/StructureA'' at byte '8'.
2009-08-26 04:12:07.796754 11492 UserTrace BIP5609I: ''/'' has been matched as a delimiter for ''/TDSPARSE/StructureA'' at byte '13'.
2009-08-26 04:12:07.800707 11492 UserTrace BIP5609I: ''/'' has been matched as a delimiter for ''/TDSPARSE/StructureA'' at byte '18'. |
If you don't see those messages, trying increasing the size of your trace file ( -c switch on mqsichangetrace ) |
|
| Back to top |
|
|
| fabyos |
| Posted: Wed Aug 26, 2009 10:45 am Post subject: |
|
|
Apprentice
Joined: 15 May 2009
Posts: 37
|
Is there a way to parse something like this:
[TAG][SIZE][DATA]
example:
ERROR100MESSAGE_WITH_SIZE_100
| Code: |
TAG = ERROR
SIZE=100
DATA=MESSAGE_WITH_SIZE_100 |
Update I use the User Data Pattern and was able to separate the data, but if there is a way to use the size to match the data it could be fine.. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Aug 26, 2009 11:12 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Look in the documentation at the Data Element Separation options.
I think you will be pleased with what you find. |
|
| Back to top |
|
|
|
|
