| Author |
Message
|
| bb1234 |
 Posted: Thu Aug 25, 2011 12:24 am Post subject: Message parsing failing using data pattern in message set |
 |
|
Newbie
Joined: 24 Aug 2011
Posts: 4
|
We use TLOG and data pattern in message set to extract the field value from the bit stream.
We are unable to extract one field value using data pattern which is having 22 at last position. Say 250022 is the value of the element to be extracted from the below mentioned bitstream.
22133af03af03af13a2500223a993a113a250022220d0a
22 - Group Indicator – ” - Mark in orange
220d0a - Group Terminator - ”<CR><LF> - mark in blue
3a – Delemeter - : - mark in bold
Data part is marked as Brown
So in message set Group Indicator is mentioned as ” and Group Terminator is mentioned as ”<CR><LF>. We have delimiter between elements as : and hex value of : is 3a.
The Hexadecimal Value of ‘22’ is ”.
We used below data pattern for field identification and extraction in message set.
Below is the data pattern we have used to get the element value.
[-”]: (([^:"])|("[^,\r])|(",[^"])|("\r[^\n]))*
It is clear that, there is value of the element is coming as 250022 and after that Group Terminator is present ("<CR><LF>) in the failed record. As hex value of Double Quote(") is also having value 22 which is coming in front of Group terminator existing data pattern can not able to segregate element value 250022 and Group Terminator ("<CR><LF>) here and we get parsing error.
Suggest what data pattern we need to use here.
P.S. For string 22133af03af03af13a2500223a993a113a252200220d0a it successfully gets parsed with the data pattern mentioned above and the element value 252200 can be extracted.
But for all xxxx22 values data pattern is failing. |
|
| Back to top |
|
 |
| kimbert |
| Posted: Thu Aug 25, 2011 11:27 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Your data pattern does not seem to fit the data that you posted. Are you sure that you posted the correct combination of data and pattern?
- I cannot see any bytes with value 0x94 in your data
- the 4 parts of the data pattern are not using the same syntax to do the same things. I suggest that you carefully check what you posted against what you *meant* to post.
If you have a string that ends with XYZ, then your data pattern needs to be
You should be able to adapt that technique to match your terminator string. |
|
| Back to top |
|
|
| bb1234 |
| Posted: Thu Aug 25, 2011 11:49 pm Post subject: |
|
|
Newbie
Joined: 24 Aug 2011
Posts: 4
|
Hi kimbert,
Thanks for checking the issue.
In TLOG we have multiple records (say 00,01,02,....13) under Root tag. And this data pattern is used in one of such record to get the value.
Kindly find below he structure of the message set.
<Root>
{Local Complex Type}
Field Identification
Data element separation - All elements delimited
Group Indicator – "
Group Terminator - "<CR><LF>
Delimeter - ","
<Record(s)>
{Local Complex Type}
Field Identification
Data element separation - All elements delimited
Group Indicator – <u+0013>:
Group Terminator - BLANK
Delimeter - :
So here [-”] is mentioned to indentify that record as per my understanding. In this example record is 13. For each record we have group indicator like (<u+0013>:) shown above.
Rest of the data is identified by the 4 parts of the data pattern
(([^:"])|("[^,\r])|(",[^"])|("\r[^\n]))*
As you suggested I tried to match the terminator string in the pattern.
My group terminator is "\r\n. If I replicate it with your example it should be as (^"|("[^\r])|("\r[^\n]))*
Which is mentioned here.
(([^:"])|("[^,\r])|(",[^"])|("\r[^\n]))*
Our real problem is " is coming in last position of data feild (value 250022) as the Hexadecimal value of ‘22’ is ". After that we have group terminator "\r\n. Two "" coming together creating the problem.
Existing data pattern can not separate data 250022 and terminator 220d0a from 250022220d0a string as occurance of 2222 or "" creating the problem).
We do not have any delemeter also between last element and group terminator.
Kindly suggest a solution which solve the problem.
Also let me know if you know any more information.
P.S. - If we have any other element except 22 in before group terminator ( say string 250021220d0a) the data pattern extracts the data correctly. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Aug 26, 2011 12:41 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
Which is mentioned here.
(([^:"])|("[^,\r])|(",[^"])|("\r[^\n]))* |
I looked at that, but I don't think it's using the same technique as I suggested. Maybe I'm just not reading it correctly, but
- Your solution has two pairs of terms separated by |. My suggested solution used one triplet of terms separated by |.
- I don't understand what the colon in the first term is doing
- I don't understand what the comma after the ^) second term is doing. Or the comma after the " in the third term. |
|
| Back to top |
|
|
| bb1234 |
| Posted: Fri Aug 26, 2011 1:25 am Post subject: |
|
|
Newbie
Joined: 24 Aug 2011
Posts: 4
|
Hi kimbert,
Kindly find the clarification below.
| Quote: |
| I don't understand what the colon in the first term is doing |
[^:"] negates the character : also because : is our delimeter.
| Quote: |
| I don't understand what the comma after the ^) second term is doing. Or the comma after the " in the third term. |
, should not present after " as Delimeter "," is mentioned at the Root level field identification section.
(",[^"]) - this also used to maintain above mentioned Root type delimeter ","
Below is the structure of the Root element.
<Root>
{Local Complex Type}
Field Identification
Data element separation - All elements delimited
Group Indicator – "
Group Terminator - "<CR><LF>
Delimeter - ","
Kindly correct if I am wrong. |
|
| Back to top |
|
|
| mapa |
| Posted: Fri Aug 26, 2011 2:02 am Post subject: |
|
|
Master
Joined: 09 Aug 2001
Posts: 257
Location: Malmö, Sweden
|
So you are using datapatterns for CSV files...
I'd recommend you to look at the CSV example in the Broker Samples Gallery and learn from that instead.
[Edit: Sorry, responded without taking the TLOG into consideration...never seen or used myself so just ignore my answer if it is not applicable] |
|
| Back to top |
|
|
| bb1234 |
| Posted: Fri Aug 26, 2011 2:53 am Post subject: |
|
|
Newbie
Joined: 24 Aug 2011
Posts: 4
|
No. We use TLOG messaging standard. TLOG is the Point of Sale (POS) Transaction Log used in retail industry. Raw TLOG messages can be modeled using the MRM Tagged/Delimited String Format (TDS).
For TLOG messages fields in the messages can be defined as character format or in a format that is specific to the message. This requires that the Messaging Standard property (at the message set level) is set to TLOG. |
|
| Back to top |
|
|
|
|
