| Author |
Message
|
| hellobond070 |
 Posted: Fri Mar 05, 2010 9:47 am Post subject: Separating large Message |
 |
|
Centurion
Joined: 18 Nov 2009
Posts: 118
|
Hello All,
I have a large message coming through a fileinput node.
Sample message looks as below:
| Quote: |
XXXXlfkjafjldfa
fjlasdjflajslf
lajfladslf
XXXXlfjaldjfldsjfl
alfjdslfjaljfl
XXXXlsjfajlf
aflajfjal |
I need to separate them as 3 messages
| Quote: |
FIRST
XXXXlfkjafjldfa
fjlasdjflajslf
lajfladslf
SECOND
XXXXlfjaldjfldsjfl
alfjdslfjaljfl
THIRD
XXXXlfjaldjfldsjfl
alfjdslfjaljfl
XXXXlsjfajlf
aflajfjal |
Everytime the characters XXXX appears, I have to start from that and till the next XXXX should consider this as one message and propagate to a file. So 3 files would get generated.
But if I use to model this as a TDS and give delimiter as XXXX, I lose the delimiter XXXX in the final output message. I can then use a compute node and concatenate XXXX but which I want to avoid.
Can someone help me to show how using a MRM I create a parse method to achieve separation of the large file to the expected output ??? |
|
| Back to top |
|
 |
| kimbert |
| Posted: Fri Mar 05, 2010 10:00 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Why not simply set the delimiter in the FileInput node to 'XXXX'? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Mar 05, 2010 10:02 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| If it comes at the *start* of every record, it's not a *delimiter*, it's a *tag*. |
|
| Back to top |
|
|
| hellobond070 |
| Posted: Fri Mar 05, 2010 10:29 am Post subject: |
|
|
Centurion
Joined: 18 Nov 2009
Posts: 118
|
Yes mqjeff , Kimbert.
I need the XXXX to be present at the beginning of each record. I do not want to lose it. And the only way to differentiate where the incoming message needs to be cut is at "XXXX".
Please let me know in case any further information is required.
Please suggest |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Mar 05, 2010 11:42 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Use TDS.
Configure it as a TAG rather than as the DELIMITER.
Then it will get output when you send the records out. |
|
| Back to top |
|
|
| kimbert |
| Posted: Fri Mar 05, 2010 2:34 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Point taken, mqjeff. I was trying to avoid the complexity and CPU cost of a TDS model for such a simple scenario. The other option would be to write a small piece of ESQL which chops the incoming BLOB into pieces and PROPAGATEs them. But that would be almost identical to using the delimiter on the FileInput node, and less maintainable.
To avoid confusion, TDS tags and TDS delimiters get output by the TDS parser when the message is written. I know you know that, but other readers might not.
The delimiter in the FileInput node never even makes it into the message flow. Not even in the environment tree, as far as I am aware ( I would love to be corrected on that, because it would make FileInput suitable for this scenario ). |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sat Mar 06, 2010 3:49 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| TDS can suppress absent element delimiters. |
|
| Back to top |
|
|
| hellobond070 |
| Posted: Wed Mar 10, 2010 11:14 am Post subject: |
|
|
Centurion
Joined: 18 Nov 2009
Posts: 118
|
Thanks mqjeff.
I tried using the below :
Complex type element
Data Element Separation -> Tag delimited
Lenght of the Tag -> 4
Simple type element of the above complex type
Tag -> XXXX
It's taking the whole thing as one message. In the MRM, I don't see 3 different messages with XXXX as tag preceding them
| Quote: |
XXXXlfkjafjldfa
fjlasdjflajslf
lajfladslf
XXXXlfjaldjfldsjfl
alfjdslfjaljfl
XXXXlsjfajlf
aflajfjal |
Can somebody please suggest the way to go ahead. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Mar 10, 2010 2:19 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I could probably work out the correct TDS model, given enough time.
However, I don't think this is really a task for TDS. You're not really parsing the incoming data stream, you're just chopping it up.
I suggest that you either
a) set the delimiter on the FileInput node to XXXX, and re-add the 'XXXX' to each BLOB before you continue with the message flow
or
b) set the Domain of the FileInput node to BLOB and write some ESQL which searches for the delimiter, extracts each sub-message using SUBSTRING and PROPAGATEs it to the remainder of the flow.
Of the two solutions, I would go for a). |
|
| Back to top |
|
|
| hellobond070 |
| Posted: Sat Mar 20, 2010 11:23 am Post subject: |
|
|
Centurion
Joined: 18 Nov 2009
Posts: 118
|
Finally, I have read this file line by line and done the computation and given it to output.
Its working fine. |
|
| Back to top |
|
|
|
|
