ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » How to handle /r/n delimiter in the message set

Post new topic  Reply to topic
 How to handle /r/n delimiter in the message set « View previous topic :: View next topic » 
Author Message
swethabala
PostPosted: Sun Jun 20, 2010 9:12 pm    Post subject: How to handle /r/n delimiter in the message set Reply with quote

Apprentice

Joined: 25 Aug 2009
Posts: 44

Hi,

We have a flat file input coming from Legacy. The file looks like this:

OP01888SHH 881LOC HH 100412SHHRAS 0000000000000000042640000000000000000000000000000000000000 095920080000
RA501514481613753001832968001724828002B706PF 000000000009550103514481100401051043
RA501514629613755001832970001724828005B706PF 000000000009550103514629100402051043
RA501514638613754001832969001724828009B706PF 000000000009550103514638100406051043
CL99888SHH 881LOC HH 100412SHHRAS 095917000000000000000000000000000000000000000000000000000003


the line starting with OP01 is a header line. after that RA lines are the body of the message (which are repeating). CL99 is Footer.

here all the lines are seperated by /r/n line seperator. in our message set this delimiter is not getting handled. eventhough we declared group terminator as <CR><LF>.

Could you please let me know how to handle the delimiter in this text message? now the message set is reading /r/n as a content of the intput so length mismatch is coming.

second doubt is: as you observed in the message RA lines are repeating of same length. After RA lines CL99 footer is coming. in the message set even though I declare an element for Footer CL99 line is read as a Body line only. because this is a blob message its not able differentiate. If the length of Footer is same as body length we could manage; but here Footer is of 101 char length and body is of 86 char length. So How to handle this Footer line in message set.

Please reply if you can answer atleast one doubt of the above two.

Thanks a lot in advance.
Back to top
View user's profile Send private message
sushilishere
PostPosted: Sun Jun 20, 2010 9:38 pm    Post subject: Reply with quote

Apprentice

Joined: 27 Jul 2008
Posts: 36

Hi,

Why is the group terminater <CR><LF> when the delimiter shd be <CR><LF>!!

The group terminator does the final delimter check for ending the parsing by the complex element.

I wld suggest you to use Tag delimited as they are very useful in this scenario!!

1 complex element - tag delimited
1st local element - OP01 - 1-1
2nd local element - RA - 1 - (-1)
3rd local element - CL99 - 1-1

this shd help you. Else you can directly use a complex element of type variable length delimited with delimiter of CRLF. No need of any group terminator.

But then u will have to validate each line for its type(header/data/trailer) in esql/JCN.

Hope this helps!!1
_________________
----------------------------
IBM MQ v6.0 Sys Admin
IBM MBv6.0 Sol Desgnr
----------------------------------------------------
By the Mountain Side.. looking at the Great Divide
----------------------------------------------------
Back to top
View user's profile Send private message
swethabala
PostPosted: Mon Jun 21, 2010 1:10 am    Post subject: Reply with quote

Apprentice

Joined: 25 Aug 2009
Posts: 44

Thanks for your reply; I tried keeping <CR><LF> in the Delimiter field in but still the message is read with /r/n. I have pasted the content appearing on Debug for a test.

payload 1043\r\nRA501514638613754001832969001724828009B706PF 000000000009550103514638100406 (I want the data to be read without /r/n; because of /r/n data from first record are passed to second record and spoiled the total data structure)

May be I am doing something wrong while creating the message set. but I tried to handle this delimiter in all the places but could not solve this problem.


Regarding the second doubt Yes I can handle it in the code but the RA line and footer are having different lengths.
Back to top
View user's profile Send private message
kimbert
PostPosted: Mon Jun 21, 2010 1:28 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

hi swethabala,

The TDS parser is not easy to configure, and you need to solve your problem one step at a time.

1. Parse the message as lines using Data Element Separation='All Elements Delimited' with Delimiter set to '<CR><LF>. This is not the solution - it is the first step toward a solution.
Code:

element name='message '
    complexType DataElementSeparation='All Elements Delimited' GroupTerminator='<CR><LF>'
        element name='Line' maxOccurs='unbounded' RepeatingElementDelimiter='<CR><LF>'


2. Add tags, and create separate elements for Header, Body and Footer
Code:

element name='message '
    complexType DataElementSeparation='Tagged Delimited' GroupTerminator='<CR><LF>' Delimiter='<CR><LF>'
        element name='Header' maxOccurs='1'
                Tag='OP01'
        element name='Body' maxOccurs='unbounded'
                Tag='RA50'  RepeatingElementDelimiter='<CR><LF>'
        element name='Line' maxOccurs='1'
                Tag='CL99'


The 'RA50' tag was a guess. Change it to 'RA' if you need to.

Quote:

May be I am doing something wrong while creating the message set. but I tried to handle this delimiter in all the places but could not solve this problem.
The TDS parser writes a lot of information to user trace ( debug level ). Don't use the debugger to diagnose problems - it can only tell you *what* happened. A debug-level user trace will tell you *why* it happened.
Back to top
View user's profile Send private message
sushilishere
PostPosted: Mon Jun 21, 2010 1:28 am    Post subject: Reply with quote

Apprentice

Joined: 27 Jul 2008
Posts: 36

Tat is weird!!

Cld u place the structure of the msgset here..

Heres a suggestion:

Have just one complex element containing a single local element. keep the cardinality of le as 1 to -1.
Set the complex element to variable length delimited and the delimiter as <CR><LF>.

PArse the msg and let me know what happens.. It shd work!!!
_________________
----------------------------
IBM MQ v6.0 Sys Admin
IBM MBv6.0 Sol Desgnr
----------------------------------------------------
By the Mountain Side.. looking at the Great Divide
----------------------------------------------------
Back to top
View user's profile Send private message
sushilishere
PostPosted: Mon Jun 21, 2010 1:33 am    Post subject: Reply with quote

Apprentice

Joined: 27 Jul 2008
Posts: 36

After lookin at kimberts post, looks like u might have missed the repeating element delimiter trick.

this option is present for the local element( useful for the one which u want to repeat)
. In physical properties --> Final option. Set that value to <CR><LF> as well..

This option makes sure that this element repeat till this delimiter condition is met!!
_________________
----------------------------
IBM MQ v6.0 Sys Admin
IBM MBv6.0 Sol Desgnr
----------------------------------------------------
By the Mountain Side.. looking at the Great Divide
----------------------------------------------------
Back to top
View user's profile Send private message
swethabala
PostPosted: Mon Jun 21, 2010 1:52 am    Post subject: Reply with quote

Apprentice

Joined: 25 Aug 2009
Posts: 44

For now our server is down. I will try your seggestions and get back with the result.

Thanks for your support
Back to top
View user's profile Send private message
swethabala
PostPosted: Mon Jun 21, 2010 10:42 pm    Post subject: Reply with quote

Apprentice

Joined: 25 Aug 2009
Posts: 44

Thanks to you both

The delimiter problem is solved. I have re constructed the message set step by step as per the code given by you. on the first run only it worked.

Still the Footer problem is there. Footer is not getting recognized. All the other lines are read correctly but footer is not getting read. I have followed the same steps given by you. in the complex type of message I have checked the Observe element length tab. shouldn't this be checked?

this Footer is coming after the body. As body is repeating multiple times the footer is not getting recognized or I am not able to understand.

I tried with some changes to the element but footer is not getting read.


Could you give me one more suggestion about this.

Anyway we have solved one major problem of delimiter
Back to top
View user's profile Send private message
sushilishere
PostPosted: Mon Jun 21, 2010 11:10 pm    Post subject: Reply with quote

Apprentice

Joined: 27 Jul 2008
Posts: 36

Im not sure which one u followed!! .. elements delimited & Tags or pure all elements delimited.

Anyways as a suggestion:
Create a parent complex element to the you have created. Have 2 local elements. The first one referring to the existing complex element.(this guy shd not have any element for trailer record)

The second one is just for trailer record.

For the parent complex element, do the foll:
1. Type Variable length delimited
2. Delimiter <CR><LF>CL99

This should cut the trailer rec initially and then goes into the elements which parse the header and data!!!
_________________
----------------------------
IBM MQ v6.0 Sys Admin
IBM MBv6.0 Sol Desgnr
----------------------------------------------------
By the Mountain Side.. looking at the Great Divide
----------------------------------------------------
Back to top
View user's profile Send private message
kimbert
PostPosted: Tue Jun 22, 2010 12:10 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Quote:
Still the Footer problem is there. Footer is not getting recognized.
Oops. My model was not quite correct...
The approach suggested by sushilishere is a nice idea. Your model should look like this ( not tested):
Code:

element name='message '
    complexType DataElementSeparation='All Elements Delimited' GroupTerminator='<CR><LF>' Delimiter='<CR><LF>'
        element name='Header' maxOccurs='1'
            complexType GroupIndicator='OP01'
                <child elements of Header>
        sequence DataElementSeparation='All Elements Delimited' Delimiter='<CR><LF>CL99'
            element name='Body' maxOccurs='unbounded'  RepeatingElementDelimiter='<CR><LF>'
                complexType GroupIndicator='RA50'
                    <child elements of Body>
            element name='Line' maxOccurs='1'
                complexType GroupIndicator=''
                    <child elements of Footer>


Note that I have changed from using tags to using group indicators.
That is because Footer does not need a tag any more, so it is more consistent to model the tags as group indicators, and use 'All Elements Delimited' throughout the model.

Quote:
I have checked the Observe element length tab. shouldn't this be checked?
Observe Element Length is a detail - it will not affect the overall parsing of your input message. Use it when you want a delimited field to be padded to some minimum length.

sushilishere said:
Quote:
For the parent complex element, do the foll:
1. Type Variable length delimited
2. Delimiter <CR><LF>CL99

Do not use Variable Length Elements Delimited. You don't need it for this model. All Elements Delimited is the correct choice here, and for most other models where Variable Length Elements Delimited gets used.
Back to top
View user's profile Send private message
swethabala
PostPosted: Tue Jun 22, 2010 1:32 am    Post subject: Reply with quote

Apprentice

Joined: 25 Aug 2009
Posts: 44

Hey,

In this way it worked the Footer is coming as expected.

But one more problem I can see. As we have given RA as group indicator RA is removed from every line of the body and sent. So now the lines are looking like this:

501514481613753001832968001724828002B706PF 000000000009550103514481100401051043
501514629613755001832970001724828005B706PF 000000000009550103514629100402051043
501514638613754001832969001724828009B706PF 000000000009550103514638100406051043

When I remove the group indicator the Footer is included in the Body loop and coming as a last repetition of Body.

How can we solve this problem? We should get RA in the file after parsing. if we dont get it then the positioning will differ.
Back to top
View user's profile Send private message
sushilishere
PostPosted: Tue Jun 22, 2010 1:49 am    Post subject: Reply with quote

Apprentice

Joined: 27 Jul 2008
Posts: 36

That is bound to happen!!
Any values used in delimiter/group indicator etc will become a part of the msgset and will not appear as such in the elemental value.

If u map it to another msgset having those values as delimiters/grp indi etc, then the final output placed in file/Q/etc will have them as required.

@kimbert - I ttly agree on "All Elements Delimited is the correct choice here, and for most other models where Variable Length Elements Delimited gets used." . nth time im being reminded to do this!!

---------------

@swethabala - Looks like ur using some node (esql/mapping/jcn) after parsing the msg. If ur picking only the data record without any manipulation to it, u can use mapping node where the target msgset have the first element default to RA and map the other element with the source data record.. just a suggestion!!
_________________
----------------------------
IBM MQ v6.0 Sys Admin
IBM MBv6.0 Sol Desgnr
----------------------------------------------------
By the Mountain Side.. looking at the Great Divide
----------------------------------------------------
Back to top
View user's profile Send private message
swethabala
PostPosted: Tue Jun 22, 2010 1:56 am    Post subject: Reply with quote

Apprentice

Joined: 25 Aug 2009
Posts: 44

Yes Sushil; I will do that, orelse I will remove the group indicator and let the footer also read in Body only. while splitting and mapping the messages I will use Cardinality of Body -1 and the last one I will use as footer.

Anyway thanks for your help. this issue is almost closed now.
Back to top
View user's profile Send private message
kimbert
PostPosted: Tue Jun 22, 2010 5:00 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Quote:
orelse I will remove the group indicator and let the footer also read in Body only. while splitting and mapping the messages I will use Cardinality of Body -1 and the last one I will use as footer.
Here I go again...
Now that ESQL has the REFERENCE data type it is hardly ever a good idea to use CARDINALITY. Just move through the records using a reference. You need to treat the final record differently, so your loop will need to to look ahead to the next item - but that is still a lot more efficient than using CARDINALITY.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » How to handle /r/n delimiter in the message set
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.