ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » how to ignore rest of line in fixed length CWF?

Post new topic  Reply to topic
 how to ignore rest of line in fixed length CWF? « View previous topic :: View next topic » 
Author Message
pcelari
PostPosted: Fri Mar 27, 2009 4:33 am    Post subject: how to ignore rest of line in fixed length CWF? Reply with quote

Chevalier

Joined: 31 Mar 2006
Posts: 411
Location: New York

Hi,

Using a FileInput node, I read a file that contains one record in each line.

although the length of a line may vary, I'm only interested in reading the first 72 characters. So I defined a CWF msgset to parse the line.

It works fine if the line is exactly 72-char long. But whenever a line contains more than 72 character I got exception "MTI. Not all the buffer was used".

How should I define the msgset so only the first 72 char in a line get read and parsed, and the next read start from the next line instead of the rest of the remaining line?

thanks for any insight!


_________________
pcelari
-----------------------------------------
- a master of always being a newbie
Back to top
View user's profile Send private message
WMBDEV1
PostPosted: Fri Mar 27, 2009 5:01 am    Post subject: Reply with quote

Sentinel

Joined: 05 Mar 2009
Posts: 888
Location: UK

You could have an extra element on the end of the line, with an unlimited length and then just ignore it
Back to top
View user's profile Send private message
kimbert
PostPosted: Fri Mar 27, 2009 5:55 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

You need TDS, not CWF. CWF cannot parse delimited data.
CWF would be able to handle a single record ( single line ) via Length Units = EndOfBitstream, but you have a file full of these things.
Back to top
View user's profile Send private message
mqseries0209
PostPosted: Fri Mar 27, 2009 6:13 am    Post subject: Reply with quote

Voyager

Joined: 30 Mar 2006
Posts: 90

I would have handled it like this, not sure if there is any better solution:

On FileInput node:

"Input Message Parsing" as BLOB

"Records and elemenst" select "Record detection" as delimited and delimiter as DOS OR UNIX line feed.

with the above configuration, each line in input file will be avaiable as BLOB in message body at output terminal.

Conenct a compute node and use SUBSTRING function to extarct 72 characters from the BLOB message and also you can parse the extracted message body to what ever the format/domain u need.

Hope this helps.
Peace.
_________________
IBM Certified Solution Developer - WebSphere Message Broker V6.1

IBM Certified Solution Developer - WebSphere Integration Developer V6.0
Back to top
View user's profile Send private message
pcelari
PostPosted: Fri Mar 27, 2009 7:23 am    Post subject: Reply with quote

Chevalier

Joined: 31 Mar 2006
Posts: 411
Location: New York

Thank you for the suggestion. Using several nodes can certainly accomplish this.

But this simple need in flexibility in input msg length should be handled pretty easily within message set.

Can an expert from Hersley please give some input?
_________________
pcelari
-----------------------------------------
- a master of always being a newbie
Back to top
View user's profile Send private message
kimbert
PostPosted: Fri Mar 27, 2009 7:32 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Quote:
Can an expert from Hersley please give some input?
I did. Do you have a follow-up question?
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Mar 27, 2009 7:41 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Let's go back to basics here.

You can not model a partial document in MRM at all. An MRM model must match the entire bitstream you give it.

There is no particular way to say "I only want to apply this model to the first X bytes of a stream". Your particular situation is even worse, as you really want to say "I only want to apply the model to a repeating subsection of the bitstream".

WMBDEV1's suggestion that you model the first 72 bytes of a record as whatever it needs to be, and then include another element that is variable length and consumes the remainder is the only workable solution. If your *output* message needs to only include the first 72 bytes, you can create a second message definition that doesn't include that last element and do a *very* simple mapping.

You need to use the linefeed as a delimiter here to indicate that the model should start over at the first 72 character sections again.
Back to top
View user's profile Send private message
kimbert
PostPosted: Fri Mar 27, 2009 7:46 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

The FileInput node has an option 'Parsed Record Sequence' which is intended for exactly this scenario. If the MRM message definition is designed to match one record ( one line ) then it will propagate a sequence of messages to the flow until the file is exhausted.
As I said earlier, TDS can handle pcelari's file format. CWF cannot.
Back to top
View user's profile Send private message
pcelari
PostPosted: Fri Mar 27, 2009 7:52 am    Post subject: Reply with quote

Chevalier

Joined: 31 Mar 2006
Posts: 411
Location: New York

Given that my msgset is CWF, and the difficulty is only with reading the last element in a line,

is it possible to truncate any character beyond the stated length of the last element?

I cannot add a dummy extra element as WMBDEV1 suggested, just to eat up the extra bytes, as the extra bytes don't exist most of the time.

Or, is it possible to let the last element to read until EOL and then truncate it to the desired length in a compute code?
_________________
pcelari
-----------------------------------------
- a master of always being a newbie
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Mar 27, 2009 8:00 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

You can't use CWF for this. As kimbert keeps saying, and as you yourself agree, it can't handle the last element.

You *can* add a dummy element. You just have to make it optional and variable length. Using TDS will let you do this.

Regardless of whether you use the FileInput Parsed Record Sequence handling to separate your data into multiple messages or not, you still need to model a full record, which includes an optional variable length element that is terminated by the line ending.
Back to top
View user's profile Send private message
kimbert
PostPosted: Fri Mar 27, 2009 8:06 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Quote:
Given that my msgset is CWF
Well, it is for now. You may need to revisit that design decision.
Quote:
is it possible to truncate any character beyond the stated length of the last element?
truncate up to where? End of line, or end of file? If 'end of line', then please explain how the CWF parser will know that the CR/LF is supposed to represent the end of your record.
Quote:
I cannot add a dummy extra element as WMBDEV1 suggested, just to eat up the extra bytes, as the extra bytes don't exist most of the time
You can if you use TDS. That final field is terminated by CR/LF. If you are using TDS, you can set CR/LF as the Group Terminator. If the CRLF is encountered at byte 73/74 you will simply get a zero-length field.

You now have to make a choice.
- stay with CWF, use mqseries0209's suggestion, and accept that you will need multiple nodes.
- switch your message set to TDS, and maintain the elegance of the solution.

If you decide on TDS, I recommend that you use 'Variable Length Elements Delimited'. Set the length on all fields in the first 72 bytes. Do not set the length of the optional final field. Set the Group Terminator to "<CR><LF>"
Back to top
View user's profile Send private message
pcelari
PostPosted: Fri Mar 27, 2009 10:37 am    Post subject: Reply with quote

Chevalier

Joined: 31 Mar 2006
Posts: 411
Location: New York

Wooow! many thanks for all the expert insights here.

I was responding to a previous one while more generous tips were already posted.

I'll try out all these.

Since experience has always shown that tinkering with message set always took a lot more time than adding more nodes, at least for me. I'll first go with mqseries0209's route.

I'll post a summary as a report back to all your suggestions.

Again, many many thanks!
_________________
pcelari
-----------------------------------------
- a master of always being a newbie
Back to top
View user's profile Send private message
francoisvdm
PostPosted: Thu Jun 04, 2009 11:52 pm    Post subject: Reply with quote

Partisan

Joined: 09 Aug 2001
Posts: 332

summary?
_________________
If you do not know the answer or you get the urge to answer with "RTFM" or "Search better in this forum", please refrain from doing so, just move on to the next question. Much appreciated.

Francois van der Merwe
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » how to ignore rest of line in fixed length CWF?
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.