| Author |
Message
|
| scravr |
 Posted: Fri Apr 27, 2012 11:36 am Post subject: Fix/Variable Record Length - FileRead |
 |
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
I have a directory with multiple files to read via File Read.
Files can be fix or variable record length.
How should I configure File Read to handle both types? |
|
| Back to top |
|
 |
| Vitor |
| Posted: Fri Apr 27, 2012 1:47 pm Post subject: Re: Fix/Variable Record Length - FileRead |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| scravr wrote: |
| How should I configure File Read to handle both types? |
By using a message set that encompasses both types.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sat Apr 28, 2012 5:43 am Post subject: Re: Fix/Variable Record Length - FileRead |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Vitor wrote: |
| scravr wrote: |
| How should I configure File Read to handle both types? |
By using a message set that encompasses both types. |
Or by having some way of telling which file is which type and then using programming logic to select the correct message set. |
|
| Back to top |
|
|
| scravr |
| Posted: Sun Apr 29, 2012 10:59 am Post subject: |
|
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
this directory gets lost of diff files with diff records (header, detail-type1, detail-type2... trailer) each file can be fix or variable length.
How do i use msg-set?
Need more details .... |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Sun Apr 29, 2012 12:05 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
How about
Read the message as a BLOB
Look for the differente headers in the BLOB
Branch to appropriate code that parses the record with the correct params.
I use this all the time on one system to discard the heartbeat messages.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Apr 30, 2012 4:20 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| smdavies99 wrote: |
How about
Read the message as a BLOB
Look for the differente headers in the BLOB
Branch to appropriate code that parses the record with the correct params.
I use this all the time on one system to discard the heartbeat messages. |
And this is more than enough for you to solve the problem.
If you can't solve the problem with this information, you need to stop everything, and spend a lot of time studying and thinking and trying things out.
it's a mysterious process known as 'learning'. |
|
| Back to top |
|
|
| scravr |
| Posted: Mon Apr 30, 2012 9:50 am Post subject: |
|
|
Partisan
Joined: 03 Apr 2003
Posts: 391
Location: NY NY USA 10021
|
You all missing the point…
Flow as MQInput, few Compute Nodes, a FileRead Node, few more Compute Nodes, and MQOutput.
FileRead Node needs to read a file named on input XML message.
Files are coming from 3rd party (Linux and/or MVS).
Files may be in Fix-Length or Variable-Length.
Files have numerous message formats (Header-Type-1, Header-Type-2,….. Detail-Type-1, Detail-Type-2….. Trailer-Type-1, Trailer-Type-2….)
I have no plans to build and maintain all these numerous msg-sets.
How can I configure single File-Read to be able to read fix and/or variable length record files? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Apr 30, 2012 9:58 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| scravr wrote: |
| I have no plans to build and maintain all these numerous msg-sets. |
Okay.
The original suggestion was to create one message set that held the definitions of more than one type of file, and use message set configuration to resolve the choice.
That doesn't work for a generic set of widely different files.
How are you planning on storing the information about how to access the contents of the fields in the messages?
What do you need to *do* with the contents of the file? If you need to *understand* it in any way, you need to *parse* it.
| scravr wrote: |
| How can I configure single File-Read to be able to read fix and/or variable length record files? |
You configure it to read the file using the BLOB parser.
Then you get exactly one record that consists of the entire contents of the file.
But you don't have any understanding of the contents of the file at that point. You just have a stream of bytes. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Apr 30, 2012 10:15 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| scravr wrote: |
How can I configure single File-Read to be able to read fix and/or variable length record files? |
It seems that you want the whole Earth but you are only willing to pay for a single blade of grass.
Sometimes you have to bite the bullet and do exactly what you don't want to do.
You could combine many of the message types into one message set but use different messages.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Apr 30, 2012 11:39 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| You all missing the point… |
I wonder how we *all* managed to do that? After you clearly said in your first post "How should I configure File Read to handle both types?"
I completely agree with mqjeff and smdavies99 - you need to understand your data, understand what your flow needs to do with it, then use your professional knowledge and experience to design or discover a solution. |
|
| Back to top |
|
|
| Esa |
| Posted: Tue May 01, 2012 11:14 pm Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| scravr wrote: |
How can I configure single File-Read to be able to read fix and/or variable length record files? |
On the Records and Elements tab.
You are obviously not going to process the files, just move them somewhere else? If the files are not large, you can use the option 'Whole File', otherwise 'Fixed Length' is a better option.
I mean, if you are not going to parse the messages, you don't have to care if the records are of fixed length or not.
| scavr wrote: |
| I have no plans to build and maintain all these numerous msg-sets. |
You don't have to. Unless you are not just moving the files, but need to access the contents. Then you will have to change your plans...
How you configure 'Fixed Length' depends very much on what you need to to with the file. If you write it in another folder with FileOutput node, you wire FileRead.Out to FileOutput.In and 'FileRead.Finish file out' to 'FileOutput.Finish file'. If you are putting it in an MQ Queue, you need to assemble the message with a Collector node (for example), or use a compute or javaCompute node to put it out as a segmented message. For example.
Or perhaps I'm missing the point, too. |
|
| Back to top |
|
|
|
|
