| Author |
Message
|
| taankyu |
 Posted: Tue Dec 02, 2014 6:05 am Post subject: Ordering of multiple messages or parsing in reverse order |
 |
|
Novice
Joined: 15 Nov 2014
Posts: 20
|
Need some help on this :
I have flat file with multi row records -
ROW1
ROW2
ROW1
ROW2...
I am parsing the file in compute node and sending ROW1 as one message and ROW2 as different message.
But , I need to send message for ROW2 followed by ROW1 - need help on this
Or is it possible to parse the file from bottom to top. |
|
| Back to top |
|
 |
| zpat |
| Posted: Tue Dec 02, 2014 6:35 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
If the file will never be too large, you could make the file node read it as a single object (record detection = whole file).
Then you can step through the data in ESQL - looking for record delimiters (if it's not fixed length) and put each record into an ESQL storage array.
Then you can process it in whatever order you want and issue propagate from the compute note to a MQoutput node for each message.
Don't forget to set up your MQMD properly (format = MQSTR, CCSID, persistent or whatever).
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Dec 02, 2014 7:00 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| You could create a message model that includes both RECORD1 and RECORD2 in a single record, and then parse the file based on the record. Then you would propagate each set as a message, and could further propagate or process each record as you felt. |
|
| Back to top |
|
|
| taankyu |
| Posted: Tue Dec 02, 2014 10:16 am Post subject: |
|
|
Novice
Joined: 15 Nov 2014
Posts: 20
|
Unfortunately the files are of large size ... is there some way we can parse the file in reverse order - bottom to top
I tried with message model but the log file has many elements in it which are not well defined and combining the two rows as one record was causing issues |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Dec 02, 2014 10:26 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| taankyu wrote: |
| Unfortunately the files are of large size ... is there some way we can parse the file in reverse order - bottom to top |
No.
| taankyu wrote: |
| I tried with message model but the log file has many elements in it which are not well defined and combining the two rows as one record was causing issues |
If the elements are not well defined, then any message model will have issues. No matter which order you're parsing the file.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Dec 02, 2014 10:27 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
None of the File nodes in Broker support reading from the file in "reverse" order.
Perhaps you could consider not writing to a file in the first place, and using something like a database instead. |
|
| Back to top |
|
|
| taankyu |
| Posted: Tue Dec 02, 2014 10:38 am Post subject: |
|
|
Novice
Joined: 15 Nov 2014
Posts: 20
|
Thanks a lot everyone ...
We get the log files from external sources [Not much control over that]
Just able to parse the required fields using FILLERS and adjustments.
Somehow , struggling in ordering of messages - ROW2 before ROW1 |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Dec 02, 2014 10:49 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
The File nodes require that you read records from the start of the file to the end of the file.
You then have all of the capabilities of Broker at your disposal to do whatever you want with those records, for example put them somewhere else for processing in a different order.
It's otherwise relatively easy to reverse the contents of a file using OS tools or a simple script. |
|
| Back to top |
|
|
| taankyu |
| Posted: Tue Dec 02, 2014 12:06 pm Post subject: |
|
|
Novice
Joined: 15 Nov 2014
Posts: 20
|
| Thanks..able to parse the required information after reversing the file |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Dec 02, 2014 11:14 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Define "large" in a more scientific way. How large?
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| maurito |
| Posted: Wed Dec 03, 2014 12:49 am Post subject: |
|
|
Partisan
Joined: 17 Apr 2014
Posts: 358
|
1- set the file input node to read one record at a time.
2- if it is ROW1 put it in SHARED
3- if is it ROW2 write it, and then write whatever is in shared, reinitialize shared.
etc.
Do not use additional instances. |
|
| Back to top |
|
|
|
|
