| Author |
Message
|
| cwazpitt3 |
 Posted: Mon Oct 21, 2013 10:27 am Post subject: Creating a flow to poll an mqsiarchive directory |
 |
|
Acolyte
Joined: 31 Aug 2011
Posts: 61
|
I am considering doing what the title suggests, creating a flow that sits on top of an mqsiarchive directory so that we can separate our input files into a folder structure that makes them more accessible.
My question is two-fold:
1) Are there any best practices or guidelines from IBM that says this is a bad idea? I have not heard or read anything like this, but just wondering if the experts know something I don't.
2) Has anyone done this and if so, did you encounter any issues? I'm worried about the file being picked up by the second flow before the first flow is done...that kind of stuff. Any insight would be great!
Thanks in advance... |
|
| Back to top |
|
 |
| gs |
| Posted: Tue Oct 22, 2013 1:30 am Post subject: |
|
|
Master
Joined: 31 May 2007
Posts: 254
Location: Sweden
|
My understanding is that you've got a mqsiarchive folder for a common file input directory used by a number of flows(?).
1) If so, can you do the archiving in your existing flow(s) using a subflow instead? Since you already know the file content in those flows it'd be easier to separate them instead of having to analyze them once again in a new flow.
2) Since the file is only moved upon completion, I can't really see the risk. However, I leave it to someone with more experience from this to give input. |
|
| Back to top |
|
|
| Tibor |
| Posted: Tue Oct 22, 2013 3:14 am Post subject: Re: Creating a flow to poll an mqsiarchive directory |
|
|
Grand Master
Joined: 20 May 2001
Posts: 1033
Location: Hungary
|
| cwazpitt3 wrote: |
| 2) Has anyone done this and if so, did you encounter any issues? I'm worried about the file being picked up by the second flow before the first flow is done...that kind of stuff. Any insight would be great! |
Don't worry, while the file is under processing by a message flow, it is in the mqsitransit directory.
But I don't really understand, what the goal is a kind of message flow. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Oct 22, 2013 3:52 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Perhaps it is the following
1) Read the file and see how big it is
2) Pass this data to second flow so that it can read the file again and process it.
This is based upon some of the more recent File related postings that have appeared here.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| cwazpitt3 |
| Posted: Tue Oct 22, 2013 9:54 am Post subject: |
|
|
Acolyte
Joined: 31 Aug 2011
Posts: 61
|
| gs wrote: |
My understanding is that you've got a mqsiarchive folder for a common file input directory used by a number of flows(?).
1) If so, can you do the archiving in your existing flow(s) using a subflow instead? Since you already know the file content in those flows it'd be easier to separate them instead of having to analyze them once again in a new flow.
2) Since the file is only moved upon completion, I can't really see the risk. However, I leave it to someone with more experience from this to give input. |
This is a pretty good idea and one that I might persue. Thanks! |
|
| Back to top |
|
|
| cwazpitt3 |
| Posted: Tue Oct 22, 2013 9:57 am Post subject: Re: Creating a flow to poll an mqsiarchive directory |
|
|
Acolyte
Joined: 31 Aug 2011
Posts: 61
|
| Tibor wrote: |
| cwazpitt3 wrote: |
| 2) Has anyone done this and if so, did you encounter any issues? I'm worried about the file being picked up by the second flow before the first flow is done...that kind of stuff. Any insight would be great! |
Don't worry, while the file is under processing by a message flow, it is in the mqsitransit directory.
But I don't really understand, what the goal is a kind of message flow. |
Well, since you asked....
We have a flow that takes in a whole bunch of files from different apps across our org and does basic auditing and then routes to a destination based on an XML routing file that we maintain. Currently, the "incoming" file from all of the applications ends up in mqsiarchive. This is ok, but it was recently suggested that maybe it would be nice to automatically organize this folder by app. I just figured the quickest (maybe not best) way to acheive this was to write a flow on top of mqsiarchive and then route it to a specific app folder based on file name matches or something like that. Hopefully that makes sense. |
|
| Back to top |
|
|
| cwazpitt3 |
| Posted: Tue Oct 22, 2013 10:04 am Post subject: |
|
|
Acolyte
Joined: 31 Aug 2011
Posts: 61
|
| smdavies99 wrote: |
Perhaps it is the following
1) Read the file and see how big it is
2) Pass this data to second flow so that it can read the file again and process it.
This is based upon some of the more recent File related postings that have appeared here. |
Interesting idea...but would that even work? I mean, if you're trying to get under the file size limit restriction, my understanding is that MB would not even read the file if it were over 100MB (unless you go record delimited) so you wouldn't even be able to determine file size, right? Am I missing something? If this would work, it would be awesome  |
|
| Back to top |
|
|
| Tibor |
| Posted: Wed Oct 23, 2013 1:15 am Post subject: |
|
|
Grand Master
Joined: 20 May 2001
Posts: 1033
Location: Hungary
|
[quote="cwazpitt3"]
| smdavies99 wrote: |
| I mean, if you're trying to get under the file size limit restriction, my understanding is that MB would not even read the file if it were over 100MB (unless you go record delimited) so you wouldn't even be able to determine file size, right? Am I missing something? If this would work, it would be awesome |
You can increase the file size limit with environment variable MQSI_FILENODES_MAXIMUM_RECORD_LENGTH up to 2GB. Of course, you should be prepared for consuming large amount of memory by message broker. |
|
| Back to top |
|
|
|
|
