| Author |
Message
|
| hal |
 Posted: Thu Nov 15, 2012 12:50 pm Post subject: FileInput node duplicate file detection |
 |
|
Acolyte
Joined: 07 Dec 2005
Posts: 67
Location: New York City, New York
|
I am using WebSphere Message Broker 8.0.0.1 FileInput nodes. Is there a way to prevent duplicate files from being processed without using a database?
This question was asked more than four and a half years ago for WMB 6.1 at http://www.mqseries.net/phpBB2/viewtopic.php?t=42765 Have there been any FileInput node duplicate-file-detection enhancements or non-database workarounds since then? WebSphere MQ FTE is also not an option for my company. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Nov 15, 2012 1:56 pm Post subject: Re: FileInput node duplicate file detection |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
The question remains "define duplicates".
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| hal |
| Posted: Thu Nov 15, 2012 5:41 pm Post subject: |
|
|
Acolyte
Joined: 07 Dec 2005
Posts: 67
Location: New York City, New York
|
By duplicates I mean duplicate file names. For now I am not concerned with preventing duplicate records contained within differently named files from being processed. What I am really looking for is a FileInput node "Action before processing" selection similar to the "Action on successful processing" selection.
One brute force approach is to write a shell script wrapper around the file sending application. The script would check if a specific file name exists anywhere within the FileInput node directory tree. If the file name is already there, the script would flag an error and not resend the duplicate file name. Either the FileInput node "Move to Archive Subdirectory" or "Add Timestamp and Move to Archive Subdirectory" option without the "Replace duplicate archive files" selection is needed for this to work.
I neither designed nor wrote the file sending and receiving applications. Nevertheless, I have a new fourth-quarter-two-minute-warning-before-production-deployment requirement from the file sending application owners for "Middleware" to prevent duplicate file names. In a perfect world the sending and receiving applications would be written with exception handlers and duplicate record checking mechanisms. |
|
| Back to top |
|
|
| Esa |
| Posted: Fri Nov 16, 2012 12:49 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
You can write a preprocessor flow that adds a timestamp in the file name and writes it in the directory your business flow is listening to. That should make the filenames unique 
Why is the application so sensitive about file names? Does the file name reflect a specific business object instance that should not be sent twice?
The possible solutions to your problem depend on the amount of files that are being sent and collected to the archive directories.
And what is the timescale for the uniqueness? A week, a year or from the Big Bang to the next one?
Can you query the receiving application if it already has a specific business object? |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Nov 16, 2012 1:18 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| The use of files is the root of all evil. |
|
| Back to top |
|
|
| Esa |
| Posted: Fri Nov 16, 2012 9:14 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
By the way, your case would have been a perfect business case for complex event handling nodes - although they do use a database under the hoods. But the support pack seems to have disappeared. It has not been incorporated in the product (AFAIK). What happened to it? The only customer using it chose another solution?
Not that I'm missing it, just curious. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Nov 16, 2012 9:25 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Complex events got replaced by Business Events. |
|
| Back to top |
|
|
| hal |
| Posted: Fri Nov 16, 2012 2:35 pm Post subject: |
|
|
Acolyte
Joined: 07 Dec 2005
Posts: 67
Location: New York City, New York
|
Does anyone know the back story of SupportPac IA03 and why it was unbundled from WebSphere Message Broker 6.1?
In early 2008 I started to evaluate IA03, WebSphere Message Broker CEP Detector Nodes, and still have versions 6.0.0.1 and 6.1. When it mysteriously vanished from WebSphere Message Broker 7, I contacted IBM Hursley. They asked me to speak to my IBM rep who directed me to an IBM salesperson who explained that I needed to purchase a separate Business Events product.
From http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg27015418&loc=en_US&cs=utf-8&lang=en
| Quote: |
Delivery Instructions for SupportPac IA0S: WebSphere Message Broker - CEP Detector Nodes
.Release notes
AbstractDelivery Instructions
ContentThis Service Offering is available from the IBM Research Laboratories team.
To obtain the WebSphere Message Broker CEP Detector Nodes v6.0 or v6.1 code, please contact your IBM Rep or send an email to csi@us.ibm.com. |
Hal |
|
| Back to top |
|
|
| Vitor |
| Posted: Sat Nov 17, 2012 5:58 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| hal wrote: |
| Does anyone know the back story of SupportPac IA03 and why it was unbundled from WebSphere Message Broker 6.1? |
| mqjeff wrote: |
| Complex events got replaced by Business Events |
That's about as good as you're likely to get in a public forum.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
