| Author |
Message
|
| jrsetters |
 Posted: Thu Nov 01, 2012 11:32 am Post subject: How best to go about using a file for reference data. |
 |
|
Acolyte
Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH
|
The scenario is this, only medical reports dictated by certain physicians are to be passed on to an external system. The list changes frequently. The physician's ID code will be part of the HL7 inbound message (MRM tree).
We are not exploring a database read as an option right now. We'd like to see if it is feasible to manage the list as a file that is read into the flow, then in a Filter node read through the list and filter the message out if a match is not made to the Physician ID code.
I was thinking along the lines of MQInput --> Read File (add as part of the message tree) --> Filter --> discard if a match is not made, or hand the original message off to another node for further processing. |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Nov 01, 2012 12:08 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
3 points immediately occur:
- Security on a file tends to be more granular and easier to manage in a database scenario
- Audit on a frequently changing list is much easier to achieve in a database scenario
- if the file's of any size it's going to cost a fortune in I/O to read it in every time there's a message
Depending on your definition of "frequently", you might also have issues trying to update it with the flow running.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Thu Nov 01, 2012 1:01 pm Post subject: Re: How best to go about using a file for reference data. |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| jrsetters wrote: |
| I was thinking along the lines of MQInput --> Read File (add as part of the message tree) --> Filter --> discard if a match is not made, or hand the original message off to another node for further processing. |
The FileRead node seems like an obvious choice. Combine that with some form of caching, to address at least some of the issues that Vitor raised. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Nov 01, 2012 1:08 pm Post subject: Re: How best to go about using a file for reference data. |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| rekarm01 wrote: |
| Combine that with some form of caching, to address at least some of the issues that Vitor raised. |
Apart from the audit, security and update issues I raised you mean. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Fri Nov 02, 2012 12:02 am Post subject: Re: How best to go about using a file for reference data. |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| Vitor wrote: |
| rekarm01 wrote: |
| Combine that with some form of caching, to address at least some of the issues that Vitor raised. |
Apart from the audit, security and update issues I raised you mean. |
Yes, apart from that. How to implement something, and whether to implement something are two entirely different issues ...  |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Nov 02, 2012 12:34 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
I would need to know the size of the data and the volume of lookups.
Caching may not be needed at all. If it is then you could consider using shared variables. You will have to decide when to refresh the cache from the file (maybe once a day).
There is a global cache feature in WMB 8. It's fairly basic though.
Using a database makes more sense, you can get the RDMS to cache the data if it's really intensively used (which I doubt). Or load it from the database to shared variables.
But it would need to be referenced several times a second to make it worth bothering about. |
|
| Back to top |
|
|
|
|
