Author |
Message
|
bbakerman |
Posted: Wed Aug 10, 2005 10:14 pm Post subject: TDS field re-ordering - appends but does not insert |
|
|
Apprentice
Joined: 17 Dec 2003 Posts: 41
|
In the MRM we have a TDS definitions for a HL7 messages. I give the PV2 segment as an example (there are tens of more segements in HL7)
NOTE : The field naming convention is segment.fieldindex.fieldname but it might just be fieldname and most fields are optional.
The group "type" is Its defined as a "sequence" and we use Content Validation of "Closed". The group inidicators is "PV2" the delimeter is "|" and the group end indicator is <CR> and Supress Absent Element Delimiter is "End Of Message"
All fields are xsd:strings with no lengths.
So the TDS definition is :
=================
PV2
PV2.1.PriorPendingLocation
PV2.2.AccomodationCode
PV2.3.AdmitReason
PV2.4.TransferReason
PV2.5.PatientValuables
PV2.6.PatientValuablesLocation
...
...
PV2.10.EstimatedLengthofInpatientStay
...
...
PV2.16.PurgeStatusCode
PV2.17.PurgeStatusDate
...
PV2.24.PatienStatusCode
PV2.25.VisitPriorityCode
...
PV2.47.ExpectedLOAReturnDateTime
==============
If we have an input message like this :
PV2||SIN|^TEST||N^NC|||||0|||||||||||||^^2
Then we get a logical tree definition of
InputRoot
MRM
PV2
PV2.2.AccomodationCode = 'SIN'
PV2.3.AdmitReason = '^TEST'
PV2.5.PatientValuables = 'N^NC'
PV2.10.EstimatedLengthofInpatientStay = '0'
PV2.24.PatienStatusCode = '^^2'
So far so good. The TDS only shows fields that are present.
The problem occurs when you set a field that is not present on input but is logically before the "last present" field in the message. For example say
the business rule is that if TransferReason is not set, it must be default to a value of 'None'.
In our compute node or mapping node we do this
set OutputRoot = InputRoot.
if InputRoot.MRM.PV2."PV2.4.TransferReason" is null then
set OutputtRoot.MRM.PV2."PV2.4.TransferReason" = 'None';
end if
...
This ends up with a logcal message tree like this :
InputRoot
MRM
PV2
PV2.2.AccomodationCode = 'SIN'
PV2.3.AdmitReason = '^TEST'
PV2.5.PatientValuables = 'N^NC'
PV2.10.EstimatedLengthofInpatientStay = '0'
PV2.24.PatienStatusCode = '^^2'
PV2.4.TransferReason = 'None'
When this is written out to a message (via output parsing by the TDS) I get a messgae that looks like this :
PV2||SIN|^TEST||N^NC|||||0|||||||||||||^^2|None
The TDS parser is not smart enough to know field 4 should be "re-ordered" back in the fourth spot. It blindly whacks in on the end of the data.
For the record it does to the right thing for fields that are not there but are "greater" than the last encountered field. For example if you set the 47th field and its not present, then the TDS parser does put in the required amount of empty delimiters up the the 47th field. So thats reassuring.
What we are after is a way for the TDs parser to to one of this things:
A. Know how to re-order the field by name so we get the expected output.
B. "Expand" the whole message tree on input so we get all possible fields with "blanks" or "empty string" for fields that are not present.
Option A is the best option as it requires the least parsing and works inteligently. But maybe the TDs parser can do this for valid reasons (such as repeating sequence handling etc...)
Option B is the next best solution because fields cannot get out of order because thery are all present. Eg if you set field index 4 then it s there and hence in order still. The downside is that you get more memory usage than you otherwise would but is prevents bad output.
We have considered some HACKS but they all SMELL VERY BAD.
For example we could "know" what fields come before what field and do an create field as sibling but the sematics and coding of this are just AWFUL. And I dont want to have to put all the knowledge of the message structure in EQSL code. This is what the MRM is for.
I have played with default values and Encoding Null options to no avail. I cant seem to make the TDS parser "auto expand" the input message tree or have it "re-order" the messages tree elements on output.
Does anyone have some suggestions on how to approach this. |
|
Back to top |
|
 |
jefflowrey |
Posted: Thu Aug 11, 2005 3:18 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
There are some mechanisms for getting TDS to order fields for you the way they are in the message set. Try searching here for, believe it or not, "unordered".
But that may break the validation that you need on input... So you might need a different message definition for input vs output. _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
kimbert |
Posted: Thu Aug 11, 2005 4:22 am Post subject: |
|
|
 Jedi Council
Joined: 29 Jul 2003 Posts: 5542 Location: Southampton
|
Jeff's solution will work for CWF, but unfortunately not for TDS. The only solution is to put the elements in the correct order in the tree. In your case, you may be able to use the naming convention to help you get things in the right sequence (i.e. PV2.4 must come after PV2.3 and before PV2.5 )
This is known requirement, by the way. |
|
Back to top |
|
 |
bbakerman |
Posted: Thu Aug 11, 2005 10:11 pm Post subject: |
|
|
Apprentice
Joined: 17 Dec 2003 Posts: 41
|
kimbert wrote: |
This is known requirement, by the way. |
Thanks for the replies. What do you mean known requirement?
Do you mean its know you have to get the message tree in order or do you mean its know that a reordering of TDS messages is a requirement fo the next version of broker?
I have come up with what I think is an ingenious solution. The MRM .mxsd file in which the message is defined is after all a W3C schema. And the structure of such as well know and well defined.
So I have written a Java node that "reads" the .mxsd file, builds an in memory dictionary via javax.xml DOM calls and then "re-orders" the message as it passes by with respect the dictionary.
I do this in a very conservative manner in that I dont validate. eg missing fields can stay missing or unknown fields (according to the schema) are passed over.
But it re-cursively puts the message tree in order with respect to the schema. Therefore the TDS parse should "ouput" as we expect.
I havent done extensive testing yet so I will let you know how I go.
(I have done a lot of JUNIT tests with faky MtElement's based on XML files and its works okay) |
|
Back to top |
|
 |
kimbert |
Posted: Fri Aug 12, 2005 12:41 am Post subject: |
|
|
 Jedi Council
Joined: 29 Jul 2003 Posts: 5542 Location: Southampton
|
Quote: |
Do you mean its know you have to get the message tree in order or do you mean its know that a reordering of TDS messages is a requirement fo the next version of broker? |
I mean that IBM is aware that some users would like this capability. That does not imply that it will be in the next release of the broker. |
|
Back to top |
|
 |
fschofer |
Posted: Fri Aug 12, 2005 2:25 am Post subject: |
|
|
 Knight
Joined: 02 Jul 2001 Posts: 524 Location: Mainz, Germany
|
Hi,
you could add a 1:1 mapping which includes all fields and enables you to set default values or manipulate fields like
Code: |
SET OutputRoot.MRM.PV2.1.PriorPendingLocation = COALESCE(InputRoot.MRM.PV2.1.PriorPendingLocation, 'defvalue1');
SET OutputRoot.MRM.PV2.2.AccomodationCode = COALESCE(InputRoot.MRM.PV2.1.AccomodationCode, 'defvalue2');
...
PV2.3.AdmitReason
PV2.4.TransferReason
PV2.5.PatientValuables
PV2.6.PatientValuablesLocation
...
...
PV2.10.EstimatedLengthofInpatientStay
...
...
PV2.16.PurgeStatusCode
PV2.17.PurgeStatusDate
...
PV2.24.PatienStatusCode
PV2.25.VisitPriorityCode
...
PV2.47.ExpectedLOAReturnDateTime |
By the way for performance reasons it is better to use references instead of full path like "OutputRoot.MRM.PV2.2.AccomodationCode".
Greetings
Frank |
|
Back to top |
|
 |
bbakerman |
Posted: Sat Aug 13, 2005 5:29 pm Post subject: |
|
|
Apprentice
Joined: 17 Dec 2003 Posts: 41
|
Yes I considered doing the "default" values thing for every field but the problem is that HL7 has about 140 well defined segements with about 15 fields each.
So that about 2100 fields to be defaulted. I would rather have this "knowledge" in the dictionary (eg MRM) than in th code. |
|
Back to top |
|
 |
eaiuser |
Posted: Fri Sep 23, 2005 11:12 am Post subject: Dsipaly all the fields in HL7 Segment |
|
|
Newbie
Joined: 23 Sep 2005 Posts: 1
|
I have a more or less same requirement as required by Baker with Option2 with different direction of usage.
HL7 TDS only shows fields that are present.
We have a Repeating field at the end of the each segment to take of extra data that can come in the message.
But when this message which has extra information comes parsed tree provides the fields that are only having data and it is a problem.
TDS definition is :
=================
PV2
PV2.1.PriorPendingLocation
PV2.2.AccomodationCode
PV2.3.AdmitReason
PV2.4.TransferReason
PV2.5.PatientValuables
PV2.6.PatientValuablesLocation
...
...
PV2.10.EstimatedLengthofInpatientStay
...
...
PV2.16.PurgeStatusCode
PV2.17.PurgeStatusDate
...
PV2.24.PatienStatusCode
PV2.25.VisitPriorityCode
...
PV2.47.ExpectedLOAReturnDateTime
Remainder
RemField - Repeats 0 - 1000
==============
If we have an input message like this :
PV2||SIN|^TEST||N^NC|||||0|||||||||||||^^2 |Rem1||Rem4|||||Last
Then we get a logical tree definition of
InputRoot
MRM
PV2
PV2.2.AccomodationCode = 'SIN'
PV2.3.AdmitReason = '^TEST'
PV2.5.PatientValuables = 'N^NC'
PV2.10.EstimatedLengthofInpatientStay = '0'
PV2.24.PatienStatusCode = '^^2'
Remainder
RemField = Rem1
RemField = Rem1
RemField = Last
In the ESQL It is hard to take care of this data if we need to take care of if it requires to access the data in say 10th field of Remainder, but that field will not be in the same position all the time based on the incoming data.
I tried changing the RemField parameters to have a DEFAULT Value, Length etc and nothing works.
Does anybody have some suggestions on how to approach this???? |
|
Back to top |
|
 |
|