ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » TDS field re-ordering - appends but does not insert

Post new topic  Reply to topic
 TDS field re-ordering - appends but does not insert « View previous topic :: View next topic » 
Author Message
bbakerman
PostPosted: Wed Aug 10, 2005 10:14 pm    Post subject: TDS field re-ordering - appends but does not insert Reply with quote

Apprentice

Joined: 17 Dec 2003
Posts: 41

In the MRM we have a TDS definitions for a HL7 messages. I give the PV2 segment as an example (there are tens of more segements in HL7)

NOTE : The field naming convention is segment.fieldindex.fieldname but it might just be fieldname and most fields are optional.

The group "type" is Its defined as a "sequence" and we use Content Validation of "Closed". The group inidicators is "PV2" the delimeter is "|" and the group end indicator is <CR> and Supress Absent Element Delimiter is "End Of Message"

All fields are xsd:strings with no lengths.

So the TDS definition is :
=================
PV2
PV2.1.PriorPendingLocation
PV2.2.AccomodationCode
PV2.3.AdmitReason
PV2.4.TransferReason
PV2.5.PatientValuables
PV2.6.PatientValuablesLocation
...
...
PV2.10.EstimatedLengthofInpatientStay
...
...
PV2.16.PurgeStatusCode
PV2.17.PurgeStatusDate
...
PV2.24.PatienStatusCode
PV2.25.VisitPriorityCode
...
PV2.47.ExpectedLOAReturnDateTime
==============



If we have an input message like this :

PV2||SIN|^TEST||N^NC|||||0|||||||||||||^^2

Then we get a logical tree definition of

InputRoot
MRM
PV2
PV2.2.AccomodationCode = 'SIN'
PV2.3.AdmitReason = '^TEST'
PV2.5.PatientValuables = 'N^NC'
PV2.10.EstimatedLengthofInpatientStay = '0'
PV2.24.PatienStatusCode = '^^2'

So far so good. The TDS only shows fields that are present.

The problem occurs when you set a field that is not present on input but is logically before the "last present" field in the message. For example say
the business rule is that if TransferReason is not set, it must be default to a value of 'None'.

In our compute node or mapping node we do this

set OutputRoot = InputRoot.
if InputRoot.MRM.PV2."PV2.4.TransferReason" is null then
set OutputtRoot.MRM.PV2."PV2.4.TransferReason" = 'None';
end if
...

This ends up with a logcal message tree like this :

InputRoot
MRM
PV2
PV2.2.AccomodationCode = 'SIN'
PV2.3.AdmitReason = '^TEST'
PV2.5.PatientValuables = 'N^NC'
PV2.10.EstimatedLengthofInpatientStay = '0'
PV2.24.PatienStatusCode = '^^2'
PV2.4.TransferReason = 'None'


When this is written out to a message (via output parsing by the TDS) I get a messgae that looks like this :

PV2||SIN|^TEST||N^NC|||||0|||||||||||||^^2|None


The TDS parser is not smart enough to know field 4 should be "re-ordered" back in the fourth spot. It blindly whacks in on the end of the data.

For the record it does to the right thing for fields that are not there but are "greater" than the last encountered field. For example if you set the 47th field and its not present, then the TDS parser does put in the required amount of empty delimiters up the the 47th field. So thats reassuring.

What we are after is a way for the TDs parser to to one of this things:

A. Know how to re-order the field by name so we get the expected output.

B. "Expand" the whole message tree on input so we get all possible fields with "blanks" or "empty string" for fields that are not present.

Option A is the best option as it requires the least parsing and works inteligently. But maybe the TDs parser can do this for valid reasons (such as repeating sequence handling etc...)

Option B is the next best solution because fields cannot get out of order because thery are all present. Eg if you set field index 4 then it s there and hence in order still. The downside is that you get more memory usage than you otherwise would but is prevents bad output.


We have considered some HACKS but they all SMELL VERY BAD.

For example we could "know" what fields come before what field and do an create field as sibling but the sematics and coding of this are just AWFUL. And I dont want to have to put all the knowledge of the message structure in EQSL code. This is what the MRM is for.

I have played with default values and Encoding Null options to no avail. I cant seem to make the TDS parser "auto expand" the input message tree or have it "re-order" the messages tree elements on output.

Does anyone have some suggestions on how to approach this.
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Thu Aug 11, 2005 3:18 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

There are some mechanisms for getting TDS to order fields for you the way they are in the message set. Try searching here for, believe it or not, "unordered".

But that may break the validation that you need on input... So you might need a different message definition for input vs output.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
kimbert
PostPosted: Thu Aug 11, 2005 4:22 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Jeff's solution will work for CWF, but unfortunately not for TDS. The only solution is to put the elements in the correct order in the tree. In your case, you may be able to use the naming convention to help you get things in the right sequence (i.e. PV2.4 must come after PV2.3 and before PV2.5 )

This is known requirement, by the way.
Back to top
View user's profile Send private message
bbakerman
PostPosted: Thu Aug 11, 2005 10:11 pm    Post subject: Reply with quote

Apprentice

Joined: 17 Dec 2003
Posts: 41

kimbert wrote:
This is known requirement, by the way.


Thanks for the replies. What do you mean known requirement?

Do you mean its know you have to get the message tree in order or do you mean its know that a reordering of TDS messages is a requirement fo the next version of broker?

I have come up with what I think is an ingenious solution. The MRM .mxsd file in which the message is defined is after all a W3C schema. And the structure of such as well know and well defined.

So I have written a Java node that "reads" the .mxsd file, builds an in memory dictionary via javax.xml DOM calls and then "re-orders" the message as it passes by with respect the dictionary.

I do this in a very conservative manner in that I dont validate. eg missing fields can stay missing or unknown fields (according to the schema) are passed over.

But it re-cursively puts the message tree in order with respect to the schema. Therefore the TDS parse should "ouput" as we expect.

I havent done extensive testing yet so I will let you know how I go.

(I have done a lot of JUNIT tests with faky MtElement's based on XML files and its works okay)
Back to top
View user's profile Send private message
kimbert
PostPosted: Fri Aug 12, 2005 12:41 am    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Quote:
Do you mean its know you have to get the message tree in order or do you mean its know that a reordering of TDS messages is a requirement fo the next version of broker?
I mean that IBM is aware that some users would like this capability. That does not imply that it will be in the next release of the broker.
Back to top
View user's profile Send private message
fschofer
PostPosted: Fri Aug 12, 2005 2:25 am    Post subject: Reply with quote

Knight

Joined: 02 Jul 2001
Posts: 524
Location: Mainz, Germany

Hi,

you could add a 1:1 mapping which includes all fields and enables you to set default values or manipulate fields like

Code:
SET OutputRoot.MRM.PV2.1.PriorPendingLocation = COALESCE(InputRoot.MRM.PV2.1.PriorPendingLocation, 'defvalue1');
SET OutputRoot.MRM.PV2.2.AccomodationCode = COALESCE(InputRoot.MRM.PV2.1.AccomodationCode, 'defvalue2');
...
PV2.3.AdmitReason
PV2.4.TransferReason
PV2.5.PatientValuables
PV2.6.PatientValuablesLocation
...
...
PV2.10.EstimatedLengthofInpatientStay
...
...
PV2.16.PurgeStatusCode
PV2.17.PurgeStatusDate
...
PV2.24.PatienStatusCode
PV2.25.VisitPriorityCode
...
PV2.47.ExpectedLOAReturnDateTime


By the way for performance reasons it is better to use references instead of full path like "OutputRoot.MRM.PV2.2.AccomodationCode".

Greetings
Frank
Back to top
View user's profile Send private message Send e-mail
bbakerman
PostPosted: Sat Aug 13, 2005 5:29 pm    Post subject: Reply with quote

Apprentice

Joined: 17 Dec 2003
Posts: 41

Yes I considered doing the "default" values thing for every field but the problem is that HL7 has about 140 well defined segements with about 15 fields each.

So that about 2100 fields to be defaulted. I would rather have this "knowledge" in the dictionary (eg MRM) than in th code.
Back to top
View user's profile Send private message
eaiuser
PostPosted: Fri Sep 23, 2005 11:12 am    Post subject: Dsipaly all the fields in HL7 Segment Reply with quote

Newbie

Joined: 23 Sep 2005
Posts: 1

I have a more or less same requirement as required by Baker with Option2 with different direction of usage.

HL7 TDS only shows fields that are present.

We have a Repeating field at the end of the each segment to take of extra data that can come in the message.

But when this message which has extra information comes parsed tree provides the fields that are only having data and it is a problem.

TDS definition is :
=================
PV2
PV2.1.PriorPendingLocation
PV2.2.AccomodationCode
PV2.3.AdmitReason
PV2.4.TransferReason
PV2.5.PatientValuables
PV2.6.PatientValuablesLocation
...
...
PV2.10.EstimatedLengthofInpatientStay
...
...
PV2.16.PurgeStatusCode
PV2.17.PurgeStatusDate
...
PV2.24.PatienStatusCode
PV2.25.VisitPriorityCode
...
PV2.47.ExpectedLOAReturnDateTime
Remainder
RemField - Repeats 0 - 1000

==============

If we have an input message like this :

PV2||SIN|^TEST||N^NC|||||0|||||||||||||^^2 |Rem1||Rem4|||||Last

Then we get a logical tree definition of

InputRoot
MRM
PV2
PV2.2.AccomodationCode = 'SIN'
PV2.3.AdmitReason = '^TEST'
PV2.5.PatientValuables = 'N^NC'
PV2.10.EstimatedLengthofInpatientStay = '0'
PV2.24.PatienStatusCode = '^^2'
Remainder
RemField = Rem1
RemField = Rem1
RemField = Last



In the ESQL It is hard to take care of this data if we need to take care of if it requires to access the data in say 10th field of Remainder, but that field will not be in the same position all the time based on the incoming data.

I tried changing the RemField parameters to have a DEFAULT Value, Length etc and nothing works.

Does anybody have some suggestions on how to approach this????
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » TDS field re-ordering - appends but does not insert
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.