MQSeries.net :: View topic - Manipulating the BLOB using REPLACE

MQSeries.net

Tech Exchange

Education

Certifications

Library

Info Center

SupportPacs

FAQÂ Â

Usergroups

RSS Feed - WebSphere MQ Support

RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Manipulating the BLOB using REPLACE

Manipulating the BLOB using REPLACE

« View previous topic :: View next topic »

Author

Message

jrsetters

Posted: Wed Dec 19, 2012 8:25 am Post subject: Manipulating the BLOB using REPLACE

Acolyte

Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH

One of our sending systems sends characters that the HL7 message set has difficulty interpreting when it is converted tot he MRM domain.

In HL7 the tilde (~) character is used to designate a repeating field, however the MQ HL7 Message set will fail to process this correctly if an empty repetition is received followed by another repetition character. What happens is that it will push the second repetition into the next field if it exists, or it is completely empty the message will fail.

Here is an example:

1) FIELD1:FIELD2:FIELD3(1)~FIELD3(2):FIELD4 is correct
2) FIELD1:FIELD2:FIELD3(1)~~FIELD3(2):FIELD4 will move FIELD3(2) into FIELD4 and FIELD4 into FIELD5

To correct this, I thought the easiest way would be to just loop through the BLOB before we reset the content to MRM and do a replace on ~~ with ~.

This character will be represented by HEX 7e in the BLOB.

I am having a bit of trouble with my code to do this. I have tried to use a REPLACE, but it doesn't seem to work.

Code:

DECLARE tmp BLOB;

   SET tmp = InputRoot.BLOB.BLOB;
   SET tmp = REPLACE (tmp, '7e7e', '7e');

This code will throw an 'unmatched data type' error.

So I switched to using the X designation for HEX:

Code:

SET tmp = InputRoot.BLOB.BLOB;
SET tmp = REPLACE (tmp, X'7e7e', X'7e');

But this just ignores the bytes I want to change altogether.

Is this even the right track to be on for what I want to accomplish?

McueMart

Posted: Wed Dec 19, 2012 8:48 am Post subject:

Chevalier

Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere

I would say that this should work! Have you tried:

Code:

REPLACE (tmp, X'7E7E', X'7E');

longshot...

kimbert

Posted: Wed Dec 19, 2012 8:50 am Post subject:

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Before anybody else points this out, the MRM parser should be doing a better job here. I think I know why the problem is occurring - the MRM TDS parser has a special rule that two consecutive delimiters terminate a sequence of repeating elements. I expect that the message set is tripping over that rule.
At this point, to avoid disappointing mqjeff, I will mention that the DFDL parser ( available in v8 ) does not have any such rule, and will be able to parser HL7 messages without any such problems.
.

Quote:

To correct this, I thought the easiest way would be to just loop through the BLOB before we reset the content to MRM and do a replace on ~~ with ~.

I assume that you know this already...but
a) the repeat delimiter can be overridden by the MSH.2 field. To be safe, you should check the contents of that field ( by properly parsing the message, not by guessing the position ) and then do the replacement using whatever you find in MSH.2.
b) if the input message contains three or more consecutive delimiters ( ~~~ ) then you will still be left with a bitstream that will not parse.

Your first use of REPLACE was clearly wrong - the info centre clearly states that you cannot mix CHARACTER and BLOB types in the same call to REPLACE. Your second usage looks correct to me - maybe somebody else can spot the problem?

jrsetters

Posted: Wed Dec 19, 2012 8:59 am Post subject:

Acolyte

Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH

Thank you. I am wondering if the problem is with the way it reads the X'7e7e' part, since I am trying to use two HEX bytes at once.

I agree on parsing out the repetition character from the message header first, that is an important thing to note.

I was also contemplating what would happen if there were more than one empty string, so I figured once I get the REPLACE to work, I would put the statement in a loop that keeps looking for instances of '~~' until it can't find any more.

I will try using upper case as mentioned above as well.

jrsetters

Posted: Wed Dec 19, 2012 9:06 am Post subject:

Acolyte

Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH

Well now is the part where I get to feel dumb. The REPLACE code is working, what is not working is the way I am copying it back to the BLOB on output.

Further down in the code is this line which is obviously just overwriting it again.

SET OutputRoot = InputRoot;

kimbert

Posted: Wed Dec 19, 2012 11:36 am Post subject:

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Programming : a job for people who don't mind being proved wrong by a machine several times every day.

jrsetters

Posted: Wed Jan 02, 2013 1:48 pm Post subject:

Acolyte

Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH

It scares me because I am a total hack.

Here is what I ended up with, which works just fine in test, although i don't think this is what you intended with 'properly parsed' but honestly I couldn't think of any other way to pull the repetition character out since I can't put this through the MRM parser until after I fix the repeating tilde issue. (I also added an item to change some superscript characters that seem to be causing us problems on medical reports.)

Code:

   DECLARE tmp BLOB;
   DECLARE REPCHAR, TWOREP, SUP2, SUP3 BLOB;
   DECLARE i INT;

   SET OutputRoot = InputRoot;

-- CORRECT BITSTREAM ISSUES <BEGIN>
-- Added 12/19/2012. The following section will attempt to correct issues with the bitstream that
-- have been encoutnered post go live.
   SET tmp = InputRoot.BLOB.BLOB;

      -- This section tries to remove empty repetition strings
      SET REPCHAR = SUBSTRING(tmp FROM 7 FOR 1);
      SET TWOREP = REPCHAR || REPCHAR;
   SET i = 1;

   WHILE i > 0 DO
   IF CONTAINS(tmp, TWOREP) THEN
      SET tmp = REPLACE (tmp, TWOREP, REPCHAR);
   ELSE
      set i = 0;
   END IF;
   END WHILE;

   -- This section replaces superscript characters with plain text equivalents
SET SUP2 = X'B2';
SET SUP3 = X'B3';
SET i = 1;

WHILE i > 0 DO
   IF CONTAINS(tmp, SUP2) THEN
      SET tmp = REPLACE (tmp, SUP2, X'32');
   ELSE
      set i = 0;
   END IF;
   END WHILE;

SET i = 1;
WHILE i > 0 DO
   IF CONTAINS(tmp, SUP2) THEN
      SET tmp = REPLACE (tmp, SUP3, X'33');
   ELSE
      set i = 0;
   END IF;
   END WHILE;

   SET OutputRoot.BLOB.BLOB = tmp;

Esa

Posted: Fri Jan 04, 2013 2:04 am Post subject:

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

Your last loop:

jrsetters wrote:

Code:

SET i = 1;
WHILE i > 0 DO
   IF CONTAINS(tmp, SUP2) THEN
      SET tmp = REPLACE (tmp, SUP3, X'33');
   ELSE
      set i = 0;
   END IF;
   END WHILE;

   SET OutputRoot.BLOB.BLOB = tmp;

would lead to an endless loop, if you hadn't replaced all instances of SUP2 in the previous loop. A typo?

Anyway, your code would look more professional if you removed the dummy loop counter and wrote the tests in a more concise way, something like this:

Code:

WHILE CONTAINS(tmp, TWOREP) DO
SET tmp = REPLACE (tmp, TWOREP, REPCHAR);
END WHILE;

Happy new year!

agrawalamit166

Posted: Fri Jan 04, 2013 9:53 am Post subject:

Voyager

Joined: 17 Aug 2009
Posts: 78
Location: NY, US

Esa wrote:

Anyway, your code would look more professional if you removed the dummy loop counter and wrote the tests in a more concise way, something like this:

Code:

WHILE CONTAINS(tmp, TWOREP) DO
SET tmp = REPLACE (tmp, TWOREP, REPCHAR);
END WHILE;

Replace keyword basically replaces all TWOREP to REPCHAR.... So is it not a good idea to use IF statement in place of WHILE loop.

jrsetters

Posted: Mon Jan 07, 2013 10:54 am Post subject:

Acolyte

Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH

Esa wrote:

Your last loop:

jrsetters wrote:

Code:

SET i = 1;
WHILE i > 0 DO
   IF CONTAINS(tmp, SUP2) THEN
      SET tmp = REPLACE (tmp, SUP3, X'33');
   ELSE
      set i = 0;
   END IF;
   END WHILE;

   SET OutputRoot.BLOB.BLOB = tmp;

Code:

WHILE CONTAINS(tmp, TWOREP) DO
SET tmp = REPLACE (tmp, TWOREP, REPCHAR);
END WHILE;

Happy new year!

Yeah that was a typo I had fixed.

jrsetters

Posted: Mon Jan 07, 2013 10:59 am Post subject:

Acolyte

Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH

agrawalamit166 wrote:

Esa wrote:

Anyway, your code would look more professional if you removed the dummy loop counter and wrote the tests in a more concise way, something like this:

Code:

WHILE CONTAINS(tmp, TWOREP) DO
SET tmp = REPLACE (tmp, TWOREP, REPCHAR);
END WHILE;

Replace keyword basically replaces all TWOREP to REPCHAR.... So is it not a good idea to use IF statement in place of WHILE loop.

I am looping in order to get rid of all instances of repeating tildes (~~, ~~~, ~~~~~ etc).

So if the REPLACE replaces all instances, I am not sure what would happen if came up against say five tildes in a row. With the loop it definitely gets rid of all of them and grinds them down to a single tilde.

I can definitely get rid of the looping on the superscript characters then.

jrsetters

Posted: Mon Jan 07, 2013 11:01 am Post subject:

Acolyte

Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH

agrawalamit166 wrote:

Esa wrote:

Anyway, your code would look more professional if you removed the dummy loop counter and wrote the tests in a more concise way, something like this:

Code:

WHILE CONTAINS(tmp, TWOREP) DO
SET tmp = REPLACE (tmp, TWOREP, REPCHAR);
END WHILE;

Replace keyword basically replaces all TWOREP to REPCHAR.... So is it not a good idea to use IF statement in place of WHILE loop.

I don't even know why I thought I had to place it in an IF to begin with, I probably just saw it on some other similar code I copied - which is basically how I do everything since I am just learning as I go.

Thanks for the tip on how to write that better.

jrsetters

Posted: Mon Jan 07, 2013 11:09 am Post subject:

Acolyte

Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH

I will give this a go. I added a new REPLACE to get rid of fields that start with an empty repetition.

Code:

   SET tmp = InputRoot.BLOB.BLOB;

      -- This section tries to remove any empty repetition characters
      SET REPCHAR = SUBSTRING(tmp FROM 7 FOR 1);
      SET FIELDCHAR = SUBSTRING(tmp FROM 5 FOR 1);
      SET TWOREP = REPCHAR || REPCHAR;
      SET EMPTYREP = FIELDCHAR || REPCHAR;

   -- This loop removes multiple consecutive repetition characters.
      WHILE CONTAINS(tmp, TWOREP) DO
         SET tmp = REPLACE (tmp, TWOREP, REPCHAR);
      END WHILE;

      -- This loop removes any remaining empty repetitions at the beginning of a field.
      WHILE CONTAINS(tmp, EMPTYREP) DO
         SET tmp = REPLACE (tmp, EMPTYREP, FIELDCHAR);
      END WHILE;

   -- This section replaces superscript 2 and 3 (as in square or cubic centimeters)
   -- characters with plain text equivalents.
SET SUP2 = X'B2';
SET SUP3 = X'B3';

   IF CONTAINS(tmp, SUP2) THEN
         SET tmp = REPLACE (tmp, SUP2, X'32');
      END IF;

      IF CONTAINS(tmp, SUP3) THEN
         SET tmp = REPLACE (tmp, SUP3, X'33');
   END IF;

   SET OutputRoot.BLOB.BLOB = tmp;

To give you some context on the havoc this has caused, in HL7 the Date of Admission precedes the Date of Discharge in the 44th and 45th fields of a patient visit segment. With the MRM interpreting empty repetition strings and double repetitions improperly, it was shifting every field afterwards one to the right. In other words we sent our admit date in the discharge date field and discharged a whole bunch of patients from our receiving vendors that were not supposed to be.

jrsetters

Posted: Mon Jan 07, 2013 11:33 am Post subject:

Acolyte

Joined: 24 Aug 2011
Posts: 72
Location: Cincinnati, OH

It seems to have worked like a champ:

MSH|^~\&|EPIC|BEH|||20130107083052|EDRN|ADT^A01|396205|T|2.3||||||||
EVN|A01|20130107083052||ADT_EVENT|EDRN^EMERGENCY^NURSE^^^^^^TH^^^^^BTLR|201301070830
PID|1||E109999^^^EPIC^EPI~00000000001132013^^^EPI^THC_EPI~1132013^^^^THC_NZ||WEBSPHERE^BETASITE||20121212|F|||^^^^^USA^^^|||||||25257|000-00-0000|||||||||||N
PD1|||BUTLER COUNTY^^500098||||||||||
PV1|1|I|B5T01^01^01^BEH^^^^^^^DEPID|Inpatient|||~DRTILDE^INFIRST^POSITION^^^^^^TRI_PROV^^^^TRI_PROV~~DRRANDOM^DOUBLETILDE^MCGEE^^^^^^TRI_PROV^^^^TRI_PROV|
PROPER^PHYSICIAN^NAME^^^^^^TRI_PROV^^^^TRI_PROV~~~EXTRAREP^NONSENSE^PHYSICIAN^^^^^^TRI_PROV^^^^TRI_PROV|TRYING^TOBREAK^WEBSPHEREMD^^^^^^TRI_PROV^^^^TRI_PROV~
TRYING2^TOBREAK2^WEBSPHEREMD2^^^^^^TRI_PROV^^^^TRI_PROV~TRYING^TOBREAK^WEBSPHEREMD^^^^^^TRI_PROV^^^^TRI_PROV~~WHAT ABOUT AN EMPTY REPETITION AT THE END OF A FIELD?~|Emerg|||||||||25257||||||||||||||||||||||Adm|^^^BEH^^^^^^^||201301070830||||||100000003618
PV2||||||||||||||||||||||N||||||||||||||||||||||||||
ZPV|THIS SEGMENT IS USELESS SO I WILL TEST IT TO MAKE SURE THAT REMOVING MUTLTIPLE EMPTY REPETITIONS TO START A FIED DOESN'T REMOVE ANY FIELDS|~|~|~|~|~|~|~|~|~|~|~|~|
OBX|1|TX|HDMCLASS^ADT: PATIENT CLASS|1|E|||||||||20130107
OBX|2|TX|THIS LINE CONTAINS a SUPERSCRIPT 2 -- CMÂ²|||||||||||
OBX|3|TX|THIS LINE CONTAINS a SUPERSCRIPT 3 -- CMÂ³|||||||||||
OBX|2|TX|THIS LINE CONTAINS 2 SUPERSCRIPT 2s -- CMÂ²CMÂ²|||||||||||
OBX|3|TX|THIS LINE CONTAINS 2 SUPERSCRIPT 3s -- CMÂ³TEXTCMÂ³|||||||||||
OBX|4|TX|THIS LINE CONTAINS SOME RANDOM TILDE COMBOS, THIS SHOULD END UP AS EMPTY FIELDS|~|~~|~~~|~~~~|~~~~~|~~~~~~|~~~~~~~|~~~~~~~~|~~~~~~~~~|~~~~~~~~~~
AL1|1|Drug Class|153^NO KNOWN DRUG ALLERGIES^^^||DG1|1||1047^Pleurisy^HRV|Pleurisy|
GT1|1|7293|GENERATED^SYSTEM^^||^^^^^USA^^^|||20121212|F|P/F|SLF|000-00-0000||||||||||||||||||||||||||||||||||||

Translated to

MSH|^~\&|EPIC|BEH|||20130107083052|EDRN|ADT^A01|396205|T|2.3||||||||
EVN|A01|20130107083052||ADT_EVENT|EDRN^EMERGENCY^NURSE^^^^^^TH^^^^^BTLR|201301070830
PID|1||E109999^^^EPIC^EPI~00000000001132013^^^EPI^THC_EPI~1132013^^^^THC_NZ||WEBSPHERE^BETASITE||20121212|F|||^^^^^USA^^^|||||||25257|000-00-0000|||||||||||N
PD1|||BUTLER COUNTY^^500098||||||||||
PV1|1|I|B5T01^01^01^BEH^^^^^^^DEPID|Inpatient|||DRTILDE^INFIRST^POSITION^^^^^^TRI_PROV^^^^TRI_PROV~DRRANDOM^DOUBLETILDE^MCGEE^^^^^^TRI_PROV^^^^TRI_PROV|
PROPER^PHYSICIAN^NAME^^^^^^TRI_PROV^^^^TRI_PROV~EXTRAREP^NONSENSE^PHYSICIAN^^^^^^TRI_PROV^^^^TRI_PROV|TRYING^TOBREAK^WEBSPHEREMD^^^^^^TRI_PROV^^^^TRI_PROV~
TRYING2^TOBREAK2^WEBSPHEREMD2^^^^^^TRI_PROV^^^^TRI_PROV~TRYING^TOBREAK^WEBSPHEREMD^^^^^^TRI_PROV^^^^TRI_PROV~WHAT ABOUT AN EMPTY REPETITION AT THE END OF A FIELD?~|Emerg|||||||||25257||||||||||||||||||||||Adm|^^^BEH^^^^^^^||201301070830||||||100000003618
PV2||||||||||||||||||||||N||||||||||||||||||||||||||
ZPV|THIS SEGMENT IS USELESS SO I WILL TEST IT TO MAKE SURE THAT REMOVING MUTLTIPLE EMPTY REPETITIONS TO START A FIED DOESN'T REMOVE ANY FIELDS|||||||||||||
OBX|1|TX|HDMCLASS^ADT: PATIENT CLASS|1|E|||||||||20130107
OBX|2|TX|THIS LINE CONTAINS a SUPERSCRIPT 2 -- CM2|||||||||||
OBX|3|TX|THIS LINE CONTAINS a SUPERSCRIPT 3 -- CM3|||||||||||
OBX|2|TX|THIS LINE CONTAINS 2 SUPERSCRIPT 2s -- CM2CM2|||||||||||
OBX|3|TX|THIS LINE CONTAINS 2 SUPERSCRIPT 3s -- CM3TEXTCM3|||||||||||
OBX|4|TX|THIS LINE CONTAINS SOME RANDOM TILDE COMBOS, THIS SHOULD END UP AS EMPTY FIELDS||||||||||
AL1|1|Drug Class|153^NO KNOWN DRUG ALLERGIES^^^||DG1|1||1047^Pleurisy^HRV|Pleurisy|
GT1|1|7293|GENERATED^SYSTEM^^||^^^^^USA^^^|||20121212|F|P/F|SLF|000-00-0000||||||||||||||||||||||||||||||||||||

Display posts from previous:

Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Manipulating the BLOB using REPLACE

Jump to:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Protected by Anti-Spam ACP