| Author |
Message
|
| NiceGuy |
 Posted: Thu Mar 25, 2010 8:26 am Post subject: [SOLVED]Remove newline(crlf) character from MQinput node? |
 |
|
Apprentice
Joined: 11 Jun 2009
Posts: 37
|
Hi community,
I'm relatively new to message broker.
I currently have an input message that is being feed into my message from an and MQInput node.
| Code: |
| MQInput ---> Message Flow ---> MQOutput |
The message that gets input is of a particular format ("edi") but regardless the input message, although accurate has these additional newline character's at the end of each and every line.
For example:
| Code: |
ISA***********CRLF
***more info***CRLF
***BIG********CRLF
|
Now, the desired message format ("edi") should actually have no newline characters so in essence I need to be able to remove
them. Again once the newline characters are removed, what is left and
remaining would in theory work fine for my "edi" message set that I've created.
I hope I have not confused anyone, just looking for some assistance.
Thanks community
Last edited by NiceGuy on Sat Mar 27, 2010 10:28 am; edited 1 time in total |
|
| Back to top |
|
 |
| MrSmith |
| Posted: Thu Mar 25, 2010 10:11 am Post subject: |
|
|
Master
Joined: 20 Mar 2008
Posts: 215
|
| you could cast the message as a BLOB and then REPLACE (ESQL keyword) in the BLOB any characters that equate to the hex representation of the CRLF character, which may differ I believe on diferent platforms as i think not all platforms have the new line. ESQL manual on replace function as well as cast may be a good place to start, you may not need to do the cast if the input node is without a parser defined. Other alternative if you think you will get use from it would be to use a message set that removes the CRLF but you might need to justify the effort in doing that. |
|
| Back to top |
|
|
| NiceGuy |
| Posted: Thu Mar 25, 2010 10:51 am Post subject: |
|
|
Apprentice
Joined: 11 Jun 2009
Posts: 37
|
Thanks for the reply.
If I were to accept the message as a blob, then inside my flow remove the CRLF from the input message .. how can I then (at that point) utilize my edi message-set to parse the remaining message (the message without the CRLF)?
I still need the message (minus the CRLF) to be parsed according to the message set I've created and be passed through the message flow.
I hope I did not confuse you with the verbiage or explanation.
Much appreciated for your tie and wisdom.
Thanks |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Mar 25, 2010 12:34 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
If your message mode is properly defined and used by both sides then you should be able to parse it regardless of the CR/LF.
I'm wondering if the sender is not formatting the output message properly and you are left to pick up the pieces.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| NiceGuy |
| Posted: Thu Mar 25, 2010 12:59 pm Post subject: |
|
|
Apprentice
Joined: 11 Jun 2009
Posts: 37
|
| Quote: |
| If your message mode is properly defined and used by both sides then you should be able to parse it regardless of the CR/LF. |
My apologies, I'm not sure what your suggesting. My message set define a
EDI message (nonetheless) ... the EDI message set, and subsequently the message Ive created does not expected CRLF.
Again I did not want to deviate from the standard EDI message set format
since its rather complex in itself.
Example
| Code: |
| ISA********GS*******BIG**** .... etc ..... *****IEA |
As you can see each segment in a EDI message is perdicable in term
of the segment keywords ("ISA*" segment, "GS*" segment, etc ....) and as such this is how my message set is (and should ideally) be defined.
However in this particular case .. I need to remove unwanted and undesired CRLF that have been added to the input message. These CRLF
occur at unpredicatle position in the real actual EDI message:
Example
| Code: |
ISA********CRLF
******GS**CRLF
**BIG*****CRLF
*****IEACRLF
|
Quite literally if the input message had all its CRLF removed then
passed thru the MQInput with the edi message set I;ve created everything would be fine.
So does anyone know how to remove the CRLF from input, then pass/parse that message so the remaining message flow can continue?
Thanks |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Mar 25, 2010 3:02 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
I did not want to deviate from the standard EDI message set format
since its rather complex in itself. |
I'm with smdavies99 on this one. The sender is supplying a valid variant of the EDI format ( they're using the 'unwrapped' format ). It just doesn't happen to be the format that your message set was designed for.
| Quote: |
Quite literally if the input message had all its CRLF removed then
passed thru the MQInput with the edi message set I;ve created everything would be fine. |
Are you 100% certain that CR/LF cannot validly occur within the EDI format that you were expecting? There is a danger that you will be too aggressive, and remove linefeeds that were intended to be part of the data.
The theoretically correct solution is to produce a variant of your message set that tolerates CR/LF after each segment. But if you're sure that you can safely strip the CR/LFs then
- parse in the BLOB domain
- remove the CR/LFs as suggested above
- use CREATE...PARSE or an RCD node to reparse the BLOB using the MRM parser. |
|
| Back to top |
|
|
| NiceGuy |
| Posted: Fri Mar 26, 2010 8:31 pm Post subject: |
|
|
Apprentice
Joined: 11 Jun 2009
Posts: 37
|
Sorry apologies ... but how do you replace a CR with nothing?
What I mean to do is remove the CR from the blob.
This does not appear to work
| Code: |
DECLARE CR CHAR CAST(CAST('X''0A''' AS BLOB) AS CHAR CCSID 819);
DECLARE empty CHAR CAST(CAST('X''00''' AS BLOB) AS CHAR CCSID 819);
DECLARE newBlob BLOB CAST(REPLACE(mymessage, CR, empty) AS BLOB);
|
The last line in the code errors out during debugging.
Thanks for the help. |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Sat Mar 27, 2010 12:57 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
There seems to be a lot of casting in there!
Anyway, what error do you get? I suspect its something along the lines of "incompatible types" probably as a result of you casting to chars (but i'm not sure as I dont know the type of "mymessage")  .
Also I noticed that you are attempting to replace the "CR" with the representation of "NULL" which is not the same as the "nothing" referred to by smdavies.
The "empty" variable should be equivalent to the empty string rather than the NULL character. |
|
| Back to top |
|
|
| NiceGuy |
| Posted: Sat Mar 27, 2010 8:24 am Post subject: |
|
|
Apprentice
Joined: 11 Jun 2009
Posts: 37
|
| Quote: |
| The "empty" variable should be equivalent to the empty string rather than the NULL character. |
Thanks ... I believe that's exactly the problem. So I guess I should rephrase my question what is the ACSII/HEX representation of nothing?
I guess what I mean to say in the REPLACE function what should the empty be?
I'll add the code for completeness .. you'll see mymessage is simply a
| Code: |
DECLARE CR CHAR CAST(CAST('X''0A''' AS BLOB) AS CHAR CCSID 819);
DECLARE LF CHAR CAST(CAST('X''0D''' AS BLOB) AS CHAR CCSID 819);
DECLARE empty CHAR CAST(CAST('X''00''' AS BLOB) AS CHAR CCSID 819);
DECLARE mymessage CHARACTER CAST(inputBlob AS CHARACTER CCSID 819);
DECLARE newBlob BLOB CAST(REPLACE(mymessage, CR, empty);
|
Thanks in bunches. |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Sat Mar 27, 2010 8:40 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
Holey Moley..... I thought you had a lot of unecessary casting!
How about just defining the "empty" variable as the empty string (''), ie:
| Code: |
DECLARE empty CHAR '';
|
saving the casts: char -> Blob -> char!
Try that first just so that you can see it working. Then when you're happy try to achieve the same result without all of them casts! |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Mar 27, 2010 8:57 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
If all you really want is change the CRLF into LF (x0d0a into x0a) you could probably verify that the message is declared as a text message with a ccsid of 437 or 850 (msdos) and set the output ccsid to 819 (unix)...
So if you have a huge blob, 2 casts of the whole blob should do it...
DECLARE mytext character cast (blob as CHARACTER ccsid InputRoot.Properties.CodedCharSetId);
DECLARE myblob CAST(mytext as BLOB CCSID 819 ); --OutputRoot.Properties.CodedCharSetId
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| NiceGuy |
| Posted: Sat Mar 27, 2010 9:11 am Post subject: |
|
|
Apprentice
Joined: 11 Jun 2009
Posts: 37
|
Thanks,
Okay so I've been able to successfully remove the CR and LF from my CHARACTER inside my compute node.
No the only thing I have remaining is, I now need to parse (or is it re-parse?) this new message using a message set I've created now that the CRLF has been removed.
If I'm inside my compute node, how do I accomplish this? I've heard a few people mention using a RCD (ResetContentDescriptor node). Of course I've never used on before.
Just wanted to see if this is correct? ... also how do I pass my new message, from my compute node, to a RCD node?
| Code: |
| MQInput --> Computer Node --> RCD Node |
Here are the properties of my theoretical RCD node
Message Domain: MRM
I checked "Reset Message Domain"
MessageSet: MessageSet_EDI_MyMessageSet
I checked: "Reset Message Set"
MessageType: MyMessageType
I checked: "Reset Message Type"
MessageFormat: Text_X12
Thanks again .. I very close I think  |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Sat Mar 27, 2010 9:28 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
If you add your message set project to the list of referenced projects then when you fill in the RCD, it will be much easier. IT will also get the answers right.
Then I suggest you put a TraceNode after the RCD node and use the following pattern
${Root}
This will force parse the message and any error will show up in a user trace. Don't use the debugger! A user trace will give you a lot more information on the operation of the parser.
If you get errors, you can post the relevant bits of the User Trace here.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| NiceGuy |
| Posted: Sat Mar 27, 2010 10:24 am Post subject: |
|
|
Apprentice
Joined: 11 Jun 2009
Posts: 37
|
Thanks for all your help. In particular I'd like to thank:
smdavies99
fjb_saper
WMBDEV1
kimbert
It works very well.
I will be closing the post and changing status to SOLVED and hope the information here can help someone else in the future.
Thank again |
|
| Back to top |
|
|
|
|
