| Author |
Message
|
| pcelari |
 Posted: Tue Jul 20, 2010 12:50 pm Post subject: How to remove unexpected binary character in msg? |
 |
|
Chevalier
Joined: 31 Mar 2006
Posts: 411
Location: New York
|
Hello,
is there a way to filter out illigal binary characters from a message? Occasionally, such character got into a database by user mistype, and result in parser exception.
I wonder if it is possible to filter them out with a function of a kind.
Any insight would be greatly appreciated.
_________________
pcelari
-----------------------------------------
- a master of always being a newbie |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Tue Jul 20, 2010 1:04 pm Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Sounds like another case for Law Enforcement. Illegal characters should be in jail. Unless your on the USA NETWORK television channel, in which case, CHARACTERS ARE WELCOME.
You could drop into a JCN, and scan the payload one character at a time, omitting the copy to the destination message tree those characters you find offensive. Unless, of course, they have already posted bail.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Amitha |
| Posted: Tue Jul 20, 2010 1:23 pm Post subject: |
|
|
Voyager
Joined: 20 Nov 2009
Posts: 80
Location: Newyork
|
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 21, 2010 4:02 am Post subject: Re: How to remove unexpected binary character in msg? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| pcelari wrote: |
| I wonder if it is possible to filter them out with a function of a kind. |
Yes - add it to the text box the user is typing into. Not WMB.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| pcelari |
| Posted: Wed Jul 21, 2010 4:14 am Post subject: Re: How to remove unexpected binary character in msg? |
|
|
Chevalier
Joined: 31 Mar 2006
Posts: 411
Location: New York
|
| Vitor wrote: |
Yes - add it to the text box the user is typing into. Not WMB. |
I absolutely agree, except I have no influence to the UI, which was built years ago, and by whom? So I'm trying to figure out sth to handle it defensively, or as a error handling mechanism.
_________________
pcelari
-----------------------------------------
- a master of always being a newbie |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 21, 2010 4:21 am Post subject: Re: How to remove unexpected binary character in msg? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| pcelari wrote: |
| or as a error handling mechanism. |
You've answered your own question - catch the exception and put the offending message in a queue for later manual correction.
Any function or solution you come up with in WMB will either strip out the illegal characters or replace them with spaces or similar. Under no circumstances will you be able to determine within WMB what the user intended to type. As these fields are presumably business data you're therefore changing the content without reference to the owner and assuming they hit 2 keys together/cut and pasted a bitmap/were struck by lightening while typing. Assumptions, as we all know, can kill.
So the best you can do is prevent WMB issues and file them.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sknrt1 |
| Posted: Wed Jul 21, 2010 7:49 am Post subject: |
|
|
Apprentice
Joined: 22 Jan 2003
Posts: 39
Location: USA
|
As suggested earlier in the discussion, one of the way is to filter out the invalid characters.
Most of the times, tts not possible to have a list of invalid characters list pre-defined, so that you can perform find /replace.
For example, if your message flow is expecting ASCII data and input contains non-ascii data, you can create a routine to filter out the non-ascii characters.
For this you have to read input byte by byte and compare whether it is in the ascii range or not. If it is outside of the ascii range, replace with a pre-defined character 'space' or '@'.
Its recommended to apply this routine part of the exception handling to avoid un-necessary checks in the normal process. Once exception is thrown in the normal process, parse the exception and confirm the exception list includes string like "Invalid character" or "Unconvertible character" or....(based the exception you have experience).
NOTE: Make sure, application is ok to have this manipulation done in Message Broker process to avoid failures. End of the day, we have to make sure not to reject transactions, because user has entered junk data where it shouldn't be.
_________________
------------------------------------------
sk
MQ User since 2001
Message Broker User since 2001
IBM Certified
Solution Designer - WebSphere MQ
Solution Developer - Message Broker |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 21, 2010 8:13 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sknrt1 wrote: |
| End of the day, we have to make sure not to reject transactions, because user has entered junk data where it shouldn't be. |
Forced to disagree for the reasons I give above. What happens if the text in question is the client's correspondence address but has junk in the middle of it. You're quite correct that it's techincally feasible to replace those characters with something useable (like spaces or @) so the transaction will go through, but what's going to happen when the next business process kicks off & they try to send the client something?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| pcelari |
| Posted: Wed Jul 21, 2010 12:04 pm Post subject: |
|
|
Chevalier
Joined: 31 Mar 2006
Posts: 411
Location: New York
|
thank you all so much for sharing these insights!
| Vitor wrote: |
| What happens if the text in question is the client's correspondence address but has junk in the middle of it. ... |
You're right now. It turned out one user just copied and pasted an email from someone into the user input field, while the email correspondence address contains formating characters, kursive, bold, etc, even a jpg picture.
Based on the input above, I'll check the exception tree for 'An invalid XML character (Unicode:' and delete the subsequent character with the translate function as Amitha suggested.
Just wonder if anyone has used this function with hex characters like 0x0 or 0x13, how should it be coded in the function, certainly not '0x0'. but what then?
_________________
pcelari
-----------------------------------------
- a master of always being a newbie |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 21, 2010 12:36 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| pcelari wrote: |
| Just wonder if anyone has used this function with hex characters like 0x0 or 0x13, how should it be coded in the function, certainly not '0x0'. but what then? |
IIRC it's X'00' and you'll need to CAST to BLOB.
I remain somewhat concerned that this method relies on the typed data being entirely valid with junk in the middle of it. If it's an email address (with a restricted format) then fair enough but what if it's a free form text field where something's been typed in French? 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jul 21, 2010 1:26 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Vitor wrote: |
| what if it's a free form text field where something's been typed in French? |
Clearly this is a case to show no merci. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jul 22, 2010 4:00 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| Vitor wrote: |
| what if it's a free form text field where something's been typed in French? |
Clearly this is a case to show no merci. |
 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
