ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » How to remove unexpected binary character in msg?

Post new topic  Reply to topic
 How to remove unexpected binary character in msg? « View previous topic :: View next topic » 
Author Message
pcelari
PostPosted: Tue Jul 20, 2010 12:50 pm    Post subject: How to remove unexpected binary character in msg? Reply with quote

Chevalier

Joined: 31 Mar 2006
Posts: 411
Location: New York

Hello,

is there a way to filter out illigal binary characters from a message? Occasionally, such character got into a database by user mistype, and result in parser exception.

I wonder if it is possible to filter them out with a function of a kind.

Any insight would be greatly appreciated.


_________________
pcelari
-----------------------------------------
- a master of always being a newbie
Back to top
View user's profile Send private message
lancelotlinc
PostPosted: Tue Jul 20, 2010 1:04 pm    Post subject: Reply with quote

Jedi Knight

Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA

Sounds like another case for Law Enforcement. Illegal characters should be in jail. Unless your on the USA NETWORK television channel, in which case, CHARACTERS ARE WELCOME.

You could drop into a JCN, and scan the payload one character at a time, omitting the copy to the destination message tree those characters you find offensive. Unless, of course, they have already posted bail.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Back to top
View user's profile Send private message Send e-mail
Amitha
PostPosted: Tue Jul 20, 2010 1:23 pm    Post subject: Reply with quote

Voyager

Joined: 20 Nov 2009
Posts: 80
Location: Newyork

If you know what those illegal characters are, you can use translate function in ESQL:

http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r0m0/topic/com.ibm.etools.mft.doc/ak05261_.htm
Back to top
View user's profile Send private message Send e-mail
Vitor
PostPosted: Wed Jul 21, 2010 4:02 am    Post subject: Re: How to remove unexpected binary character in msg? Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

pcelari wrote:
I wonder if it is possible to filter them out with a function of a kind.


Yes - add it to the text box the user is typing into. Not WMB.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
pcelari
PostPosted: Wed Jul 21, 2010 4:14 am    Post subject: Re: How to remove unexpected binary character in msg? Reply with quote

Chevalier

Joined: 31 Mar 2006
Posts: 411
Location: New York

Vitor wrote:

Yes - add it to the text box the user is typing into. Not WMB.


I absolutely agree, except I have no influence to the UI, which was built years ago, and by whom? So I'm trying to figure out sth to handle it defensively, or as a error handling mechanism.


_________________
pcelari
-----------------------------------------
- a master of always being a newbie
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jul 21, 2010 4:21 am    Post subject: Re: How to remove unexpected binary character in msg? Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

pcelari wrote:
or as a error handling mechanism.


You've answered your own question - catch the exception and put the offending message in a queue for later manual correction.

Any function or solution you come up with in WMB will either strip out the illegal characters or replace them with spaces or similar. Under no circumstances will you be able to determine within WMB what the user intended to type. As these fields are presumably business data you're therefore changing the content without reference to the owner and assuming they hit 2 keys together/cut and pasted a bitmap/were struck by lightening while typing. Assumptions, as we all know, can kill.

So the best you can do is prevent WMB issues and file them.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
sknrt1
PostPosted: Wed Jul 21, 2010 7:49 am    Post subject: Reply with quote

Apprentice

Joined: 22 Jan 2003
Posts: 39
Location: USA

As suggested earlier in the discussion, one of the way is to filter out the invalid characters.
Most of the times, tts not possible to have a list of invalid characters list pre-defined, so that you can perform find /replace.
For example, if your message flow is expecting ASCII data and input contains non-ascii data, you can create a routine to filter out the non-ascii characters.
For this you have to read input byte by byte and compare whether it is in the ascii range or not. If it is outside of the ascii range, replace with a pre-defined character 'space' or '@'.
Its recommended to apply this routine part of the exception handling to avoid un-necessary checks in the normal process. Once exception is thrown in the normal process, parse the exception and confirm the exception list includes string like "Invalid character" or "Unconvertible character" or....(based the exception you have experience).

NOTE: Make sure, application is ok to have this manipulation done in Message Broker process to avoid failures. End of the day, we have to make sure not to reject transactions, because user has entered junk data where it shouldn't be.
_________________
------------------------------------------
sk
MQ User since 2001
Message Broker User since 2001

IBM Certified
Solution Designer - WebSphere MQ
Solution Developer - Message Broker
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jul 21, 2010 8:13 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

sknrt1 wrote:
End of the day, we have to make sure not to reject transactions, because user has entered junk data where it shouldn't be.


Forced to disagree for the reasons I give above. What happens if the text in question is the client's correspondence address but has junk in the middle of it. You're quite correct that it's techincally feasible to replace those characters with something useable (like spaces or @) so the transaction will go through, but what's going to happen when the next business process kicks off & they try to send the client something?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
pcelari
PostPosted: Wed Jul 21, 2010 12:04 pm    Post subject: Reply with quote

Chevalier

Joined: 31 Mar 2006
Posts: 411
Location: New York

thank you all so much for sharing these insights!

Vitor wrote:
What happens if the text in question is the client's correspondence address but has junk in the middle of it. ...


You're right now. It turned out one user just copied and pasted an email from someone into the user input field, while the email correspondence address contains formating characters, kursive, bold, etc, even a jpg picture.

Based on the input above, I'll check the exception tree for 'An invalid XML character (Unicode:' and delete the subsequent character with the translate function as Amitha suggested.

Just wonder if anyone has used this function with hex characters like 0x0 or 0x13, how should it be coded in the function, certainly not '0x0'. but what then?
_________________
pcelari
-----------------------------------------
- a master of always being a newbie
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Jul 21, 2010 12:36 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

pcelari wrote:
Just wonder if anyone has used this function with hex characters like 0x0 or 0x13, how should it be coded in the function, certainly not '0x0'. but what then?


IIRC it's X'00' and you'll need to CAST to BLOB.

I remain somewhat concerned that this method relies on the typed data being entirely valid with junk in the middle of it. If it's an email address (with a restricted format) then fair enough but what if it's a free form text field where something's been typed in French?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Wed Jul 21, 2010 1:26 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Vitor wrote:
what if it's a free form text field where something's been typed in French?


Clearly this is a case to show no merci.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Jul 22, 2010 4:00 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

mqjeff wrote:
Vitor wrote:
what if it's a free form text field where something's been typed in French?


Clearly this is a case to show no merci.



_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » How to remove unexpected binary character in msg?
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.