ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » FileInput Node: Special Character

Post new topic  Reply to topic
 FileInput Node: Special Character « View previous topic :: View next topic » 
Author Message
firoj.badsa
PostPosted: Fri May 06, 2011 1:43 am    Post subject: FileInput Node: Special Character Reply with quote

Centurion

Joined: 16 Feb 2007
Posts: 104

Hi,

I have flow like FileInput -> Compute -> HTTPRequest.

FileInput node doing a FTP to a remote server and trying to send a the message over HTTPRequest. But Webservice was failing for some reason. When I did a debug, I saw FileInput node adding some special character at the begining of the first line(). And the error I am getting like 'Invalid byte 1 of 1-byte UTF-8 sequence'. can somebody help here.

thanks in advance,
Firoj
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri May 06, 2011 1:59 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Have you told FileInput to perform the ftp in ascii mode or in binary mode?

Have you told FileInput to process the file in the correct CCSID?
Back to top
View user's profile Send private message
artykito
PostPosted: Fri May 06, 2011 4:19 am    Post subject: Reply with quote

Apprentice

Joined: 24 Jun 2010
Posts: 33

Check that jeff told you.

Anyway, did you check the input file if is it correct? I had a problem with a file sometime ago with a special char that I saw like a space in the input file and like other weird char when the FileInput node parsed the message, but it was the "Abort Output" <AO> character. I detected it with Notepad++ converting the message to other format...maybe your problem is similar.

I have one question for whoever, I don't have too much experience with web services, what's the difference between use SOAPRequest and HTTPRequest nodes? thanks!
Back to top
View user's profile Send private message
firoj.badsa
PostPosted: Fri May 06, 2011 9:09 pm    Post subject: Reply with quote

Centurion

Joined: 16 Feb 2007
Posts: 104

Thanks for your reply Jeff. I have created a new file(.TXT) and copied the content of the previous file and it looks fine now. So, the issue could be there were some issue while the source application creating the file. Actually the original file is a .TXT file and there are renaming it to .CSV before asking me to process the data. Maybe issue is coming while the renaming happens.

thanks,
Firoj
Back to top
View user's profile Send private message
mqjeff
PostPosted: Sat May 07, 2011 3:55 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

firoj.badsa wrote:
Thanks for your reply Jeff. I have created a new file(.TXT) and copied the content of the previous file and it looks fine now. So, the issue could be there were some issue while the source application creating the file. Actually the original file is a .TXT file and there are renaming it to .CSV before asking me to process the data. Maybe issue is coming while the renaming happens.


The renaming almost certainly has nothing to do with the problem.

The problem as stated is that you are receiving a file with a Byte Order Mark on it, and are using a code page that does not expect a Byte Order Mark.

One reason for this could be that you have transfered the file as FTP ASCII, which has caused a code page conversion.

Another reason for this could be that you have specified a Unicode CCSID that does not require a BOM and they have created the file with a CCSID that does.
Back to top
View user's profile Send private message
firoj.badsa
PostPosted: Sat May 07, 2011 5:54 am    Post subject: Reply with quote

Centurion

Joined: 16 Feb 2007
Posts: 104

Hi Jeff,

Actually the transfer method I was using is Binary then I got the special character . But just for a change I changed it to ASCII then I got '???' at the beginning!.
And Input Message Parsing Tab I have selected
CCSID: Broker System Default
Message Encoding: Broker System Determined.

thanks,
Firoj
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Sat May 07, 2011 8:15 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

firoj.badsa wrote:
Hi Jeff,

Actually the transfer method I was using is Binary then I got the special character . But just for a change I changed it to ASCII then I got '???' at the beginning!.
And Input Message Parsing Tab I have selected
CCSID: Broker System Default
Message Encoding: Broker System Determined.

thanks,
Firoj

mqjeff wrote:
The problem as stated is that you are receiving a file with a Byte Order Mark on it, and are using a code page that does not expect a Byte Order Mark.

Read again very carefully what mqjeff said. If you do not understand any of those terms, (like byte order mark) google them and then ask here for more clarification.

Your last attempt to fix this is showing more guesswork than actual reflection. You have a CCSID problem there. You need to deal with it.

Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
firoj.badsa
PostPosted: Sat May 07, 2011 9:17 am    Post subject: Reply with quote

Centurion

Joined: 16 Feb 2007
Posts: 104

I did not understood what guesswork I did.. I just explained what I did ..
Back to top
View user's profile Send private message
kimbert
PostPosted: Sat May 07, 2011 12:42 pm    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

mqjeff is 100% correct. Your input message has a BOM.
Quote:
I did not understood what guesswork I did.. I just explained what I did ..
There is a standard process for solving problems like this:
1 Try to understand the problem
2 Design a fix for the problem, based on the output of step 1.

Vitor ( edit: fjb_saper actually) thinks that you did this
1. Ask a question on this forum
2. Ignore the ( correct ) answer, and try some random change to see whether it works.

I think I agree with fjb_saper. And I agree with his suggestion:
Quote:
Read again very carefully what mqjeff said. If you do not understand any of those terms, (like byte order mark) google them and then ask here for more clarification.


Last edited by kimbert on Sat May 07, 2011 2:12 pm; edited 1 time in total
Back to top
View user's profile Send private message
mqjeff
PostPosted: Sat May 07, 2011 12:52 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

kimbert wrote:
I think I agree with Vitor.

Presumably in some other thread, however.
Back to top
View user's profile Send private message
firoj.badsa
PostPosted: Sat May 07, 2011 8:19 pm    Post subject: Reply with quote

Centurion

Joined: 16 Feb 2007
Posts: 104

Ok.. ok. let me clarify .. all I was saying is I have tried all the things only on friday .. and surely I have to look for the BOM on monday . I can not afford to ignore any suggestion when I need a solution .
Back to top
View user's profile Send private message
rekarm01
PostPosted: Sun May 08, 2011 6:16 am    Post subject: Re: FileInput Node: Special Character Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

firoj.badsa wrote:
And the error I am getting like 'Invalid byte 1 of 1-byte UTF-8 sequence'. ...
And Input Message Parsing Tab I have selected
CCSID: Broker System Default

The receiver needs to use the same ccsid to read the file that the sender used to write the file. Confirm that the sender wrote the file correctly, and that the receiver read the file correctly.

While some applications will use a BOM for UTF-8, WMB does not expect one, so the sender or receiver may need to remove it.

firoj.badsa wrote:
I have created a new file(.TXT) and copied the content of the previous file and it looks fine now.

Copying the file introduces a new receiver, sender, and (possibly converted) file between the original sender and receiver.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Sun May 08, 2011 8:55 am    Post subject: Re: FileInput Node: Special Character Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

rekarm01 wrote:
firoj.badsa wrote:
And the error I am getting like 'Invalid byte 1 of 1-byte UTF-8 sequence'. ...
And Input Message Parsing Tab I have selected
CCSID: Broker System Default

The receiver needs to use the same ccsid to read the file that the sender used to write the file.


Which means that the FileInput node needs to use the correct CodedCharSetID. Which may not be Broker System Default.
Back to top
View user's profile Send private message
rekarm01
PostPosted: Sun May 08, 2011 11:42 am    Post subject: Re: FileInput Node: Special Character Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

Although ...
  • if the message contains a UTF-8 BOM, this suggests that the sender intended to write UTF-8
  • if the error complains about a malformed UTF-8 sequence, this suggests that the receiver attempted to read UTF-8
So, if the File Input node ccsid is correct, check the other relevant properties on the Input Message Parsing tab (message domain, set, type, format, encoding). If there's still a problem, it's likely the bytes are corrupt; the next step is to check them at each point between sender and receiver, to narrow down where the corruption(s) occurred.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Sun May 08, 2011 4:58 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

The receiver could entirely have meant UTF-16 BOM and the receiver attempted UTF-16 NO BOM.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » FileInput Node: Special Character
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.