| Author |
Message
|
| dcjs |
 Posted: Tue Mar 18, 2008 3:43 am Post subject: Encoding French |
 |
|
Acolyte
Joined: 10 Nov 2006
Posts: 53
Location: IBM Bangalore
|
Environment(WBIMB 5.0 on AIX 5.0)
Iam using SendMailPlugIn node where it picks a message from a queue and updates the table and sends an email confirmation.
My problem is iam getting french characters in input XML message and Broker is not able to encode them properly.
Could anyone help me what needs to be done
_________________
TRY TRY UNTIL U SUCCEED |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Mar 18, 2008 3:55 am Post subject: Re: Encoding French |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| dcjs wrote: |
My problem is iam getting french characters in input XML message and Broker is not able to encode them properly.
|
Is the Broker using a French code page?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| dcjs |
| Posted: Tue Mar 18, 2008 4:00 am Post subject: |
|
|
Acolyte
Joined: 10 Nov 2006
Posts: 53
Location: IBM Bangalore
|
Its using the default code page and UTF-8 was set in the SendMailPluginNode properties.My doubt is UTF-8 should support French too right..
_________________
TRY TRY UNTIL U SUCCEED |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Mar 18, 2008 4:11 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| dcjs wrote: |
| My doubt is UTF-8 should support French too right.. |
I think you want Unicode rather than UTF-8. AFAIK UTF-8 only holds standard ASCII (will stand correction on that, but your experience does seem to support this view).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| dcjs |
| Posted: Tue Mar 18, 2008 4:20 am Post subject: |
|
|
Acolyte
Joined: 10 Nov 2006
Posts: 53
Location: IBM Bangalore
|
Thanks Vitor for the reply.
So how would i support french characters.
Should i update the encoding value or ccsid value in MQMD.
If so could you tell the values for those
_________________
TRY TRY UNTIL U SUCCEED |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Mar 18, 2008 5:24 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| dcjs wrote: |
Should i update the encoding value or ccsid value in MQMD.
If so could you tell the values for those |
It won't be the encoding value I'd have thought - unless they're using French tactics on the numbers! 
(Anti-French British jibe - retaliatory comments about Britsh cooking welcomed....)
Seriously, I would expect the MQMD to be already coded to indicate the presence of French characters (you said the French was in the input). What you need is a Broker configuration that can support all the required characters and a get with convert. As I said, I'd be inclined to use Unicode throughout (IIRC Broker's using it internally anyway).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Mar 18, 2008 5:30 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| AFAIK UTF-8 only holds standard ASCII |
Not true. It works like this:
- Unicode defines character code points from 0x00000000 to 0xFFFFFFFF. Not all of them are legal characters.
- A string of Unicode characters can be encoded in UTF-8. UTF-16 or UTF-32. There are no restrictions - UTF-8 and UTF-16 can encode any Unicode code point by using multi-byte characters where necessary.
- The first 127 Unicode code points correspond to the first 127 ASCII characters. That means that a large proportion of English text can be encoded using a single-byte UTF-8 character.
dcjs : All of the XML parsers in message broker will automatically encode the output message in the code page specified by OutputRoot.Properties.CodedCharSetId. What exactly is going wrong in your flow? |
|
| Back to top |
|
|
| servi |
| Posted: Wed Mar 19, 2008 4:51 am Post subject: |
|
|
Novice
Joined: 19 Mar 2008
Posts: 22
Location: Madrid, España
|
You can try usign at the first a Compute Node and change the CCSID to Unicode:
OutputRoot.Properties.CodedCharSetId=1200
This will allow the Broker "see" correctly the special characters. |
|
| Back to top |
|
|
| sunny_30 |
| Posted: Fri Mar 21, 2008 7:57 am Post subject: |
|
|
Master
Joined: 03 Oct 2005
Posts: 258
|
Let the XML be in UTF-8.
If the message is not coming with CCSID-1208, first convert the message to CCSID(1208).
Later convert it to the AIX platform - MQMD.Encoding-273 & CCSID-819. |
|
| Back to top |
|
|
| broker_new |
| Posted: Tue May 26, 2009 11:12 am Post subject: |
|
|
Yatiri
Joined: 30 Nov 2006
Posts: 614
Location: Washington DC
|
Guys, i had the same problem.
Could anyone help me on this.
(WMB v6.0.3)
I am getting an UTF-8 message (input properties ccsid=1208).
Message flow automatically converting the french characters from À to À.....
_________________
IBM ->Let's build a smarter planet |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue May 26, 2009 1:56 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
So what is your originating codepage? 437? 850?
Is your message truly in that codepage?
Does your message advertize to be in that codepage?
Make sure your input is right. If you then generate output in UTF-8 you should be fine... on condition that your target platform can deal with UTF-8 ...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Tue May 26, 2009 2:47 pm Post subject: Re: Encoding French |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| broker_new wrote: |
I am getting an UTF-8 message (input properties ccsid=1208).
Message flow automatically converting the french characters from À to À..... |
Is the message flow actually changing the characters, or does it just seem to be changing them? If an application displays UTF-8 (ccsid=1208) characters, but the display device is configured to display ISO8859-1 (ccsid=819) characters, then 'À' might look like 'À', but it's really not.
| Code: |
chars bytes
----- -----
ccsid=1208: 'À' <--> X'C380'
ccsid= 819: 'À' <--> X'C380'
ccsid= 819: 'À' <--> X'C0' |
What is the ccsid, and what are the actual bytes, for the input message?
What is the ccsid, and what are the actual bytes, for the output message?
What is the format of the message? "MQSTR "? WMB won't automatically convert characters, unless it knows which bytes represent characters. |
|
| Back to top |
|
|
| broker_new |
| Posted: Tue May 26, 2009 6:45 pm Post subject: |
|
|
Yatiri
Joined: 30 Nov 2006
Posts: 614
Location: Washington DC
|
Thanks for the reply.
Here is the trace output
(0x01000000):Properties = (
(0x03000000):MessageSet = NULL
(0x03000000):MessageType = NULL
(0x03000000):MessageFormat = NULL
(0x03000000):Encoding = NULL
(0x03000000):CodedCharSetId = 1208
(0x03000000):Transactional = TRUE
(0x03000000):Persistence = FALSE
(0x03000000):CreationTime = GMTTIMESTAMP '2009-05-27 02:35:05.950'
(0x03000000):ExpirationTime = -1
(0x03000000):Priority = 0
(0x03000000):ReplyIdentifier = X'000000000000000000000000000000000000000000000000'
(0x03000000):ReplyProtocol = 'MQ'
(0x03000000):Topic = NULL
(0x03000000):ContentType = NULL
)
(0x01000000):MQMD = (
(0x03000000):SourceQueue = 'EMAIL_IN'
(0x03000000):Transactional = TRUE
(0x03000000):Encoding = 546
(0x03000000):CodedCharSetId = 1208
(0x03000000):Format = 'MQHRF2'
(0x03000000):Version = 2
(0x03000000):Report = 0
(0x03000000):MsgType = 8
(0x03000000):Expiry = -1
(0x03000000):Feedback = 0
(0x03000000):Priority = 0
(0x03000000):Persistence = 0
(0x03000000):MsgId = X'414d51205745424f45443134202020204949279e2017d502'
(0x03000000):CorrelId = X'000000000000000000000000000000000000000000000000'
(0x03000000):BackoutCount = 0
(0x03000000):ReplyToQ = 'EMAIL_REPLYQ'
(0x03000000):ReplyToQMgr = 'DQM019 '
(0x03000000):UserIdentifier = 'mqm '
(0x03000000):AccountingToken = X'1601051500000028501556d136445d8b2de024f812020000000000000000000b'
(0x03000000):ApplIdentityData = ' '
(0x03000000):PutApplType = 11
(0x03000000):PutApplName = ' '
(0x03000000):PutDate = DATE '2009-05-27'
(0x03000000):PutTime = GMTTIME '02:35:05.950'
(0x03000000):ApplOriginData = ' '
(0x03000000):GroupId = X'000000000000000000000000000000000000000000000000'
(0x03000000):MsgSeqNumber = 1
(0x03000000):Offset = 0
(0x03000000):MsgFlags = 0
(0x03000000):OriginalLength = -1
)
(0x03000000):Format = ''
(0x01000000):mcd = (
(0x01000000):Msd = (
(0x02000000): = 'xml'
)
)
(0x03000000):CodedCharSetId = 1208
)
(0x01000010):XML = (
(0x01000000):Message = (
(0x01000000):From = (
(0x02000000): = 'wmb@staples.com'
)
(0x01000000):To = (
(0x02000000): = 'inquiry@staples.com'
(0x01000000):Subject = (
(0x02000000): = 'General feedback'
)
(0x01000000):Body = (
(0x01000000):Line = (
(0x02000000): = ' '
)
(0x01000000):Line = (
(0x02000001): = 'First Name : ^Z ^Z ^Z ^Z ^Z ^Z ^Z ^Z ^Z ^Z'
)
(0x01000000):Line = (
(0x02000001): = 'Last Name : ^Z ^Z ^Z ^Z ^Z ^Z ^Z ^Z ^Z ^Z'
)
)
)
)
The input is
<Customer_First_Name>À È Ì Ò Ù à è ì ò ù</Customer_First_Name>
<Customer_Last_Name>À È Ì Ò Ù à è ì ò ù</Customer_Last_Name>
the output is
First Name : └ ╚ ╠╥ ┘ α Φ ∞ ≥ ∙
Last Name : └ ╚ ╠╥ ┘ α Φ ∞ ≥ ∙
_________________
IBM ->Let's build a smarter planet |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue May 26, 2009 7:41 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Use RFHUtil and give us the char displayed and the equivalent hex value for the char...
Remember that you need a device driver that can display UTF-8 correctly.
Usually I use Java or JMS for that... Most of the display programs will not deal with a charset that is not native...(pc ccsid different from 437 or 850 to name the most popular ones...)
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| broker_new |
| Posted: Tue May 26, 2009 8:11 pm Post subject: |
|
|
Yatiri
Joined: 30 Nov 2006
Posts: 614
Location: Washington DC
|
Right, but the problem is here with sendmailplugin which is converting the french characters. the output which i meant was the one which i copied and posted from the email.I tried by populated the XML encoding as UTF-8 but it is breaking the sendmailplugin node.
the output from the email is
First Name : └ ╚ ╠╥ ┘ α Φ ∞ ≥ ∙
Last Name : └ ╚ ╠╥ ┘ α Φ ∞ ≥ ∙
_________________
IBM ->Let's build a smarter planet |
|
| Back to top |
|
|
|
|
