ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportConverting Unicode to 819 with default character on error

Post new topicReply to topic
Converting Unicode to 819 with default character on error View previous topic :: View next topic
Author Message
mrlizard
PostPosted: Mon Jun 18, 2018 6:39 am Post subject: Converting Unicode to 819 with default character on error Reply with quote

Newbie

Joined: 03 Jul 2017
Posts: 5

Anybody know if there is anything out of the box for IIB that will allow me to implement a substitution character when a code page conversion error occurs? It's an interim solution for Unicode to ISO-8859-1 whilst we have a long think about alternative options such as ICU transliterators or migrating a legacy system off ISO-8859-1.

At the moment, we are on 10.0.0.7 and we are mapping from UTF-8 JSON into ISO-8859-1 DFDL. There is a DFDL feature which we have explored, known as 'dfdl:encodingErrorPolicy "replace"' which is not implemented in IIB at 10.0.0.12. So I'm wondering whether there is a secret switch in IIB.

dfdl:encodingErrorPolicy link
https://www.ibm.com/support/knowledgecenter/en/SSMKHH_10.0.0/com.ibm.etools.mft.doc/df00150_.htm


Note : I have seen IBM SAP adapters convert unsupported characters to ? out of the box.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Mon Jun 18, 2018 4:27 pm Post subject: Reply with quote

Padawan

Joined: 25 Mar 2003
Posts: 1872
Location: Melbourne, Australia

In general, substituting unsupported characters results in a loss of information that may cause issues for downstream systems. The best solution is to implement reliable 1 to 1 conversion, or require the source system to only use character sets that you support.

We use IIB on AIX and Linux. We generally use plain old ASCII (where there are no special character requirements) or UTF-8.
_________________
Glenn
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Jun 18, 2018 10:14 pm Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19744
Location: LI,NY

And remember some character substitutions are deadly.
Example when happening inside of an XML document and rendering the document invalid because the substitution char would need to be escaped...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
timber
PostPosted: Tue Jun 19, 2018 7:45 am Post subject: Reply with quote

Sentinel

Joined: 25 Aug 2015
Posts: 839

Quote:
Anybody know if there is anything out of the box for IIB that will allow me to implement a substitution character when a code page conversion error occurs?
You could install your own code page converter.

I don't know whether there is an undocumented feature for supporting that DFDL property. You would need to open a PMR to find out.
Back to top
View user's profile Send private message
mrlizard
PostPosted: Mon Jun 25, 2018 9:57 am Post subject: Reply with quote

Newbie

Joined: 03 Jul 2017
Posts: 5

timber wrote:
Quote:
Anybody know if there is anything out of the box for IIB that will allow me to implement a substitution character when a code page conversion error occurs?
You could install your own code page converter.

I don't know whether there is an undocumented feature for supporting that DFDL property. You would need to open a PMR to find out.


Thanks Timber. That seems to be what I want. Looks quite well documented on the info centre too.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker SupportConverting Unicode to 819 with default character on error
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.