| Author |
Message
|
| mrlizard |
 Posted: Mon Jun 18, 2018 6:39 am Post subject: Converting Unicode to 819 with default character on error |
 |
|
Newbie
Joined: 03 Jul 2017
Posts: 5
|
Anybody know if there is anything out of the box for IIB that will allow me to implement a substitution character when a code page conversion error occurs? It's an interim solution for Unicode to ISO-8859-1 whilst we have a long think about alternative options such as ICU transliterators or migrating a legacy system off ISO-8859-1.
At the moment, we are on 10.0.0.7 and we are mapping from UTF-8 JSON into ISO-8859-1 DFDL. There is a DFDL feature which we have explored, known as 'dfdl:encodingErrorPolicy "replace"' which is not implemented in IIB at 10.0.0.12. So I'm wondering whether there is a secret switch in IIB.
dfdl:encodingErrorPolicy link
https://www.ibm.com/support/knowledgecenter/en/SSMKHH_10.0.0/com.ibm.etools.mft.doc/df00150_.htm
Note : I have seen IBM SAP adapters convert unsupported characters to ? out of the box. |
|
| Back to top |
|
 |
| gbaddeley |
| Posted: Mon Jun 18, 2018 4:27 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
In general, substituting unsupported characters results in a loss of information that may cause issues for downstream systems. The best solution is to implement reliable 1 to 1 conversion, or require the source system to only use character sets that you support.
We use IIB on AIX and Linux. We generally use plain old ASCII (where there are no special character requirements) or UTF-8.
_________________
Glenn |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jun 18, 2018 10:14 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
And remember some character substitutions are deadly.
Example when happening inside of an XML document and rendering the document invalid because the substitution char would need to be escaped... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| timber |
| Posted: Tue Jun 19, 2018 7:45 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| Anybody know if there is anything out of the box for IIB that will allow me to implement a substitution character when a code page conversion error occurs? |
You could install your own code page converter.
I don't know whether there is an undocumented feature for supporting that DFDL property. You would need to open a PMR to find out. |
|
| Back to top |
|
|
| mrlizard |
| Posted: Mon Jun 25, 2018 9:57 am Post subject: |
|
|
Newbie
Joined: 03 Jul 2017
Posts: 5
|
| timber wrote: |
| Quote: |
| Anybody know if there is anything out of the box for IIB that will allow me to implement a substitution character when a code page conversion error occurs? |
You could install your own code page converter.
I don't know whether there is an undocumented feature for supporting that DFDL property. You would need to open a PMR to find out. |
Thanks Timber. That seems to be what I want. Looks quite well documented on the info centre too. |
|
| Back to top |
|
|
|
|
