| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| DFDL and Latin characters |
« View previous topic :: View next topic » |
| Author |
Message
|
| smuktineni |
 Posted: Thu Jan 23, 2014 12:11 pm Post subject: DFDL and Latin characters |
 |
|
Apprentice
Joined: 28 Aug 2003
Posts: 33
Location: Omaha
|
WebSphere MQ @ 7.5.0.0
Broker @ 8.0.0.3
Aix 6.1.0.0
Its a coordinated request-reply pattern
MQ Input --> MQ Get (ReplyId) --> Compute --> SoapReply
Input Data has latin characters in element address at offset/position 448 "PASEO TOLLOCAN Ñ128 LOC K "
On MQ Input
| Code: |
Message Domain: DFDL
Message: {....}Response
Encoding: 273
CCSID: 819
|
Flow fails at
| Code: |
| SET OutputRoot.XMLNSC.ns:CustomerResponse = InputRoot.DFDL.ns:CustomerResponse; |
Before above statement Encoding results in 273 and CodedCharSetId is 819
Service Trace
| Code: |
2014-01-23 10:56:07.053203 6170 UserTrace BIP5841I: ''Offset: 432. Starting to process element 'address1'.''
2014-01-23 10:56:07.053287 6170 UserTrace BIP5841I: ''Offset: 487. Parser was unable to resolve data on the current branch and will evaluate the next available branch beginning at offset '312' owned by the 'sequence' group contained within element 'CustomerResponse'.''
2014-01-23 10:56:07.053314 6170 UserTrace BIP5841I: ''Offset: 312. Optional element 'ratingConfig' encountered. The DFDL parser will return to this position if the element is not present in the data stream.''
|
Guessing exception from debug & trace is a bit misleading based on trace as the offset went from 487 to 312 and parsing continued and the entire output was wrong.
| Code: |
Text:CHARACTER:SET OutputRoot.XMLNSC.ns:CustomerResponse = InputRoot.DFDL.ns:CustomerResponse;
ParserException
File:CHARACTER:/build/slot1/S800_P/src/MTI/MTIforBroker/DfdlParser/ImbDFDLErrorHandler.cpp
Line:INTEGER:174
Function:CHARACTER:ImbDFDLErrorHandler::handleParserErrors
Type:CHARACTER:ComIbmMQInputNode
Name:CHARACTER:CustomerProfileSynchResponse#FCMComposite_1_1
Label:CHARACTER:CustomerProfileSynchResponse.MQ Input
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:3
Number:INTEGER:5807
Text:CHARACTER:An error occurred whilst parsing with DFDL
Insert
Type:INTEGER:5
Text:CHARACTER:CTDP3002E: Unexpected data found at offset '352' after parsing completed. Data: '0x20...'. |
Will post DFDL if required. I don't have encoding specified in DFDL
| Code: |
| <?xml version="1.0" ?> |
I strongly feel somewhere the encoding needs to be set properly, maybe not! 
FYI... the flow works good if there are no latin characters.
Probably should be a separate post, how would broker treat below 2 be different?
| Code: |
SET OutputRoot.XMLNSC = InputRoot.DFDL;
vs
DECLARE inBitStream BLOB ASBITSTREAM(InputRoot.DFDL, inEncoding, inCCSID);
CREATE LASTCHILD OF OutputRoot DOMAIN('XMLNSC')
PARSE(inBitStream, inEncoding, inCCSID, '', '{namespace}:CustomerResponse'); |
Thanks in advance
_________________
-Satish |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Thu Jan 23, 2014 12:27 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
find out what the HEX value(s) of the errant character(s) is BEFORE the message is read by the Flow.
Then go to the various web pages for the different ISO-8859-x Character Sets and see what ones match the actual character you are expecting.
Then if the actual character is NOT in ISO-8859-1 (== CCSID 819) then you have to go back to the originator and get them to change their application so that the data uses the right CCSID in the Message MQMD.
IMHO a lot of these problems can be avoided by enforcing the 'UTF' everywhere rule. It solves a lot of problems in the long run.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| kimbert |
| Posted: Thu Jan 23, 2014 12:38 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| I don't have encoding specified in DFDL |
If true, then this may be your problem. Even if it is not, it is a problem waiting to happen. But it might not be true...
First, check your DFDL schema. The DFDL property 'encoding' should be set to $dfdl:encoding. This causes the DFDL model to inherit its character encoding from the message flow's CCSID ( so in your case, it will get the value 819 ).
Important: Every part of a DFDL model has its own 'encoding' property. In principle, you could set the encoding differently on every element. But 99.9% of formats use the same encoding for the entire message. So DFDL allows the 'encoding' property ( and any other property ) to be defaulted. So make sure that you set the *default* value of the 'encoding' property ( in the format block for the main xsd ) and not some localised version of it.
The DFDL trace ( included in WMB's debug-level user trace ) will tell you which encoding DFDL is using. If you find that setting dfdl:encoding doesn't fix the problem, then take a debug-level user trace and tell us what you find.
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| smuktineni |
| Posted: Thu Jan 23, 2014 1:34 pm Post subject: |
|
|
Apprentice
Joined: 28 Aug 2003
Posts: 33
Location: Omaha
|
worked
You are the best.
@smdavies99
FYI... I had UTF-8 and it didn't work. I removed it with the assumption that broker will default based on documentation and hoping it will work, but it didn't.
_________________
-Satish |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
