| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Data conversion problem |
« View previous topic :: View next topic » |
| Author |
Message
|
| mqmqmq |
 Posted: Tue Dec 16, 2008 6:11 am Post subject: Data conversion problem |
 |
|
Novice
Joined: 02 Jul 2008
Posts: 22
|
Hi all,
I have some problems with data conversion.
I have WebSphere MQ (version 6.0.2.5) running on Solaris. Broker (version 6.1.0.2) runs also on Solaris. Queue managers CCSID is 923 (ISO8859-15).
I have a Windows XP laptop where MB Toolkit is installed (version 6.1.0). There is also rhfutilc -utility (version 6.0.0) on that laptop.
Scenario 1:
- bar file is already deployed and it is working
- one queue is already created
- connection from Rfhutilc to queue above is ok.
- user trace in execution group is set to debug
I send very simple xml file (see below) from rfhutilc to MQ Queue and read the file with one flow. Code page in Rfhutil is 437.
In the flow I just read the data from HEADER -element and check that in debug.
| Code: |
<?xml version="1.0" encoding="UTF-8"?>
<START>
<HEADER>TestData11ÄÅÖäåö22</HEADER>
</START> |
| Code: |
DECLARE gg CHAR;
SET gg = InputBody.START.HEADER; |
After file is handled by a flow I checked the logfile (mqsireadlog and mqsiformatlog). In the log file the data in HEADER -element has incorrect characters.
Example: TestData1122. So the special characters are printed out as squeres.
I tried to change the code page in MQMD -section of Rfhutilc (819, 923, 1252, 1208 etc.) before sending the file, but without any success. When I check the codedcharsetid from the message tree (see below) after the flow received the file, I will receive exactly the same value than it is defined in Rfhutilc.
| Code: |
| SET cCCSID = InputRoot.Properties.CodedCharSetId; |
I should put the data in xml file also into DB, but there is the same thing, the special characters are not printed correctly.
I have also Trace node right after MQInput node and there I print everything out:
Same thing occurs there, ie. no visible special characters in Trace's log file.
I have also tried to do the conversion in MQInput node (Properties->Advanced->Convert) but no luck so far.
Now the questions:
- What might be the steps to print out the special characters correctly (examples?)?
- Should I configure something? If so, what might it be and where?
- Do I need to configure Solaris itself?
I'm sure that there is a clear answer for this, but I missing that right now.
If you have time, could you please send me an early Christmas present and reply to this question.
Thanks a lot!
BR,
--
mqmqmq |
|
| Back to top |
|
 |
| kimbert |
| Posted: Tue Dec 16, 2008 2:41 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Taking your points in the order in which you listed them...
| Quote: |
| bar file is already deployed and it is working |
Do you mean that your message flow is working without any problems, and this question is all about how to read the user trace/Trace node output?
| Quote: |
| user trace in execution group is set to debug |
Good 
| Quote: |
In the log file the data in HEADER -element has incorrect characters.
Example: TestData1122. So the special characters are printed out as squeres. |
Don't worry about that. It's probably a problem with your text viewer. It doesn't necessarily mean that your message flow is going wrong.
| Quote: |
| I should put the data in xml file also into DB, but there is the same thing, the special characters are not printed correctly. |
Once again, I can't work out whether your message flow is working properly, or whether you are asking about how to view the DB contents.
| Quote: |
I have also Trace node right after MQInput node and there I print everything out:
...
Same thing occurs there, ie. no visible special characters in Trace's log file. |
Did you forget that all character data in the message tree is in UTF16?
| Quote: |
| What might be the steps to print out the special characters correctly (examples?)? |
Why do you need them to be printed out? Where do you want to view them?
| Quote: |
| Should I configure something? |
Only if you understand the problem, and you know what the 'something' will do. At this point, I'm not making any suggestions, because I don't yet understand the problem. |
|
| Back to top |
|
|
| mqmqmq |
| Posted: Tue Dec 16, 2008 10:36 pm Post subject: |
|
|
Novice
Joined: 02 Jul 2008
Posts: 22
|
Hi,
Thanks for your reply. Below you can find some answers to your answers/comments:
| Quote: |
Example: TestData1122. So the special characters are printed out as squeres.
Don't worry about that. It's probably a problem with your text viewer. It doesn't necessarily mean that your message flow is going wrong. |
Yes, I believe that my flow is OK. It is too simple to have some errors 
| Quote: |
I should put the data in xml file also into DB, but there is the same thing, the special characters are not printed correctly.
Once again, I can't work out whether your message flow is working properly, or whether you are asking about how to view the DB contents. |
My flow is OK. DB is OK. Only the data with special characters are not showing up corretly. I use SQL Developer to view the data in Oracle DB.
| Quote: |
| What might be the steps to print out the special characters correctly (examples?)? Why do you need them to be printed out? Where do you want to view them? |
My goals is to make sure that also special characters will be correctly inserted in DB and perhaps in log files.
| Quote: |
At this point, I'm not making any suggestions, because I don't yet understand the problem. |
The only problem that I have is that I have not seen correctly printed special characters anywhere.
If I execute INSERT statement in SQL Developer (and data has special characters), I can see those special characters when I execute SELECT statement. So the DB is ok. But when the data has gone through Broker flow (where the data is inserted in the same DB), I checked the DB with the same SELECT statement as above, I can see two types of data. Type one is with correct special characters (these are inserted via SQL Developer) and the type two those special characters are incorrect (these are inserted from Broker flow).
Thanks again!
BR,
--
mqmqmq |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Wed Dec 17, 2008 12:50 am Post subject: Re: Data conversion problem |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| mqmqmq wrote: |
| I send very simple xml file (see below) from rfhutilc to MQ Queue and read the file with one flow. Code page in Rfhutil is 437. |
Which ccsid was used to write the file? rfhutil should use the same ccsid to read it. If this is wrong, then the input message will be wrong, before it ever gets to the message flow.
| mqmqmq wrote: |
| Code: |
<?xml version="1.0" encoding="UTF-8"?>
<START>
<HEADER>TestData11ÄÅÖäåö22</HEADER>
</START> |
After file is handled by a flow I checked the logfile (mqsireadlog and mqsiformatlog). In the log file the data in HEADER -element has incorrect characters. |
mqsireadlog and mqsiformatlog often do their own character set conversions. For example, mqsireadlog may write a trace log file as UTF-8; mqsiformatlog would then read the trace log file as UTF-8, and may convert it to some other "native" character set. In that case, <HEADER> would look different in each of the two trace log files, and would appear to be "incorrect" in at least one of them.
Whatever tool that's used to read (or "print out") a file ought to be configured to use the same ccsid that was used to write it.
| mqmqmq wrote: |
| Example: TestData1122. So the special characters are printed out as squares. |
The X'001A' (substitute) character often indicates that a character could not be converted properly, as it doesn't exist in the target character set.
One way to work around display/printing issues is to see what <HEADER> looks like as a BLOB:
| Code: |
ESQL:
DECLARE b BLOB CAST(InputBody.START.HEADER AS BLOB CCSID InputRoot.Properties.CodedCharSetId);
mqsiformat trace log:
... UserTrace BIP2539I:
Evaluating expression ''CAST(InputBody.START.HEADER AS BLOB CCSID InputRoot.Properties.CodedCharSetId)''.
This resolved to ''CAST('TestData11ÄÅÖäåö22' AS BLOB CCSID 923 )''.
The result was ''X'54657374446174613131c4c5d6e4e5f63232'''. |
... and manually compare the hex to characters. I wouldn't recommend this approach for very long strings.
| mqmqmq wrote: |
| I have also tried to do the conversion in MQInput node (Properties->Advanced->Convert) but no luck so far. |
The MQInput node (almost) never needs to convert XML. The broker's parser is just going to convert it again anyway. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
