| Author |
Message
|
| vijeer2001 |
 Posted: Mon Mar 08, 2010 9:54 am Post subject: MQ - Having issue in processing special character |
 |
|
Novice
Joined: 08 Mar 2010
Posts: 16
|
Looking for help....
We have a custom .NET code to convert file received from interface on to MQ (Local Queue on Windows, to be read by IBM 3rd party application WDI for processing). The file has some specific special characters, which are getting converted when the IBM application is reading the data (Note: Most of the special characters are showing-up fine, just issue with few ones).
Alternatively IBM has provided one utility to put data on MQ (Q Utility), and this is giving correct results, but this utility is not supposed to be used as it is not supported by IBM.
Example:
Input File has character "Å " (Hex value 8A)
Using Custom .Net Code - Output has character "`" (Hex value 60)
Using Q Utility - Output has same character "Å " (Hex value 8A)
We have used MQFMT_STRING and MQCCSI_Q_MGR to make sure there is no conversion (again, here we are just putting to a local queue and not to remote queue, queue manager CCSID is 1208 and default encoding 546). We have to fix the .Net utility and cannot use Q utility.
Any leads to resolve this issue will be very helpful. |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Mon Mar 08, 2010 11:03 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| We have used MQFMT_STRING and MQCCSI_Q_MGR to make sure there is no conversion (again, here we are just putting to a local queue and not to remote queue |
This does not suppress conversion. Conversion can be specified on a sender channel definition, or the consuming application can request conversion.
| Quote: |
| queue manager CCSID is 1208 and default encoding 546). We have to fix the .Net utility and cannot use Q utility. |
Are the apps that create the message AND the apps that process/consume the message on the same qmgr?
Does the message traverse a network?
Is the consuming/processing application an MQ client app? If so, what is the CCSID of the client platform?
| Quote: |
| getting converted when the IBM application is reading the data |
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Mon Mar 08, 2010 11:27 am Post subject: Re: MQ - Having issue in processing special character |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| vijeer2001 wrote: |
Example:
Input File has character "Å " (Hex value 8A)
Using Custom .Net Code - Output has character "`" (Hex value 60)
Using Q Utility - Output has same character "Å " (Hex value 8A) |
The hex value associated with a character depends on the ccsid. For example:For ccsid=1252, "Å " has hex value X'8A'.
For ccsid=1208, "Å " has hex value X'C5A0'. It's helpful to keep track of the current ccsid associated with a message, as it moves from point to point.
| vijeer2001 wrote: |
| We have used MQFMT_STRING and MQCCSI_Q_MGR to make sure there is no conversion ... queue manager CCSID is 1208 ... |
Within a .NET application, it's the MQGET options, and choice of Read/Write methods that determine whether conversion takes place. Setting the message Format and CCSID does affect the behavior indirectly, but it's not sufficient to make sure there is no conversion.
| vijeer2001 wrote: |
| We have to fix the .Net utility |
For more help with that, post some code. |
|
| Back to top |
|
|
| vijeer2001 |
| Posted: Mon Mar 08, 2010 12:12 pm Post subject: |
|
|
Novice
Joined: 08 Mar 2010
Posts: 16
|
| Application that creates the message is in CZECH country. File is being FTPed from CZECH server (IBM-1250) encoding to windows server runing MQ mgr. Net program picks the file from the file system and loads it to MQ using CCSID of 1208. Processing application is using MQ server with CCSID of 1208. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Mar 08, 2010 1:47 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Is the character you are having an issue with being corrupted by FTP? Look at the FTP'd file before you run the app that creates messages from it.
| Quote: |
| . Net program picks the file from the file system and loads it to MQ using CCSID of 1208. |
How does this app do the conversion?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| vijeer2001 |
| Posted: Mon Mar 08, 2010 3:36 pm Post subject: |
|
|
Novice
Joined: 08 Mar 2010
Posts: 16
|
[/quote]Example:
Input File has character "Å " (Hex value 8A)
Using Custom .Net Code - Output has character "`" (Hex value 60)
Using Q Utility - Output has same character "Å " (Hex value 8A)
| Quote: |
We are doing the comparison after FTP is complete. Input file character is the value shown in the windows system before .Net program picks up the data. After using the .net program, MQ shows the character as "`" (Hex value 60). We wanted to know how can stop this character conversion? |
|
|
| Back to top |
|
|
| vijeer2001 |
| Posted: Mon Mar 08, 2010 7:17 pm Post subject: |
|
|
Novice
Joined: 08 Mar 2010
Posts: 16
|
More insight...
We are recieving data from european country using FTP (Windows-1250 system) on our Windows server.
We verified the character with the interface and it is actually "Å " and therefore no curruption on FTP transfer.
Now, we use the .NET code to MQPUT this file onto a listen queue, here the both MQPUT and MQGET operations are happenning on the same Queue Manager (CCSID 1208). MQGET is not visible to us as it is done by IBM WDI application.
The fact that there is no sender channel involved and that we are using the same CCSID as Queue Manager, no conversion is expected. Following are the options which we are using:
mqMsg.CharacterSet = MQC.MQCCSI_Q_MGR;
mqMsg.Format = MQC.MQFMT_STRING;
We are not using a client channel here as we are on the same server where Queue Manager is installed.
As mentioned before, IBM Q Utility is working fine. We are unable to find where we are going wrong.
I can attach the code and data examples, not sure how to do it on this forum as there is no option to attch. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Mar 08, 2010 7:40 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| vijeer2001 wrote: |
| Now, we use the .NET code to MQPUT this file onto a listen queue, here the both MQPUT and MQGET operations are happenning on the same Queue Manager (CCSID 1208). |
And what is the hex value of the character in the message on the queue after the MQPUT?
| vijeer2001 wrote: |
| As mentioned before, IBM Q Utility is working fine. We are unable to find where we are going wrong. |
What is the key difference between what works and what doesn't? Hint: the Q utility is written in C not .NET
| vijeer2001 wrote: |
| I can attach the code and data examples, not sure how to do it on this forum as there is no option to attch. |
The forum does not allow the attachment of files. You can post samples; please use the code tags to aid readability.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| vijeer2001 |
| Posted: Mon Mar 08, 2010 10:14 pm Post subject: |
|
|
Novice
Joined: 08 Mar 2010
Posts: 16
|
Input File has character "Å " (Hex value 8A)
Using Custom .Net Code - MQ has character "`" (Hex value 60)
Using Q Utility - MQ has same character "Å " (Hex value 8A)
We could see that the value is changing on MQ only, as just before issuing the WriteBytes commnad, the .Net variable has the right character. Following is the code snippet doing the job:
| Code: |
mqMsg = new MQMessage();
mqMsg.MessageFlags = MQC.MQMF_LAST_SEGMENT;
mqMsg.CharacterSet = MQC.MQCCSI_Q_MGR;
mqMsg.Format = MQC.MQFMT_STRING;
mqMsg.WriteBytes(alMessage[intIdx].ToString()); |
Following is the input file snippet:
| Code: |
| LINC0544 Roman Mazák - NUGET Prùmyslová 3062/5 Šumperk 787 01 |
Following the output file snippet:
| Code: |
| LINC0544 Roman Mazák - NUGET Prùmyslová 3062/5 `umperk 787 01 |
See the difference between the lines. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Mar 09, 2010 3:36 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| You should a) set the CCSID on the message before you write bytes to it, b) make sure that ToString() isn't changing the characters, c) make sure that you should really be writing bytes instead of characters, d) make sure you know that passing ToString to WriteBytes() is the right thing to do. |
|
| Back to top |
|
|
| vijeer2001 |
| Posted: Tue Mar 09, 2010 5:44 am Post subject: |
|
|
Novice
Joined: 08 Mar 2010
Posts: 16
|
We are writing the CCSID (i.e. MQCCSI_Q_MGR) before WriteBytes, plus we checked ToString() that it does have the right characters.
| Code: |
mqMsg = new MQMessage();
mqMsg.MessageFlags = MQC.MQMF_LAST_SEGMENT;
mqMsg.CharacterSet = MQC.MQCCSI_Q_MGR;
mqMsg.Format = MQC.MQFMT_STRING;
mqMsg.WriteBytes(alMessage[intIdx].ToString()); |
|
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Mar 09, 2010 7:21 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Is the CCSID of the queue manager the CCSID of the DATA? |
|
| Back to top |
|
|
| vijeer2001 |
| Posted: Tue Mar 09, 2010 8:35 am Post subject: |
|
|
Novice
Joined: 08 Mar 2010
Posts: 16
|
Yes...
Summary:
Interface sends data file using FTP, all characters verified that they have been recieved correctly (file contains many european characters, sent from IBM-1250 system to our Windows Server).
.Net application used to put the file on MQ. Data read correctly by .Net app, but when put on MQ, some of the special characters gets converted. Expectation is that none should get converted.. Example, "Å " (Hex value 8A) gets converted to "`" (Hex value 60).
We are putting on a Local Queue and QMGR CCSID is 1208. The application reading data is IBM WDI Tool, reading from the same local Queue on same Queue Manager.
Alternatoively, using IBM Q uility, there is no conversion all special charcters are fine. But we can't use it and .Net code needs to be fixed.
Refer to chain for more details. |
|
| Back to top |
|
|
| vijeer2001 |
| Posted: Tue Mar 09, 2010 8:39 am Post subject: |
|
|
Novice
Joined: 08 Mar 2010
Posts: 16
|
Correction:
The data CCSID of the data is 1250, but as we do not want any conversion on queue, we intentionally pass CCSID as MQCCSI_Q_MGR i.e. 1208. We even tried using 1250 as mqMsg.CharacterSet, but this also gave same result.
Summary:
Interface sends data file using FTP, all characters verified that they have been recieved correctly (file contains many european characters, sent from IBM-1250 system to our Windows Server).
.Net application used to put the file on MQ. Data read correctly by .Net app, but when put on MQ, some of the special characters gets converted. Expectation is that none should get converted.. Example, "Å " (Hex value 8A) gets converted to "`" (Hex value 60).
We are putting on a Local Queue and QMGR CCSID is 1208. The application reading data is IBM WDI Tool, reading from the same local Queue on same Queue Manager.
Alternatoively, using IBM Q uility, there is no conversion all special charcters are fine. But we can't use it and .Net code needs to be fixed.
Refer to chain for more details. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Mar 09, 2010 1:56 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Warning -- if possible do not use default CCSID's as the default has been known to change depending on version of MQ.
You would probably be better off to set your CCSID explicitely on the message. Also make sure that the content you pass to the message is in the correct CCSID.
This should solve your problems.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
