ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Encoding and CCSID

Post new topic  Reply to topic Goto page 1, 2  Next
 Encoding and CCSID « View previous topic :: View next topic » 
Author Message
akkypaul
PostPosted: Wed Feb 03, 2016 3:47 am    Post subject: Encoding and CCSID Reply with quote

Apprentice

Joined: 30 Jun 2014
Posts: 44
Location: India

Here is the problem
- Extracting Data from Database
- Using an XML file as an input to trigger the interface
- Using ASBITSTREAM function which requires the Encoding and CCSID from the properties of InputRoot to convert the Environment Variable message tree containing the database values to BLOB
- But I have using InputRoot's Encoding and CCSID.
- At MQ Input, the parser is XMLNSC
- This is the error that I am getting: PROPAGATE FINALIZE DEFAULT DELETE NONE; Caught exception and rethrowing : XXXX.XXX.sap.XXXXXX.Transform Error detected, rethrowing : XXXX.XXX.sap.XXXXXX_Transform.Main 11.3 SET outputMsgBlob = ASBITSTREAM(OutputRoot.DataObject ENCODING InputRoot.Properties.Encoding CCSID InputRoot.Properties.CodedCharSetId); XML Writing Errors have occurred : Unexpected XML type at this point in document. : SapPexr2002 folderType : Flow
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Feb 03, 2016 5:26 am    Post subject: Re: Encoding and CCSID Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

akkypaul wrote:
- Extracting Data from Database
- Using an XML file as an input to trigger the interface


So you get an XML document and on receipt of that extract data from the database?

akkypaul wrote:
Using ASBITSTREAM function which requires the Encoding and CCSID from the properties of InputRoot to convert the Environment Variable message tree containing the database values to BLOB


Why?

akkypaul wrote:
But I have using InputRoot's Encoding and CCSID.


Why? Do they match the database?

akkypaul wrote:
At MQ Input, the parser is XMLNSC


So you said

akkypaul wrote:
This is the error that I am getting: PROPAGATE FINALIZE DEFAULT DELETE NONE; Caught exception and rethrowing : XXXX.XXX.sap.XXXXXX.Transform Error detected, rethrowing : XXXX.XXX.sap.XXXXXX_Transform.Main 11.3 SET outputMsgBlob = ASBITSTREAM(OutputRoot.DataObject ENCODING InputRoot.Properties.Encoding CCSID InputRoot.Properties.CodedCharSetId); XML Writing Errors have occurred : Unexpected XML type at this point in document. : SapPexr2002 folderType : Flow


Why is this statement talking about SAP & DataObjects when this post does not?

What exactly are you trying to PROPAGATE? Specifically what IIB domain is the last child of Root at that point?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
smdavies99
PostPosted: Wed Feb 03, 2016 5:38 am    Post subject: Reply with quote

Jedi Council

Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.

Have you....

1) Put a trace node BEFORE the MQOutput Node
2) set the output pattern to be ${Root}
3) enabled usertrace on the flow
4) Passed a message through th flow.
5) disables usertrace, extracted the trace and formatted it?

Then looked at your message tree before it goes into the MQOutput Node.

You never know, it might be enlightening and even help you solve your problem.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995

Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Back to top
View user's profile Send private message
akkypaul
PostPosted: Wed Feb 03, 2016 7:38 am    Post subject: Reply with quote

Apprentice

Joined: 30 Jun 2014
Posts: 44
Location: India

Gentlemen,

Please get enlightened with the following facts:-

1) The source system triggers an XML message which is of no use but to trigger the flow to start extracting the rows from Database.

2) Values of the database that are extracted are stored in Environment.Variables tree. For the sake of auditing, I need to record the data extracted from Database in Environment.Variables Tree into a column of another table. This is why I am using ASBITSTREAM.

SET outputMsgBlob = ASBITSTREAM(OutputRoot.DataObject ,InputRoot.Properties.Encoding,InputRoot.Properties.CodedCharSetId);
SET outputMsgChar = CAST(outputMsgBlob AS CHAR CCSID InputRoot.Properties.CodedCharSetId);

3) Any finally, about the DataObjet: that's because the output node is not MQOutput node but a SAP Request node from where it is sent to the SAP system.

4) PROPAGATE statement is used because I want a customized propagation of the message.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Wed Feb 03, 2016 7:42 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

How do you know that the CCSID of the MQ message matches the CCSID of the data in the table?

Why are you using an MQ trigger, versus a DatabaseInput node?
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
akkypaul
PostPosted: Wed Feb 03, 2016 7:55 am    Post subject: Reply with quote

Apprentice

Joined: 30 Jun 2014
Posts: 44
Location: India

I do not know the jeff which exactly is my question.
I know that it's wrong.
I want to know that what should be the CCSID in such cases.
I just wrote InputRoot's CCSID to see if it works but I guess, I will have to manually code in something like 1208
And I am using MQ Trigger because they do not want constant monitoring of the database and the access to database is provided by the source system only for some time after triggering that XML message.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed Feb 03, 2016 8:10 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

akkypaul wrote:
I do not know the jeff which exactly is my question.
I know that it's wrong.
I want to know that what should be the CCSID in such cases.
I just wrote InputRoot's CCSID to see if it works but I guess, I will have to manually code in something like 1208

The CCSID always need to adequately describe the data.
So if the Data is in UTF-8 use CCSID 1208...
Setting the CCSID on OutputRoot.Properties and OutputRoot.MQMD will do that for you. The parser will then take care of serializing the output in the correct CCSID. If you use the BLOB parser, you'll need to make sure the data is in the correct CCSID as no conversion is done there.
akkypaul wrote:
And I am using MQ Trigger because they do not want constant monitoring of the database and the access to database is provided by the source system only for some time after triggering that XML message.
That is an effective reason for using an the XML message as a trigger...
Have you given some thought as to what happens if you get a poison entry in the DB???
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
mqjeff
PostPosted: Wed Feb 03, 2016 8:10 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Typically, any data populated in a logical message tree will be in 1200... (waits for tkimber to correct)

Also, it's a very common error to try and store XML data in the Environment tree without taking the necessary steps to prepare the Environment tree to handle XMLNSC data properly.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Feb 03, 2016 8:13 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

akkypaul wrote:
I want to know that what should be the CCSID in such cases.


The code page your database is using, or the code page used for the result set returned to IIB (if different).
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
akkypaul
PostPosted: Wed Feb 03, 2016 11:38 am    Post subject: Reply with quote

Apprentice

Joined: 30 Jun 2014
Posts: 44
Location: India

Quote:
The CCSID always need to adequately describe the data.
So if the Data is in UTF-8 use CCSID 1208...
Setting the CCSID on OutputRoot.Properties and OutputRoot.MQMD will do that for you. The parser will then take care of serializing the output in the correct CCSID. If you use the BLOB parser, you'll need to make sure the data is in the correct CCSID as no conversion is done there.


Yes. I could use the OutputRoot.Properties and OutputRoot.MQMD if I were to send that resulting data to MQ Output Node. But, I need to pass it to a function that accepts BLOB as input.

Quote:
Have you given some thought as to what happens if you get a poison entry in the DB???


What exactly to you mean by poison entry?
Does it mean incorrect entry?
Well, I don't have to worry about that.


Quote:
Also, it's a very common error to try and store XML data in the Environment tree without taking the necessary steps to prepare the Environment tree to handle XMLNSC data properly.

I am not storing the XML Data anywhere. I am storing the data from Database in Environment.Variable


Quote:
The code page your database is using, or the code page used for the result set returned to IIB (if different).

I am not using IIB. I am using MB v8
It does not matters too much but I need to know about the encoding. What can be the encoding for such data that has been derived out of Database
Back to top
View user's profile Send private message
Vitor
PostPosted: Wed Feb 03, 2016 1:08 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

akkypaul wrote:
Quote:
The code page your database is using, or the code page used for the result set returned to IIB (if different).

I am not using IIB. I am using MB v8


Well you didn't say but it doesn't matter in the sense of the issue.

akkypaul wrote:
It does not matters too much but I need to know about the encoding.


The code page doesn't matter too much??

Also if you're worried about encoding (i.e. the existence of packed data) how does that work in the BLOB? How does the target (which you're propagating to) handle that?

akkypaul wrote:
What can be the encoding for such data that has been derived out of Database


Any valid value for that database software. Same as for the possible values for code page.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
akkypaul
PostPosted: Wed Feb 03, 2016 5:18 pm    Post subject: Reply with quote

Apprentice

Joined: 30 Jun 2014
Posts: 44
Location: India

The BLOB is later converted to a CHARACTER string which is sent to a function.
I am only worried about encoding because it is one of the parameters of ASBITSTREAM function that I have to use.
The function that this CHARACTER string is sent to is hidden to me so that is not important.
What are some of the valid values for code page?
Is 1208 good enough?
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Thu Feb 04, 2016 5:37 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

akkypaul wrote:
The BLOB is later converted to a CHARACTER string which is sent to a function.
I am only worried about encoding because it is one of the parameters of ASBITSTREAM function that I have to use.
The function that this CHARACTER string is sent to is hidden to me so that is not important.
What are some of the valid values for code page?
Is 1208 good enough?

Wrong question to ask....
The questions to ask yourself is:
  1. What is the CCSID of the Data in the DB?
  2. How is that data extracted (BLOB, CLOB, VARCHAR, fields, ...
  3. Is the CCSID still relevant once the data is in WMB: i.e. what is the form of the data in the result set.
  4. Does it [the data] need to be manipulated before output
  5. What form does the output have: serialized by a parser or as BLOB

Once you have an answer to these questions, you'll know what to do...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
joebuckeye
PostPosted: Thu Feb 04, 2016 6:32 am    Post subject: Reply with quote

Partisan

Joined: 24 Aug 2007
Posts: 364
Location: Columbus, OH

We can't tell you what the "correct" CCSID or encoding is because that is determined by who is generating the data you are using.

This link is a good starting point for understanding CCSID and encoding.

http://www.joelonsoftware.com/articles/Unicode.html
Back to top
View user's profile Send private message
akkypaul
PostPosted: Fri Oct 18, 2019 5:05 am    Post subject: Reply with quote

Apprentice

Joined: 30 Jun 2014
Posts: 44
Location: India

joebuckeye wrote:
We can't tell you what the "correct" CCSID or encoding is because that is determined by who is generating the data you are using.

This link is a good starting point for understanding CCSID and encoding.

http://www.joelonsoftware.com/articles/Unicode.html


Thank you! This was insightful!
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Encoding and CCSID
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.