| Author |
Message
|
| akkypaul |
 Posted: Wed Feb 03, 2016 3:47 am Post subject: Encoding and CCSID |
 |
|
Apprentice
Joined: 30 Jun 2014
Posts: 44
Location: India
|
Here is the problem
- Extracting Data from Database
- Using an XML file as an input to trigger the interface
- Using ASBITSTREAM function which requires the Encoding and CCSID from the properties of InputRoot to convert the Environment Variable message tree containing the database values to BLOB
- But I have using InputRoot's Encoding and CCSID.
- At MQ Input, the parser is XMLNSC
- This is the error that I am getting: PROPAGATE FINALIZE DEFAULT DELETE NONE; Caught exception and rethrowing : XXXX.XXX.sap.XXXXXX.Transform Error detected, rethrowing : XXXX.XXX.sap.XXXXXX_Transform.Main 11.3 SET outputMsgBlob = ASBITSTREAM(OutputRoot.DataObject ENCODING InputRoot.Properties.Encoding CCSID InputRoot.Properties.CodedCharSetId); XML Writing Errors have occurred : Unexpected XML type at this point in document. : SapPexr2002 folderType : Flow |
|
| Back to top |
|
 |
| Vitor |
| Posted: Wed Feb 03, 2016 5:26 am Post subject: Re: Encoding and CCSID |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akkypaul wrote: |
- Extracting Data from Database
- Using an XML file as an input to trigger the interface |
So you get an XML document and on receipt of that extract data from the database?
| akkypaul wrote: |
| Using ASBITSTREAM function which requires the Encoding and CCSID from the properties of InputRoot to convert the Environment Variable message tree containing the database values to BLOB |
Why?
| akkypaul wrote: |
| But I have using InputRoot's Encoding and CCSID. |
Why? Do they match the database?
| akkypaul wrote: |
| At MQ Input, the parser is XMLNSC |
So you said
| akkypaul wrote: |
| This is the error that I am getting: PROPAGATE FINALIZE DEFAULT DELETE NONE; Caught exception and rethrowing : XXXX.XXX.sap.XXXXXX.Transform Error detected, rethrowing : XXXX.XXX.sap.XXXXXX_Transform.Main 11.3 SET outputMsgBlob = ASBITSTREAM(OutputRoot.DataObject ENCODING InputRoot.Properties.Encoding CCSID InputRoot.Properties.CodedCharSetId); XML Writing Errors have occurred : Unexpected XML type at this point in document. : SapPexr2002 folderType : Flow |
Why is this statement talking about SAP & DataObjects when this post does not?
What exactly are you trying to PROPAGATE? Specifically what IIB domain is the last child of Root at that point?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Feb 03, 2016 5:38 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Have you....
1) Put a trace node BEFORE the MQOutput Node
2) set the output pattern to be ${Root}
3) enabled usertrace on the flow
4) Passed a message through th flow.
5) disables usertrace, extracted the trace and formatted it?
Then looked at your message tree before it goes into the MQOutput Node.
You never know, it might be enlightening and even help you solve your problem.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| akkypaul |
| Posted: Wed Feb 03, 2016 7:38 am Post subject: |
|
|
Apprentice
Joined: 30 Jun 2014
Posts: 44
Location: India
|
Gentlemen,
Please get enlightened with the following facts:-
1) The source system triggers an XML message which is of no use but to trigger the flow to start extracting the rows from Database.
2) Values of the database that are extracted are stored in Environment.Variables tree. For the sake of auditing, I need to record the data extracted from Database in Environment.Variables Tree into a column of another table. This is why I am using ASBITSTREAM.
SET outputMsgBlob = ASBITSTREAM(OutputRoot.DataObject ,InputRoot.Properties.Encoding,InputRoot.Properties.CodedCharSetId);
SET outputMsgChar = CAST(outputMsgBlob AS CHAR CCSID InputRoot.Properties.CodedCharSetId);
3) Any finally, about the DataObjet: that's because the output node is not MQOutput node but a SAP Request node from where it is sent to the SAP system.
4) PROPAGATE statement is used because I want a customized propagation of the message. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Feb 03, 2016 7:42 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
How do you know that the CCSID of the MQ message matches the CCSID of the data in the table?
Why are you using an MQ trigger, versus a DatabaseInput node?
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| akkypaul |
| Posted: Wed Feb 03, 2016 7:55 am Post subject: |
|
|
Apprentice
Joined: 30 Jun 2014
Posts: 44
Location: India
|
I do not know the jeff which exactly is my question.
I know that it's wrong.
I want to know that what should be the CCSID in such cases.
I just wrote InputRoot's CCSID to see if it works but I guess, I will have to manually code in something like 1208
And I am using MQ Trigger because they do not want constant monitoring of the database and the access to database is provided by the source system only for some time after triggering that XML message. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Feb 03, 2016 8:10 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| akkypaul wrote: |
I do not know the jeff which exactly is my question.
I know that it's wrong.
I want to know that what should be the CCSID in such cases.
I just wrote InputRoot's CCSID to see if it works but I guess, I will have to manually code in something like 1208 |
The CCSID always need to adequately describe the data.
So if the Data is in UTF-8 use CCSID 1208...
Setting the CCSID on OutputRoot.Properties and OutputRoot.MQMD will do that for you. The parser will then take care of serializing the output in the correct CCSID. If you use the BLOB parser, you'll need to make sure the data is in the correct CCSID as no conversion is done there.
| akkypaul wrote: |
| And I am using MQ Trigger because they do not want constant monitoring of the database and the access to database is provided by the source system only for some time after triggering that XML message. |
That is an effective reason for using an the XML message as a trigger...
Have you given some thought as to what happens if you get a poison entry in the DB??? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Feb 03, 2016 8:10 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Typically, any data populated in a logical message tree will be in 1200... (waits for tkimber to correct)
Also, it's a very common error to try and store XML data in the Environment tree without taking the necessary steps to prepare the Environment tree to handle XMLNSC data properly.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 03, 2016 8:13 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akkypaul wrote: |
| I want to know that what should be the CCSID in such cases. |
The code page your database is using, or the code page used for the result set returned to IIB (if different).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| akkypaul |
| Posted: Wed Feb 03, 2016 11:38 am Post subject: |
|
|
Apprentice
Joined: 30 Jun 2014
Posts: 44
Location: India
|
| Quote: |
The CCSID always need to adequately describe the data.
So if the Data is in UTF-8 use CCSID 1208...
Setting the CCSID on OutputRoot.Properties and OutputRoot.MQMD will do that for you. The parser will then take care of serializing the output in the correct CCSID. If you use the BLOB parser, you'll need to make sure the data is in the correct CCSID as no conversion is done there. |
Yes. I could use the OutputRoot.Properties and OutputRoot.MQMD if I were to send that resulting data to MQ Output Node. But, I need to pass it to a function that accepts BLOB as input.
| Quote: |
| Have you given some thought as to what happens if you get a poison entry in the DB??? |
What exactly to you mean by poison entry?
Does it mean incorrect entry?
Well, I don't have to worry about that.
| Quote: |
| Also, it's a very common error to try and store XML data in the Environment tree without taking the necessary steps to prepare the Environment tree to handle XMLNSC data properly. |
I am not storing the XML Data anywhere. I am storing the data from Database in Environment.Variable
| Quote: |
| The code page your database is using, or the code page used for the result set returned to IIB (if different). |
I am not using IIB. I am using MB v8
It does not matters too much but I need to know about the encoding. What can be the encoding for such data that has been derived out of Database |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 03, 2016 1:08 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| akkypaul wrote: |
| Quote: |
| The code page your database is using, or the code page used for the result set returned to IIB (if different). |
I am not using IIB. I am using MB v8 |
Well you didn't say but it doesn't matter in the sense of the issue.
| akkypaul wrote: |
| It does not matters too much but I need to know about the encoding. |
The code page doesn't matter too much??
Also if you're worried about encoding (i.e. the existence of packed data) how does that work in the BLOB? How does the target (which you're propagating to) handle that?
| akkypaul wrote: |
| What can be the encoding for such data that has been derived out of Database |
Any valid value for that database software. Same as for the possible values for code page.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| akkypaul |
| Posted: Wed Feb 03, 2016 5:18 pm Post subject: |
|
|
Apprentice
Joined: 30 Jun 2014
Posts: 44
Location: India
|
The BLOB is later converted to a CHARACTER string which is sent to a function.
I am only worried about encoding because it is one of the parameters of ASBITSTREAM function that I have to use.
The function that this CHARACTER string is sent to is hidden to me so that is not important.
What are some of the valid values for code page?
Is 1208 good enough? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Feb 04, 2016 5:37 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| akkypaul wrote: |
The BLOB is later converted to a CHARACTER string which is sent to a function.
I am only worried about encoding because it is one of the parameters of ASBITSTREAM function that I have to use.
The function that this CHARACTER string is sent to is hidden to me so that is not important.
What are some of the valid values for code page?
Is 1208 good enough? |
Wrong question to ask....
The questions to ask yourself is:
- What is the CCSID of the Data in the DB?
- How is that data extracted (BLOB, CLOB, VARCHAR, fields, ...
- Is the CCSID still relevant once the data is in WMB: i.e. what is the form of the data in the result set.
- Does it [the data] need to be manipulated before output
- What form does the output have: serialized by a parser or as BLOB
Once you have an answer to these questions, you'll know what to do...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| joebuckeye |
| Posted: Thu Feb 04, 2016 6:32 am Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 365
Location: Columbus, OH
|
|
| Back to top |
|
|
| akkypaul |
| Posted: Fri Oct 18, 2019 5:05 am Post subject: |
|
|
Apprentice
Joined: 30 Jun 2014
Posts: 44
Location: India
|
| joebuckeye wrote: |
We can't tell you what the "correct" CCSID or encoding is because that is determined by who is generating the data you are using.
This link is a good starting point for understanding CCSID and encoding.
http://www.joelonsoftware.com/articles/Unicode.html |
Thank you! This was insightful!  |
|
| Back to top |
|
|
|
|
