| Author |
Message
|
| akil |
 Posted: Sat Nov 21, 2015 9:36 pm Post subject: |
 |
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
Is there a way to instruct JSON to parse all integers to DECIMAL, including values with no precision?
I ask this because, IBM WL Javascript, strips away precision when it serialises a float ( 1.00 is sent as 1 ) which is then interpreted as a INTEGER, and then the subsequent ESQL / XSD fail with CAST / Validation exceptions, as they expect a DECIMAL
_________________
Regards |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Sat Nov 21, 2015 11:36 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
time for a PMR?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| kimbert |
| Posted: Sun Nov 22, 2015 1:08 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
I have to agree - the auto-casting behaviour of the JSON parser seems to rely on things that the developer cannot control ( e.g. behaviour of upstream applications ).
Having said that, I don't understand akil's last point:
| Quote: |
| IBM WL Javascript, strips away precision when it serialises a float ( 1.00 is sent as 1 ) which is then interpreted as a INTEGER, and then the subsequent ESQL / XSD fail with CAST / Validation exceptions, as they expect a DECIMAL |
Why does the ESQL fail? ESQL is normally very tolerant, and will automatically handle any compatible input type. Furthermore, it is quite difficult to determine which numeric type ESQL is using ( FIELDTYPE does not return that, contrary to some peoples' expectations ). So what is it that is being tripped up by these unexpected INTEGERs?
_________________
Before you criticize someone, walk a mile in their shoes. That way you're a mile away, and you have their shoes too. |
|
| Back to top |
|
|
| stoney |
| Posted: Mon Nov 23, 2015 1:42 am Post subject: |
|
|
Centurion
Joined: 03 Apr 2013
Posts: 140
|
| ghoshly wrote: |
We even tried after restarting the broker. Broker user profile is also show showing the MQSI variable, but the JSON parser is not working as expected in AIX environment.
Do we need to restart the server? Please help how we can debug the reason of not working. |
Did you do this? Did you remember the "export"?
| Code: |
mqsistop IB10NODE
export MQSI_JSON_NUMBER_PRECISION_TYPE=DECIMAL
mqsistart IB10NODE |
You can check the running environment variables for your DataFlowEngine process on AIX by running the following command against the DataFlowEngine process ID - check to see if MQSI_JSON_NUMBER_PRECISION_TYPE is listed:
| Code: |
| ps ewww <process ID> |
Sometimes exporting the environment variable from your user ID isn't enough.
If your broker is started as an MQ service, the broker is started under a different environment and possibly even a different user ID.
You can check to see if your broker is started as an MQ service by running mqsireportbroker:
| Code: |
mqsireportbroker IB10NODE
...
Start as WebSphere MQ Service = 'defined' |
If it states "defined" then the easiest thing to do is create a custom profile under /var/mqsi/common/profiles.
For example, create the file /var/mqsi/common/profiles/json_decimals.sh and give it the following content:
| Code: |
#!/bin/sh
export MQSI_JSON_NUMBER_PRECISION_TYPE=DECIMAL |
Save the file, log off, log back on, run mqsiprofile, and then restart the broker.
When starting the broker as an MQ service, mqsiprofile and that file will get automatically run by MQ. |
|
| Back to top |
|
|
| stoney |
| Posted: Mon Nov 23, 2015 1:46 am Post subject: |
|
|
Centurion
Joined: 03 Apr 2013
Posts: 140
|
| akil wrote: |
Is there a way to instruct JSON to parse all integers to DECIMAL, including values with no precision?
I ask this because, IBM WL Javascript, strips away precision when it serialises a float ( 1.00 is sent as 1 ) which is then interpreted as a INTEGER, and then the subsequent ESQL / XSD fail with CAST / Validation exceptions, as they expect a DECIMAL |
Apparently so!
| Code: |
MQSI_JSON_NUMBERFORMATPARSING_OFF=1
MQSI_JSON_NUMBER_PRECISION_TYPE=DECIMAL |
But I agree with the others concerns - what exactly is failing to cope with INTEGERs where DECIMALs are expected? |
|
| Back to top |
|
|
| ghoshly |
| Posted: Mon Nov 23, 2015 6:55 am Post subject: environment variable |
|
|
Partisan
Joined: 10 Jan 2008
Posts: 325
|
Hello... Just to confirm we are still using WMB V8.0.0.6 not IIB.
Our broker is defined to start as MQ service. I want to show the output of the command.
| Quote: |
tghosh@sdxbf-m23n6 [/u/tghosh]> ps -ef | grep DataFlow | grep JSON
mqm 7340186 6881346 0 Nov 21 - 0:00 /bin/ksh /opt/esb/mqsi/8.0/bin/startDataFlowEngine MBIRS01 a54bda66-5001-0000-0080-c910b1de8628 ICE_JSON_GATEWAY
mqm 18481258 7340186 0 Nov 21 - 1:02 DataFlowEngine MBIRS01 a54bda66-5001-0000-0080-c910b1de8628 ICE_JSON_GATEWAY
|
| Quote: |
tghosh@sdxbf-m23n6 [/u/tghosh]> ps ewww 18481258
PID TTY STAT TIME COMMAND
18481258 - A 1:02 DataFlowEngine MBIRS01 a54bda66-5001-0000-0080-c910b1de8628 ICE_JSON_GATEWAY
|
I don't see the variable listed via this command. Does this show its not setup properly? |
|
| Back to top |
|
|
| stoney |
| Posted: Mon Nov 23, 2015 7:10 am Post subject: Re: environment variable |
|
|
Centurion
Joined: 03 Apr 2013
Posts: 140
|
| ghoshly wrote: |
| I don't see the variable listed via this command. Does this show its not setup properly? |
No - you have to run the command as the user ID that owns the process (mqm) or as root, as your user ID (tghosh) does not have access to the environment variables of the DataFlowEngine process. |
|
| Back to top |
|
|
| ghoshly |
| Posted: Mon Nov 23, 2015 8:10 am Post subject: Thanks a lot.. |
|
|
Partisan
Joined: 10 Jan 2008
Posts: 325
|
Thanks a lot Kimbert and Stoney for your extended help with patience. Finally we can see as
| Quote: |
(0x01000000:Object):POLineItemsDetail = (
(0x01001000:Array):POLineItem = (
(0x01000000:Object):Item = (
(0x03000000:NameValue):Price = 13.75 (DECIMAL)
(0x03000000:NameValue):Currency = '' (CHARACTER)
(0x03000000:NameValue):Quantity = 0 (INTEGER)
|
I am sorry for my limited knowledge. |
|
| Back to top |
|
|
| akil |
| Posted: Mon Nov 23, 2015 10:31 am Post subject: |
|
|
Partisan
Joined: 27 May 2014
Posts: 338
Location: Mumbai
|
Hi
What is failing for us is taking the value from Json and passing it to an external esql database ( oracle ) function call. The Integer from Json requires an explicit cast to a float ( the datatype for an oracle number ) during the invocation.
Regards
_________________
Regards |
|
| Back to top |
|
|
| mrlizard |
| Posted: Tue Oct 24, 2017 4:42 am Post subject: |
|
|
Newbie
Joined: 03 Jul 2017
Posts: 5
|
I'm on IIB 10.0.0.7.
I voted for RFE 82248 (Parser option for decimal numbers in JSON) since I have experienced this undesirable/implicit FLOAT casting in the JSON parser which messes up my arithmetic if I don't cast it to decimal.
I'm also hoping that improved support for Swagger and JSON parsing downstream, might help with this, but looking at the latest swagger spec v3.0, I can still only see a JSON Number type which doesn't support a decimal format. So, it would be good if IBM could be more transparent about their direction in this space and suggest best practice (short terms/long term) as I am also considering implementing these variables in the meantime as a temporary enhancement.
MQSI_JSON_NUMBERFORMATPARSING_OFF=1
MQSI_JSON_NUMBER_PRECISION_TYPE=DECIMAL
In the XML world, we are equipped to deal with this using the XMLNSC parser option "Build tree using XML schema data types". Are IBM planning to do something similar with Swagger? When are they planning to release that enhancement? Version 11? |
|
| Back to top |
|
|
| souciance |
| Posted: Tue Oct 24, 2017 10:36 am Post subject: |
|
|
Disciple
Joined: 29 Jun 2010
Posts: 169
|
| If you are comfortable with java then there are various frameworks that validate against a json schema. Include the framework in a jcn and do the validation there instead of transforming it to an XML. |
|
| Back to top |
|
|
| timber |
| Posted: Tue Oct 24, 2017 10:52 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1280
|
@mrlizard: I am sympathetic to the requests made here; the current default behaviour does appear to be unexpected for many users.
However...IBM is complying with the JSON specification, which does not have a decimal data type. Support for JSON validation will not change that. |
|
| Back to top |
|
|
| timber |
| Posted: Tue Oct 24, 2017 10:55 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1280
|
@souciance: Be aware that JSON validation is nowhere near as carefully-specified as XML validation is. Implementations will probably always vary. So a message that validates correctly at the sender's end won't necessarily validate at your end. Or vice versa.
You may be OK with that situation, but best to go in with your eyes open. |
|
| Back to top |
|
|
|
|
