Author |
Message
|
souciance |
Posted: Mon Oct 23, 2017 12:29 am Post subject: JSON integer value converted to scientific notation |
|
|
Disciple
Joined: 29 Jun 2010 Posts: 169
|
Hello,
We are using the REST API in IIB 10. We call the REST API who fetches a JSON message from another system. IIB seems to be converting a json integer value to scientific notation. This is what the IIB is sending back "DocumentId":4.143E+4. But the sending system is sending "DocumentId":41430. We cannot do any casting here either since everything is done in a generic loop and there many fields in this list.
Why is the format of the integer value being changed? Anything we can do to prevent this from happening? |
|
Back to top |
|
 |
souciance |
Posted: Mon Oct 23, 2017 1:15 am Post subject: |
|
|
Disciple
Joined: 29 Jun 2010 Posts: 169
|
Just to clarify, we use
Code: |
FIELDVALUE(inputField) |
function to get the value of the input field and set it to the output field. In this process it seems a field defined as an integer in JSON is auto converted to scientific notation. |
|
Back to top |
|
 |
fjb_saper |
Posted: Mon Oct 23, 2017 6:08 am Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
What do you mean, you cannot do any casting?...
Are you even using the IIB JSON parser?
You should be able to inspect the value returned, determine it's type, and handle it appropriately in your loop.
Have fun  _________________ MQ & Broker admin |
|
Back to top |
|
 |
Vitor |
Posted: Mon Oct 23, 2017 6:14 am Post subject: |
|
|
 Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
fjb_saper wrote: |
What do you mean, you cannot do any casting?...
Are you even using the IIB JSON parser?
You should be able to inspect the value returned, determine it's type, and handle it appropriately in your loop. |
Given that the value is an integer, 4.143E+4 and 41430 is the same value. If you need it as "41430" that's a string representation of an integer. At which point the comments made by my worthy associate kick into play. _________________ Honesty is the best policy.
Insanity is the best defence. |
|
Back to top |
|
 |
souciance |
Posted: Mon Oct 23, 2017 11:22 am Post subject: |
|
|
Disciple
Joined: 29 Jun 2010 Posts: 169
|
fjb_saper wrote: |
What do you mean, you cannot do any casting?...
Are you even using the IIB JSON parser?
You should be able to inspect the value returned, determine it's type, and handle it appropriately in your loop.
Have fun  |
Off course I am using the JSON parser.
I don't want to inspect the value and do any casting. I just want IIB to keep the value and format as is.
The sending system is sending the field with a value type of integer:
Code: |
{"DocumentId": 41420} |
I use
function and copy that value to another json field in my output root. The output message is another json message. I want the format and type to remain the same.
Somehow through this copy and using
the value is converted to scientific notation
Code: |
{"DocumentId":4.143E+4} |
.
Yes they mean the same but why is the format being changed? |
|
Back to top |
|
 |
souciance |
Posted: Mon Oct 23, 2017 11:23 am Post subject: |
|
|
Disciple
Joined: 29 Jun 2010 Posts: 169
|
Vitor wrote: |
fjb_saper wrote: |
What do you mean, you cannot do any casting?...
Are you even using the IIB JSON parser?
You should be able to inspect the value returned, determine it's type, and handle it appropriately in your loop. |
Given that the value is an integer, 4.143E+4 and 41430 is the same value. If you need it as "41430" that's a string representation of an integer. At which point the comments made by my worthy associate kick into play. |
Yes, the value is the same but the format is not. The input is as 41430 and not scientific format. Somehow IIB is formatting it to the scientific format.
I don't want to change the type to string. The type should remain as integer. It is the IIB parser which is changing the format either through copying the value or using FIELDVALUE. |
|
Back to top |
|
 |
Vitor |
Posted: Mon Oct 23, 2017 11:43 am Post subject: |
|
|
 Grand High Poobah
Joined: 11 Nov 2005 Posts: 26093 Location: Texas, USA
|
souciance wrote: |
Yes, the value is the same but the format is not. The input is as 41430 and not scientific format. Somehow IIB is formatting it to the scientific format. |
So (as I indicate above) what's the problem?
souciance wrote: |
I don't want to change the type to string. The type should remain as integer. |
Which it is. Anything downstream parsing the JSON will still get a value of 41430 even if it's expressed in scientific notation.
souciance wrote: |
It is the IIB parser which is changing the format either through copying the value or using FIELDVALUE. |
And you've not explained why that is a problem. _________________ Honesty is the best policy.
Insanity is the best defence. |
|
Back to top |
|
 |
souciance |
Posted: Mon Oct 23, 2017 12:02 pm Post subject: |
|
|
Disciple
Joined: 29 Jun 2010 Posts: 169
|
For the simple reason that I didn't ask IIB to change the format for me. Why is the IIB altering the input format? Semantially they mean the same, but syntactically they are different. That's the problem. It is also not stated anywhere in the documentation that this is the default behavior of the parser. |
|
Back to top |
|
 |
timber |
Posted: Mon Oct 23, 2017 11:21 pm Post subject: |
|
|
 Grand Master
Joined: 25 Aug 2015 Posts: 1292
|
|
Back to top |
|
 |
souciance |
Posted: Mon Oct 23, 2017 11:36 pm Post subject: |
|
|
Disciple
Joined: 29 Jun 2010 Posts: 169
|
Thanks timber,
I have to say I find it strange that the JSON parser is interpreting the value given as a float. Unless I explicitly cast it, I would expect the actual value to remain as is.
And from what I can tell from the thread you linked, there is actually nothing I can do in my case since I am actually receiving a plain integer. |
|
Back to top |
|
 |
timber |
Posted: Tue Oct 24, 2017 10:48 am Post subject: |
|
|
 Grand Master
Joined: 25 Aug 2015 Posts: 1292
|
Quote: |
I have to say I find it strange that the JSON parser is interpreting the value given as a float. |
It's not interpreting it as a float (unless you would like to supply the output from a Trace node to prove that it is).
It's probably interpreting it as an integer, but it is outputting the integer in scientific notation. As Vitor pointed out, the value has not been changed at all - just the representation of the value. |
|
Back to top |
|
 |
rekarm01 |
Posted: Thu Oct 26, 2017 4:52 pm Post subject: Re: JSON integer value converted to scientific notation |
|
|
Grand Master
Joined: 25 Jun 2008 Posts: 1415
|
souciance wrote: |
Just to clarify, we use
Code: |
FIELDVALUE(inputField) |
|
For JSON numbers, "inputField" works just as well all by itself. There is no need to use the FIELDVALUE() function here.
souciance wrote: |
It is the IIB parser which is changing the format either through copying the value or using FIELDVALUE. |
No, it isn't. The IIB uses a parser to copy data from an input bitstream to a message tree, or from a message tree to an output bitstream, but generally not from one message tree element to another.
And the JSON parser doesn't change the format of an INTEGER; it is a sequence of digits for both input and output bitstreams. The parser only uses E-notation to write FLOATs, and it only creates FLOATs when the input contains a decimal point or exponent.
Something in the message flow seems to be converting INTEGERs to FLOATs, when copying elements from one tree to another, but it is more likely in the user code, not the JSON parser.
Use Trace nodes to confirm where the data types of the relevant input and output fields might be changing from INTEGER to FLOAT, and work backwards from there, looking more closely at the code that creates the output fields (and associated usertrace). |
|
Back to top |
|
 |
fjb_saper |
Posted: Fri Oct 27, 2017 5:43 am Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
For the purpose of troubleshooting suggested by rekarm01, consider a decimal to be treated the same as a float  _________________ MQ & Broker admin |
|
Back to top |
|
 |
souciance |
Posted: Sun Oct 29, 2017 4:32 pm Post subject: Re: JSON integer value converted to scientific notation |
|
|
Disciple
Joined: 29 Jun 2010 Posts: 169
|
rekarm01 wrote: |
souciance wrote: |
Just to clarify, we use
Code: |
FIELDVALUE(inputField) |
|
For JSON numbers, "inputField" works just as well all by itself. There is no need to use the FIELDVALUE() function here.
souciance wrote: |
It is the IIB parser which is changing the format either through copying the value or using FIELDVALUE. |
No, it isn't. The IIB uses a parser to copy data from an input bitstream to a message tree, or from a message tree to an output bitstream, but generally not from one message tree element to another.
And the JSON parser doesn't change the format of an INTEGER; it is a sequence of digits for both input and output bitstreams. The parser only uses E-notation to write FLOATs, and it only creates FLOATs when the input contains a decimal point or exponent.
Something in the message flow seems to be converting INTEGERs to FLOATs, when copying elements from one tree to another, but it is more likely in the user code, not the JSON parser.
Use Trace nodes to confirm where the data types of the relevant input and output fields might be changing from INTEGER to FLOAT, and work backwards from there, looking more closely at the code that creates the output fields (and associated usertrace). |
Well if it isn't the JSON parser, then what could it be? The data itself is a simple 42341 or 56770. There are no decimal points or anything like that to even suggest it should be interpreted as a float or decimal, let alone change the format.
The user trace showed the data to be an integer when the JSON parser first receives the message. There should therefor be no need to change the format after a FIELDVALUE. |
|
Back to top |
|
 |
timber |
Posted: Mon Oct 30, 2017 1:07 am Post subject: |
|
|
 Grand Master
Joined: 25 Aug 2015 Posts: 1292
|
Quote: |
The user trace showed the data to be an integer |
Good. So JSON has not changed the number to another data type when parsing the input message.
It looks as if JSON is writing an INTEGER using the scientific notation when there is no need to do that. It's still an integer, though. An integer written down in a way that you did not expect. I agree that it's strange, but it's not a float. |
|
Back to top |
|
 |
|