| Author |
Message
|
| matuwe |
 Posted: Tue Oct 10, 2017 11:16 am Post subject: Combine two arrays into one |
 |
|
Master
Joined: 05 Dec 2007
Posts: 296
|
Hi
I am using IIB 10, I am receiving data as Jason.Data.embeded[].items[]
Th input message sample looks like this
InputRoot.Jason.Data.embeded[1].items[1].value ='hello1';
InputRoot.Jason.Data.embeded[1].items[2].value ='hello2'
InputRoot.Jason.Data.embeded[1].items[3].value ='hello3'
InputRoot.Jason.Data.embeded[1].items[4].value ='hello4'
InputRoot.Jason.Data.embeded[2].items[1].value ='hello5'
InputRoot.Jason.Data.embeded[2].items[2].value ='hello6'
InputRoot.Jason.Data.embeded[2].items[3].value ='hello100'
My Output message needs to combine all these values into one array
OutputRoot.Jason.Data.items[1].value ='hello1'
OutputRoot.Jason.Data.items[2].value ='hello2'
OutputRoot.Jason.Data.items[3].value ='hello3'
OutputRoot.Jason.Data.items[4].value ='hello4'
OutputRoot.Jason.Data.items[5].value ='hello5'
OutputRoot.Jason.Data.items[6].value ='hello6'
OutputRoot.Jason.Data.items[7].value ='hello100'
Is there anyway that I can do this without using two WHILE LOOPS?
I do not how many items I am expecting, so far I my results ends up with almost 5000 items and while loop affects performance.
Please help suggestions
   |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Oct 10, 2017 11:21 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
ESQL does not have arrays.
Never use [x] to access elements.
You need to use a loop, maybe two.
You should use references, or a foreach loop (maybe two).
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Oct 10, 2017 12:01 pm Post subject: Re: Combine two arrays into one |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| matuwe wrote: |
| I do not how many items I am expecting, so far I my results ends up with almost 5000 items and while loop affects performance. |
It's not the while loop that's affecting performance, it's the use of indexes ([1],[2],etc.). If you ask ESQL for the [nth] element, it finds it by starting at 1 and counting through the elements until it reaches n. Thus the performance is inversely proportional to the size of the index, and by 5000 elements I'm not surprised you notice it.
Use reference variables testing with WHILE...LASTMOVE / FOR...EACH loops instead; these are also much simpler ways to handle an unknown number of elements. Use CREATE...LASTCHILD to build the output tree; the same index problem affects output as well as input (ask ESQL to create element 5000 and it will count from 1 to 4999 to find where that is).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| timber |
| Posted: Tue Oct 10, 2017 3:07 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| I'll put that even more strongly. For a large majority of cases, a FOR loop is the tidiest option. And I agree with Vitor re: the use of CREATE LASTCHILD OF |
|
| Back to top |
|
|
| matuwe |
| Posted: Wed Oct 11, 2017 11:06 am Post subject: |
|
|
Master
Joined: 05 Dec 2007
Posts: 296
|
THanks Guys, the code performance has really improved.
But I need more help..
My Flow is
http Input >> Compute (loop 5) >> Retrieve Data From DB >> Compute Node (StoreData) >> http reply
My constraint is the retrieve from MongoDB via restHeart is maximum 1000 records. I number of records on DB is 5000. So I have to loop 5 times to get all the data, 5 pages. My doubt is the code snippets attached bellow, I store all the datarecords on Envionment.Data till all data is retrieved, and then try to build all the results into one Json array to send out. This is taking way to long.
Is there anyway I can avoiding using the Environment.Variables.result
| Code: |
IF ((pageCount > 1) AND (5 >= loopCount)) THEN
-- Save Payload
SET Environment.Variables.result[loopCount] = InputRoot.JSON.Data._embedded[1];
DECLARE ref REFERENCE TO Environment.Variables;
SET Environment.Variables.loopCount = envLoopCount;
--IF loopCount = InputRoot.JSON.Data._total_pages THEN
IF loopCount = 5 THEN
CREATE FIELD OutputRoot.JSON.Data IDENTITY (JSON.Array)Data;
DECLARE outputRef REFERENCE TO OutputRoot.JSON.Data;
DECLARE rResource REFERENCE TO Environment.Variables.result[1];
DECLARE LineCount INTEGER 1;
WHILE LASTMOVE(rResource) = TRUE DO
DECLARE rResourceItem REFERENCE TO rResource.Item[1];
WHILE LASTMOVE(rResourceItem) = TRUE DO
SET outputRef.Item[LineCount] = rResourceItem;
SET LineCount = LineCount + 1;
MOVE rResourceItem NEXTSIBLING REPEAT TYPE NAME;
END WHILE;
MOVE rResource NEXTSIBLING REPEAT TYPE NAME;
END WHILE;
PROPAGATE TO TERMINAL 'out1' DELETE NONE;
RETURN FALSE;
ELSEIF loopCount < InputRoot.JSON.Data._total_pages THEN
--PROPAGATE TO TERMINAL 'out2' DELETE NONE;
END IF; |
|
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Oct 11, 2017 11:11 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| matuwe wrote: |
| This is taking way to long. |
Both @timber and I told you not to do this:
| Code: |
SET outputRef.Item[LineCount] = rResourceItem;
SET LineCount = LineCount + 1; |
Fix that, then worry about using the Environment if performance is still a problem.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| timber |
| Posted: Wed Oct 11, 2017 12:02 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
hi matuwe,
Let me suggest simple rule:
Never use an indexed array in your ESQL.
Yes, I really did say that. Before some smart individual pops up and points it out, I know that IBM's own boilerplate code for every ESQL module uses *gasp* indexes. But it really should not - and anyway that code was written long before ESQL had REFERENCE variables. It is harmless because the number of headers on any particular message will usually be quite a lot less than 5000  |
|
| Back to top |
|
|
|
|
