| Author |
Message
|
| madrox |
 Posted: Mon Jun 12, 2023 7:49 am Post subject: Shared Variable in Shared Library |
 |
|
Acolyte
Joined: 11 Mar 2015
Posts: 71
|
Hello All
Here is my scenario:
We are on ACE 12 and I have one REST service that i need invoke to get a API access token.
I have 3 services that require the same API access token from the first service which expires in 30 mins.
My initial thought was that i'd create a shared library and write the api token to a shared row and access it across the other 3 flow on the same EG. Which i realized that it does not work.
My second thought was that i'd create one flow with all the 3 services and the API access token call as well, but i was thinking that may be a better way
I'm here looking for suggestions on ways to implement this and i'm sure that this has already been done by someone here. |
|
| Back to top |
|
 |
| timber |
| Posted: Wed Jul 05, 2023 12:20 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1280
|
| If you need non-persistent storage for shared state, then I think the global cache is the way to go. |
|
| Back to top |
|
|
| mgk |
| Posted: Wed Jul 05, 2023 1:50 pm Post subject: |
|
|
Padawan
Joined: 31 Jul 2003
Posts: 1638
|
| Quote: |
| "My initial thought was that i'd create a shared library and write the api token to a shared row and access it across the other 3 flow on the same EG. Which i realized that it does not work." |
What did not work here? I would expect this to be possible given the flows are in the same EG?
_________________
MGK
The postings I make on this site are my own and don't necessarily represent IBM's positions, strategies or opinions. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Jul 06, 2023 6:07 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
| mgk wrote: |
| Quote: |
| "My initial thought was that i'd create a shared library and write the api token to a shared row and access it across the other 3 flow on the same EG. Which i realized that it does not work." |
What did not work here? I would expect this to be possible given the flows are in the same EG? |
I'd guess that different sessions could not share the same token... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| timber |
| Posted: Mon Jul 10, 2023 1:22 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1280
|
mgk said:
| Quote: |
| What did not work here? I would expect this to be possible given the flows are in the same EG? |
That's a fair point - I thought the requirement was for accessing across multiple EGs.
I seem to remember that it can help to define an ESQL function (in the same library where the SHARED variable is defined) that returns the value of the shared variable. For some reason it seems to work better than directly accessing the value of the shared variable from outside of the library where it is defined. |
|
| Back to top |
|
|
| mgk |
| Posted: Tue Jul 11, 2023 1:09 pm Post subject: |
|
|
Padawan
Joined: 31 Jul 2003
Posts: 1638
|
| Quote: |
| I would expect this to be possible given the flows are in the same EG? |
So I'm going to correct myself here  It's been a while but I've remembered that the scope of a SHARED variable is all threads running against a flow in the same EG but not across flows. However, there is a simple way to do this in ACE, which is to put the SHARED variable into a "callable flow" and access it from a "callable flow invoke" node. This way any number of flows can call the same flow that returns the value of the SHARED variable safely and quickly as it is simply accessing across threads in the same process. The flows would look something like this:
Flow1 ... FlowN:
| Code: |
| [Any Input Node] -> [more nodes to do work] -> ["Callable Flow Invoke" node] -> [more nodes for more work] -> [Any Transport Reply Node] |
Callable Flow:
| Code: |
| ["Callable Input" Node] -> [Compute Node to get/store/return the SHARED token variable] -> ["Callable Reply" Node] |
In this example, as long as the Callable Flow has no additional instances, only 1 thread will access the SHARED token and return it safely.
I hope this helps and makes sense...
Kinds regards,
_________________
MGK
The postings I make on this site are my own and don't necessarily represent IBM's positions, strategies or opinions. |
|
| Back to top |
|
|
| petervh1 |
| Posted: Thu Jul 20, 2023 6:10 am Post subject: |
|
|
Centurion
Joined: 19 Apr 2010
Posts: 123
|
Hi MGK
I'm trying to implement your suggestion to use a Callable Flow, but I am unable to get it to pass the value back to the calling flow.
I tried the following:
| Quote: |
Define the same SHARED ROW variable in both the calling andf callable flows.
Store the SHARED ROW value in the Callable Flow in an Environment variable. The calling flow does not see any Environment variable when it receives control back from the callable flow.
|
If I try the following, I can get the value returned to the calling flow:
| Quote: |
Store the SHARED ROW value in the Callable Flow in a structure like OutputRoot.XMLNSC.root.xyz
The calling flow now sees the value in InputRoot.XMLNSC.root.xyz.
|
This second scenario is obviously not ideal, as I want to pass an array of cached values back to the calling program.
Can you tell me what is missing from my flow please?
TIA |
|
| Back to top |
|
|
| mgk |
| Posted: Sat Jul 22, 2023 3:39 am Post subject: |
|
|
Padawan
Joined: 31 Jul 2003
Posts: 1638
|
| Quote: |
Store the SHARED ROW value in the Callable Flow in a structure like OutputRoot.XMLNSC.root.xyz
The calling flow now sees the value in InputRoot.XMLNSC.root.xyz. |
This is the correct way to do this. However, it does not have to be a single value returned - you can return an array or an entire tree back to the calling flow - as much data as you need. Just create the output message you want to send back that has the "shape" you need as you would do for any other response message. You can use XML, JSON or any of the other parsers etc...
I hope this helps...
_________________
MGK
The postings I make on this site are my own and don't necessarily represent IBM's positions, strategies or opinions. |
|
| Back to top |
|
|
| petervh1 |
| Posted: Tue Jul 25, 2023 12:58 am Post subject: |
|
|
Centurion
Joined: 19 Apr 2010
Posts: 123
|
Thanks. I can now return the values in an XML structure.
I want to be able to create a cache that can be shared by all flows in an EG. My aim is to load the cache on a once-off basis using a small flow, then have any other flow running in the same EG access that pre-loaded cache. Is this possible, based on what you said?
| Quote: |
This way any number of flows can call the same flow that returns the value of the SHARED variable safely and quickly as it is simply accessing across threads in the same process. |
I tried the following:
1. Run a small flow that populates a cache as follows:
| Code: |
DECLARE smallCache SHARED ROW;
CREATE COMPUTE MODULE PopulateCache_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
SET smallCache.[1] = 'ABC';
SET smallCache.[2] = 'DEF';
SET smallCache.[3] = 'GHI';
RETURN TRUE;
END;
END MODULE;
|
2. Run another flow in the same EG with a Compute node that is coded as follows:
| Quote: |
DECLARE smallCache SHARED ROW;
CREATE COMPUTE MODULE ReadTry_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
SET A = smallCache .[1];
END;
END MODULE;
|
When I debug the second flow, I see that the value of A is NULL after the SET statement.
Do I need to write the values from smallCache to an output structure first? But that would defeat the object of having a cache, surely? |
|
| Back to top |
|
|
| mgk |
| Posted: Tue Jul 25, 2023 12:56 pm Post subject: |
|
|
Padawan
Joined: 31 Jul 2003
Posts: 1638
|
| Quote: |
| I want to be able to create a cache that can be shared by all flows in an EG |
To do this the cache must exist in one flow and one flow only and this must be the "Callable Flow" I mentioned previously.
| Quote: |
| My aim is to load the cache on a once-off basis using a small flow |
Right, so the code to load the cache must also exist in the same "Callable Flow" as the cache itself. There are different ways you can initialise the cache depending on where you need to get the data from. For example the "Callable Flow" could initialise the cache itself by calling a DB or a REST API if it is empty when accessed. Alternatively you could have a second Input node in the flow to allow an external caller to initialise the cache.
| Quote: |
| Is this possible, based on what you said |
Yes, as long as all the other flows that want to access the cache in the "Callable Flow" do so by invoking the "Callable Flow" with a "Callable Flow Invoke" node.
Does that make help?
_________________
MGK
The postings I make on this site are my own and don't necessarily represent IBM's positions, strategies or opinions. |
|
| Back to top |
|
|
| petervh1 |
| Posted: Sun Jul 30, 2023 10:54 pm Post subject: |
|
|
Centurion
Joined: 19 Apr 2010
Posts: 123
|
I got it working.
Thanks for your help. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Jul 31, 2023 12:19 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9399
Location: US: west coast, almost. Otherwise, enroute.
|
| petervh1 wrote: |
I got it working.
Thanks for your help. |
For the benefit of others, please tell us how you got it working.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| petervh1 |
| Posted: Mon Jul 31, 2023 9:37 pm Post subject: |
|
|
Centurion
Joined: 19 Apr 2010
Posts: 123
|
I set up 2 flows:
Flow 1 - Callable flow
Callable Input -> Compute -> Callable Output
Compute Niode:
| Quote: |
DECLARE confTable SHARED ROW;
CREATE COMPUTE MODULE RetrieveCache_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
(populate cache)
(create output structure e.g. OutputRoot.JSON.Data.xyz that
contains cache values to be returned to the calling flow)
|
Flow 2 - Calling flow
Compute1 -> Callable Flow Invoke -> Compute2
Compute Node:
| Quote: |
CREATE COMPUTE MODULE CallingFlow_Compute1
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
SET OutputRoot.JSON.Data.paramName[1] = xyz;
(this is the name of the cache value you want to retrieve in the callable flow)
|
|
|
| Back to top |
|
|
| petervh1 |
| Posted: Tue Aug 15, 2023 2:37 am Post subject: |
|
|
Centurion
Joined: 19 Apr 2010
Posts: 123
|
I've come across a problem with the solution that I posted on Aug 1.
Flow 1 (Callable flow) has a check to see if the cache exists. If not, it populates the cache by reading the contents of a DB table into the SHARED ROW before returning any values to Flow 2 (Calling flow). All fine and good so far.
Here's the problem: I then add another row to the DB table mentioned above and then run a separate flow whose sole job is to populate the same confTable cache (SHARED ROW) that is normally populated by Flow 1. If I run Flow 2 again (after redeploying it) to call the Callable Flow and request the value of the new entry that I have just added, it doesn't find this row in the cache. I can see that the Reload Cache flow has run, and the user trace shows me that it has added the row to the cache.
All flows run in the same EG.
What am I doing wrong? |
|
| Back to top |
|
|
| petervh1 |
| Posted: Wed Aug 16, 2023 3:28 am Post subject: |
|
|
Centurion
Joined: 19 Apr 2010
Posts: 123
|
I solved the issue.
The flow that you use to repopulate the cache after adding a new row to the DB table to be cached must literally be the same flow that is used for the initial populating of the cache - not even another flow in the same EG.
I clearly didn't expect this, although that is what mgk said in his post:
| Quote: |
Alternatively you could have a second Input node in the flow to allow an external caller to initialise the cache.
|
|
|
| Back to top |
|
|
|
|
