ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportReasons to copy shared-row cache into environment?

Post new topicReply to topic
Reasons to copy shared-row cache into environment? View previous topic :: View next topic
Author Message
simonalexander2005
PostPosted: Wed Apr 19, 2017 4:45 am Post subject: Reasons to copy shared-row cache into environment? Reply with quote

Novice

Joined: 13 Jun 2016
Posts: 12

Hi,

I have inherited some ESQL code that follows the "ESQL Cache" pattern to store data from a database:

Code:
DECLARE CACHE SHARED ROW;
CREATE PROCEDURE populate_CACHE(INOUT envRef REFERENCE)
BEGIN
    BEGIN ATOMIC
        IF CACHE.Populated IS NULL THEN
            -- populate CACHE from database, using style similar to:
            SET CACHE.{keyFromDatabase} = valueFromDatabase;
            -- .....
            SET CACHE.Populated = TRUE
        END IF
       -- This is the weird bit
        DECLARE db REFERENCE TO CACHE;
        CREATE LASTCHILD OF envRef.copiedCACHE FROM db;
END;


This code is called for every message that enters the flow. So, the CACHE is populated from the database the first time; then every time a message is read the contents of CACHE are copied into the environment tree.

The data is then read from the cache in the main Compute node as required, using code similar to:

Code:
DECLARE startDate TIMESTAMP;      
     SET startDate = envRef.copiedCACHE.Root.{ID};


If the cache is ever updated in the Compute node (due to the contents of the message being such that it updates the cache), then the database, the SHARED ROW and the envRef.copiedCACHE are all updated at that point.

My question therefore is:

- Why copy the SHARED ROW into the envRef.copiedCACHE at all? Is there some performance benefit to doing so? As far as I can see, all it does it remove the benefit of using SHARED ROW in that if it's updated in another instance (thread) it won't be seen. Is there any other benefit to this?

Our CACHE contains tens of thousands of entries, so if removing this copy and accessing the SHARED ROW cache directly is likely to improve performance, I would like to do so - but I was wondering if I was missing anything first.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Wed Apr 19, 2017 5:32 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17094

I'm guessing the point is to ensure that the rest of the flow is always working with the same set of values - not having one set of values in one node, and then having things changed before you need it again...

You could probably limit what you copy to what your flow needs, rather than the whole shebang. That would help performance, assuming it's easy to grab the pieces you need - without doing a huge select/sort/merge/etc.
_________________
Read, Think, Try, Repeat
Back to top
View user's profile Send private message
adubya
PostPosted: Thu Apr 20, 2017 11:08 am Post subject: Reply with quote

Partisan

Joined: 25 Aug 2011
Posts: 365
Location: GU12, UK

As Jeff said, it's probably to protect against cache changes made by other flow instances and to stop having to litter the code with ATOMIC blocks whenever the cache is accessed.
_________________
Independent Middleware Consultant
andy@knownentity.com
Back to top
View user's profile Send private message Send e-mail
mqjeff
PostPosted: Thu Apr 20, 2017 11:25 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17094

adubya wrote:
litter the code with ATOMIC blocks whenever the cache is accessed.

Even a bit of nuclear waste wouldn't stop the cache from changing behind the scenes, unless some kind of copy to in-flow storage happened.
_________________
Read, Think, Try, Repeat
Back to top
View user's profile Send private message
adubya
PostPosted: Thu Apr 20, 2017 11:28 am Post subject: Reply with quote

Partisan

Joined: 25 Aug 2011
Posts: 365
Location: GU12, UK

mqjeff wrote:
adubya wrote:
litter the code with ATOMIC blocks whenever the cache is accessed.

Even a bit of nuclear waste wouldn't stop the cache from changing behind the scenes, unless some kind of copy to in-flow storage happened.


Yes, I meant to protect against a potential cache reload taking place at the same time as another flow instance was accessing it.
_________________
Independent Middleware Consultant
andy@knownentity.com
Back to top
View user's profile Send private message Send e-mail
mqjeff
PostPosted: Thu Apr 20, 2017 11:33 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17094

adubya wrote:
Yes, I meant to protect against a potential cache reload taking place at the same time as another flow instance was accessing it.


I know. I wanted to make a joke about ATOMIC blocks and nuclear waste... and make sure that future readers weren't supported in bad understandings.
_________________
Read, Think, Try, Repeat
Back to top
View user's profile Send private message
simonalexander2005
PostPosted: Tue Apr 25, 2017 2:41 am Post subject: Reply with quote

Novice

Joined: 13 Jun 2016
Posts: 12

So, is the access to a SHARED ROW variable not threadsafe then? That would explain it, as really we'd want updates to the cache to be propagated immediately to other threads - i.e. we'd want the cache updating to affect another instance (although clearing and reloading it wouldn't be so good... hmm... - but if the clear happens in an ATOMIC block, the other threads would have to wait anyway, right?).

If I were to remove the "copy to environment" and access the shared row directly each time, what should I be aware of? I can't see anything on thread safety of shared row variables in the documentation anywhere

--- edit ---

found it! - "It is ... unnecessary to use the BEGIN ATOMIC construct on reads and writes to shared variables. The broker always safely writes a new value to, and safely reads the latest value from, a shared variable. ATOMIC is only required when the application is sensitive to seeing intermediate results." - for anyone else looking in the future: https://www.ibm.com/support/knowledgecenter/en/SSMKHH_9.0.0/com.ibm.etools.mft.doc/ak04940_.htm
Back to top
View user's profile Send private message
adubya
PostPosted: Tue Apr 25, 2017 2:53 am Post subject: Reply with quote

Partisan

Joined: 25 Aug 2011
Posts: 365
Location: GU12, UK

A SHARED variable is available to all instances of the same message flow running in an execution group. As such, you do need to be careful with how you're using SHARED variables and if need be use concepts such as the ATOMIC block to protect access to the variable.
_________________
Independent Middleware Consultant
andy@knownentity.com
Back to top
View user's profile Send private message Send e-mail
simonalexander2005
PostPosted: Tue Apr 25, 2017 3:31 am Post subject: Reply with quote

Novice

Joined: 13 Jun 2016
Posts: 12

adubya wrote:
A SHARED variable is available to all instances of the same message flow running in an execution group. As such, you do need to be careful with how you're using SHARED variables and if need be use concepts such as the ATOMIC block to protect access to the variable.


It's the "if need be" situations that I'm trying to figure out. It seems from the docs that read/write are threadsafe
Back to top
View user's profile Send private message
adubya
PostPosted: Tue Apr 25, 2017 3:48 am Post subject: Reply with quote

Partisan

Joined: 25 Aug 2011
Posts: 365
Location: GU12, UK

I guess each individual SHARED variable access statement is guaranteed to be atomic then. You do need to be careful of cases where you have multiple SHARED variable accesses in one piece of code.

i.e. a cache reload

DELETE cacheRef.cacheEntries;
SET cacheRef.cacheEntries.entry[] = SELECT <blah> FROM <blah>;

each statement would be threadsafe it its own right but the code block as a whole isn't. The thread could be suspended after the DELETE statement and another instance swapped in which then finds an empty cache.
_________________
Independent Middleware Consultant
andy@knownentity.com
Back to top
View user's profile Send private message Send e-mail
Vitor
PostPosted: Tue Apr 25, 2017 4:46 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 24267
Location: Ohio, USA

simonalexander2005 wrote:
It seems from the docs that read/write are threadsafe


Post the link that says / implies that.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
adubya
PostPosted: Tue Apr 25, 2017 4:50 am Post subject: Reply with quote

Partisan

Joined: 25 Aug 2011
Posts: 365
Location: GU12, UK

He has done, up there ^^^^^
_________________
Independent Middleware Consultant
andy@knownentity.com
Back to top
View user's profile Send private message Send e-mail
Vitor
PostPosted: Tue Apr 25, 2017 5:12 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 24267
Location: Ohio, USA

adubya wrote:
He has done, up there ^^^^^


"ATOMIC is only required when the application is sensitive to seeing intermediate results" doesn't sound threadsafe to me. Perhaps this is some new usage of the word "safe" that I was previously unaware of.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
adubya
PostPosted: Tue Apr 25, 2017 5:39 am Post subject: Reply with quote

Partisan

Joined: 25 Aug 2011
Posts: 365
Location: GU12, UK

Now I've actually read the full contents of the linked infocenter piece it's clear that shared variable access isn't threadsafe at the statement level. It's guaranteed that garbage won't be returned but that's it. The "SET count = count + 1" example illustrates what could go wrong without proper protection of the shared resource.
_________________
Independent Middleware Consultant
andy@knownentity.com
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker SupportReasons to copy shared-row cache into environment?
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.