ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Global Cache Issue

Post new topic  Reply to topic
 Global Cache Issue « View previous topic :: View next topic » 
Author Message
DP1234
PostPosted: Wed Aug 13, 2014 1:29 am    Post subject: Global Cache Issue Reply with quote

Apprentice

Joined: 25 May 2014
Posts: 40

Am facing some serious issue related to Global Cache in multi thread scenario..i have been maintaning a Global cache which holds the count of number of input messages/events occur in the input. For every incoming message/event the count is incremented and the same is Updated in global cache as well..in an load test of 200k messages i found that count is not updated in cache properly..say for 200k messages the count would be some 30000...The instance to my flow be 1 or more than one the count is poorly reflecting in cache.

My analysis on the issue was when multi threads are approaching to update the cache in Java block the current thread which is in process is locking the procedure doesnt letting other, resulting false value of count reflecting in Cache...

Making Update Cache Java Procedure Synchronised didn't help

Am using IIB.

Badly need ur help on this issue

Thanks
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed Aug 13, 2014 4:55 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Seems to me you need to create a singleton to hold the count information. In turn the singleton would update the cache.
If the value gets changed from multiple execution groups, your singleton needs to be a rmi accessible object.

It might be judicious to have the location of the singleton in the cache...

As this gets quite complicated, I'd suggest you subscribe to the flow statistics and get the number of invocations from there...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
mqjeff
PostPosted: Wed Aug 13, 2014 5:33 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

fjb_saper wrote:
Seems to me you need to create a singleton to hold the count information. In turn the singleton would update the cache.
If the value gets changed from multiple execution groups, your singleton needs to be a rmi accessible object.

It might be judicious to have the location of the singleton in the cache...

As this gets quite complicated, I'd suggest you subscribe to the flow statistics and get the number of invocations from there...


A singleton doesn't help across multiple EGs.

But clearly the cache needs to be updated within an atomic block.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed Aug 13, 2014 5:50 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

mqjeff wrote:
fjb_saper wrote:

If the value gets changed from multiple execution groups, your singleton needs to be a rmi accessible object.

It might be judicious to have the location of the singleton in the cache...

As this gets quite complicated, I'd suggest you subscribe to the flow statistics and get the number of invocations from there...


A singleton doesn't help across multiple EGs.

But clearly the cache needs to be updated within an atomic block.


Clearly if the singleton is accessed via RMI from other eg's it would still have the "single point of update"...
And even within an atomic block (java sychronized) it won't work across multiple egs as each JVM only synchronizes it's set and not the cache...
My understanding is that retrieving the cache object you actually retrieve a copy / clone. This means that a synchronize (cache object) would not do the trick. A synchronized method won't do it either as the different egs will just overwrite the last known value.

Using a remote object singleton as cache and having the location of it in the global cache effectively synchronizes across egs but also creates a bottleneck... and would need to synchronize with the actual global cache at intervals... The flow would need to access the singleton via RMI to get and update the value. The singleton will then need to synchronize internally to keep the count thread safe...

Like I said, quite complicated and I don't see the advantage of it over a count after the fact from the flow statistics...


_________________
MQ & Broker admin


Last edited by fjb_saper on Wed Aug 13, 2014 6:00 am; edited 1 time in total
Back to top
View user's profile Send private message Send e-mail
mqjeff
PostPosted: Wed Aug 13, 2014 5:58 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Yes, if you go to all the trouble you mention to implement it in a straight Java means, then it gets quite complicated.

IF, instead, you think of "atomic block" as a generic notion, rather than a java implementation of a generic notion, that might lead one to either looking at available functions with GlobalCache (or at least WS XtremeScale) or otherwise for a solution within the larger Broker product.

And, yes, I agree that if this count is needed for any kind of *monitoring* purpose, then coding it at all rather than using stats is entirely the wrong solution.

But *surely* DP1234 wouldn't be looking to implement this for *monitoring* purposes. Surely there's a business requirement to include this number in the message data.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed Aug 13, 2014 6:08 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Correct me if I'm wrong, but the update to the global cache is always done in an "atomic" way. However the global cache only allows you to get the object and not to implement a (thread safe) method on the cached object.
Thus each e.g. will overwrite the last value with it's own, even if you manage to make the update in thread safe manner within the e.g. ...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
DP1234
PostPosted: Mon Aug 18, 2014 2:09 am    Post subject: Reply with quote

Apprentice

Joined: 25 May 2014
Posts: 40

fjb_saper and mqjeff Thanks for ur replies

I tried making java update/retrive block synchronised, implemented singleton and also tried using reetrant locks it didn't help.

Making ESQL block atomic didn't help.

Count value is same or getting still lowered

One thing i wanna know is this multi thread cache update issue is limitation of Global Cache by any way..?

wat could be done to overcome any alternate...least possibilities
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Aug 18, 2014 4:27 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

In order for it to work like you intend to, you would need an add function implemented in the global cache object, accessible through the global cache and residing in the global cache. This function would of course need to be synchronized in some way, either internally or through global cache access protocol.

What you are doing is a replace of the global cache with the new value, when in fact your intention was to do an add.
So if thread one sets the value to 5 and thread 2 sets the value to 3, which will be the value in the global cache?

Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
DP1234
PostPosted: Mon Aug 18, 2014 7:28 am    Post subject: Reply with quote

Apprentice

Joined: 25 May 2014
Posts: 40

fjb_saper wrote:

So if thread one sets the value to 5 and thread 2 sets the value to 3, which will be the value in the global cache?



its 3 literally
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Global Cache Issue
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.