| Author |
Message
|
| chris boehnke |
 Posted: Wed Jul 08, 2015 7:41 am Post subject: Global cache and design question |
 |
|
Partisan
Joined: 25 Jul 2006
Posts: 369
|
Hi All,
I am planning to use global cache. The requirement is as below.
1. Client sends the request to WMB.
2. Based on router xml , wmb call two different backend services.
3. WMB transforms the response from backend services and properties file to client response.
4. Responds to the client.
Here, i am planning to use global cache for routeing xml and properties file.
I have created a seperate flow with fileinput node for loading the properties and routing file.Deployed this flow to seperate EG(Catalog server)
Whenever there is a change in the files, the files will be placed in the directory , so that they are loaded into cache.
I am wondering if this is good design?Pls suggest |
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Wed Jul 08, 2015 7:53 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
if there is a cache miss how the cache will get updated?
howz the backend services, how about thinking about loader scripts to load the Cache?
are you planning to use Inline Cache or Side line cache?
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| chris boehnke |
| Posted: Wed Jul 08, 2015 7:59 am Post subject: |
|
|
Partisan
Joined: 25 Jul 2006
Posts: 369
|
| Gaya3 wrote: |
if there is a cache miss how the cache will get updated?
howz the backend services, how about thinking about loader scripts to load the Cache?
are you planning to use Inline Cache or Side line cache? |
I am not sure what inline cache and side line cahe is..
I was initially thing of using shared variables. The size of the props file is 1MB in size. I was suggested to use global cache. |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Wed Jul 08, 2015 8:24 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
Global cache is very different from shared variable.. 
Inline Cache where your application goes thru the cache and hit the backend ( the cache will get updated by your app while writing or getting)
Side line cache your application connects to cache and backend (if the data is available in cache then all the requests are satisfied from the cache at memory speed)
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jul 08, 2015 8:25 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
the design as lain out is not bad off the cuff.
It's probably more of a sideline cache, in that it's providing information to augment the message, rather than being a destination for the message.
I personally still think it would be better to separate the UI from the flow. |
|
| Back to top |
|
|
| chris boehnke |
| Posted: Wed Jul 08, 2015 8:40 am Post subject: |
|
|
Partisan
Joined: 25 Jul 2006
Posts: 369
|
| mqjeff wrote: |
the design as lain out is not bad off the cuff.
It's probably more of a sideline cache, in that it's providing information to augment the message, rather than being a destination for the message.
I personally still think it would be better to separate the UI from the flow. |
mqjeff,
We are planning to seperate the UI from the flow, but the timelines for the release are not inline.
Cant I use shared/esql variables for this scenario rather than global cache? |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 08, 2015 8:43 am Post subject: Re: Global cache and design question |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| chris boehnke wrote: |
| I am wondering if this is good design?Pls suggest |
When you do a restart, someone will need to remember to put a file in the directory to reload the cache.
Better to incorporate logic in the cache reading function (as a subflow or similar) that checks for the existence of cached details and uses a FileRead node to load the cache if they're missing. When the files legitimately change, part of the change process to drop the new files includes a step to flush the cache with an administrative command.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jul 08, 2015 8:43 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| You can use shared variables. But it's going to be harder to load.... |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 08, 2015 8:44 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| chris boehnke wrote: |
| Cant I use shared/esql variables for this scenario rather than global cache? |
Probably, but the global cache is easier to use and administer than shared variables. It's a better fit for your use case.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| chris boehnke |
| Posted: Wed Jul 08, 2015 8:47 am Post subject: |
|
|
Partisan
Joined: 25 Jul 2006
Posts: 369
|
| Vitor wrote: |
| chris boehnke wrote: |
| Cant I use shared/esql variables for this scenario rather than global cache? |
Probably, but the global cache is easier to use and administer than shared variables. It's a better fit for your use case. |
In case of global cache, if the broker is bounced, we should move the files to fileinputnode directory to load files.
In case if we forget to do that we get errors.
Is there any other way , I could load the files to global cache? |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jul 08, 2015 8:49 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| chris boehnke wrote: |
| Is there any other way , I could load the files to global cache? |
| Vitor wrote: |
| Better to incorporate logic in the cache reading function (as a subflow or similar) that checks for the existence of cached details and uses a FileRead node to load the cache if they're missing. When the files legitimately change, part of the change process to drop the new files includes a step to flush the cache with an administrative command. |
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| nelson |
| Posted: Wed Jul 08, 2015 11:13 am Post subject: Re: Global cache and design question |
|
|
Partisan
Joined: 02 Oct 2012
Posts: 313
|
| Vitor wrote: |
| chris boehnke wrote: |
| I am wondering if this is good design?Pls suggest |
When you do a restart, someone will need to remember to put a file in the directory to reload the cache.
Better to incorporate logic in the cache reading function (as a subflow or similar) that checks for the existence of cached details and uses a FileRead node to load the cache if they're missing. When the files legitimately change, part of the change process to drop the new files includes a step to flush the cache with an administrative command. |
We have a Timeout Notification node in automatic operation mode. So when the flow starts a file is read and the data is cached. If we need an update, we just change the file and restart the flow. The timeout interval also could work as a refresh interval. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Jul 08, 2015 12:22 pm Post subject: Re: Global cache and design question |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| nelson wrote: |
We have a Timeout Notification node in automatic operation mode. So when the flow starts a file is read and the data is cached. If we need an update, we just change the file and restart the flow. The timeout interval also could work as a refresh interval. |
We do the same but reading a DB Table. The Key is the name of the Flow.
It works very well.
If you are using Shared Variable then to need to handle the locking with an ATOMIC section.
The Global Cache will take time to load up initially. If you have more than one broker the data needs to be replicated to every broker.
We looked at moving to Global Cache but it was more trouble than it was worth in our setup. It is worth some experimentation to determine which solution is best for your site.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Wed Jul 08, 2015 12:57 pm Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
another important fact is - the Global Cache in MB is V8.5 (Websphere Extreme Scale)
Websphere Extreme Scale on WAS ND is having the most updated version - V8.6
Its good to go with V8.6.
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| chris boehnke |
| Posted: Thu Jul 09, 2015 6:20 am Post subject: |
|
|
Partisan
Joined: 25 Jul 2006
Posts: 369
|
We are using WMB V8.0.0.2.
Looks like global cache is available from V8. I will do a POC with single broker first.
I was thinking of ESQL cache also.
We will first read the file and cache the data to shared variables. Then move the data to ESQL variable and run a select query against this variable. This way the performance might not get impacted much.
Suggestions pls.
Also I want to know if caching 1MB impacts the broker performance ? |
|
| Back to top |
|
|
|
|
