ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportHow to load all data into the global cache at startup?

Post new topicReply to topic
How to load all data into the global cache at startup? View previous topic :: View next topic
Author Message
pmarie
PostPosted: Tue Jul 23, 2019 10:04 am Post subject: How to load all data into the global cache at startup? Reply with quote

Novice

Joined: 04 Apr 2002
Posts: 22

My customer wants to have all the data needed to process requests loaded (from a database) into the global cache as soon as the integration server is started, so that, when the first requests are received, they can be processed without delay, all the data being already in the cache.
This is an ACE on premise configuration.
Any idea about how to achieve this?
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Jul 23, 2019 10:15 am Post subject: Re: How to load all data into the global cache at startup? Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25841
Location: Texas, USA

pmarie wrote:
Any idea about how to achieve this?


- Give your customer a nice cool glass of water, sit them down and tell them that unless they have an unfeasibly large amount of data and/or a ludicrously short SLA, the extra complexity required to achieve this isn't worth the extra time it will take the first request to load the cache (this sounds a lot like a manager trying to sound clever, like trying to get a transaction down from 3 seconds to 2 seconds when the SLA is 5 seconds)

- Have a separate flow that populates the cache triggered at start up that in turn starts the request flow when the cache is loaded (because you don't want a request to start processing in parallel with the cache load).

- Push a dummy request through on start up to trigger the cache load.

I would seriously ask your customer exactly how much time they need to save on the first transaction. If I was looking for time savings in this situation, I'd optimize the SQL queries. I'd also ask the DBAs how much buffer they've got assigned, and how many queries actually go through to the spinning disc. It'll be a lot easier for you (and them) to have all this data cached on the DB side.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
timber
PostPosted: Tue Jul 23, 2019 12:58 pm Post subject: Reply with quote

Grand Master

Joined: 25 Aug 2015
Posts: 1039

Another option would be to do the loading of the cache in a JavaCompute node. I'm fairly sure that there is an option to perform once-only initialisation tasks in a JavaCompute node. If you want to do the actual work by calling out to some ESQL from the JavaCompute, that's probably do-able also. Not tried this - just throwing out ideas.
Back to top
View user's profile Send private message
pmarie
PostPosted: Wed Jul 31, 2019 8:32 am Post subject: Reply with quote

Novice

Joined: 04 Apr 2002
Posts: 22

Thanks for your answers.

I finally recommended the customer to do the following:
- Add a script in the directory "<work_path>/config/<broker_name>/<server_name>/profiles".
- In that script, put the following actions:
- Stop all the flows which process requests and need data in the global cache for that purpose.
- Start using CURL a specific flow (to be implemented) which will load all the data into the global cache. Wait for its response (meaning all the data are ready in the cache).
- Restart all the flows which process requests and need data in the global cache for that purpose.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed Jul 31, 2019 9:09 pm Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20119
Location: LI,NY

you also want to think about your global cache and it's lifetime cycle.
What if you never have to load the global cache ever again once it is initialized because it spans multiple integration servers?

Just because one server goes down does not mean the global cache is down...

_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexWebSphere Message Broker SupportHow to load all data into the global cache at startup?
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.