| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Caching in WMB - Choices & Performance considerations. |
« View previous topic :: View next topic » |
| Author |
Message
|
| brokerguy |
 Posted: Fri Feb 07, 2014 11:04 pm Post subject: Caching in WMB - Choices & Performance considerations. |
 |
|
Apprentice
Joined: 18 Mar 2013
Posts: 26
Location: Cosmos
|
Hi All,
I am at WMB v8003 level and would like to know your thoughts on caching within WMB and its impact to the system performance. I would be highly obliged if you can read through the below text and put in your valued thoughts.
As we know we can perform caching of data using the following ways:
1. Shared row variables - cache scope is at flow level only.
2. Use of CacheNode support pac - scope at EG level, can be extended to broker level.
3. Global cache - scope at multi EG + multi broker level.
My requirement is to perform caching for the following scenarios:
A) Cache static data usually part of properties file as key value pairs, which is never going to change
B) Cache correlation id and contextual information about an asynchronous (coordinated request/reply) MQ flow, thus allowing the reply flow to leverage the cache to get the necessary information without resorting to MQ or Database lookups.
C) Cache crucial data that are usually unchanged for a day or two. These data are obtained by making backend invocations such as CICS transactions. This scenario may require get from two maps, about 10-20 times per service invocations.
In all the above scenarios, I have been using Global caching (default policy) with the intent of moving to the configuration where in dedicated EGs will hold the catalog/container servers and the EGs hosting the services would act as clients.
Now, with the above background, I have the following questions:
1. Did someone had hands-on experience in performance measurement vis-a-vis application of Global cache, particularly around CPU utilization. What is your view point when using Global cache vs. falling back to Shared variables (although EG wide caching cannot be achieved) ?
2. In case of global cache, when the EGs communicate with the
container servers, how does the internal communication happen ? An inter process communication is happening for sure, but how does that happen ? For each calls from esql to java a JNI call is made ?
3. Would you recommend using the cacheNode support pac vs. the Global cache, given the fact that we would supposedly negate any inter-process communication that takes place for Global Cache ?
4. Why would one use Global cache in scenarios (A) and (C) cited above, if the following slight trade-off is made:
- load data to cache (via shared variables) and OK to loose data on EG
reloads/msg flow stop-start. The re-load to cache would takes place
again at a per flow level during the first few sets of transactions
anyway.
- OK to have multiple msg flows stand up its own cache (via shared
variables).
Thanks in advance!
_________________
Learning to unlearn, re-learn |
|
| Back to top |
|
 |
| brokerguy |
| Posted: Tue Feb 11, 2014 11:26 am Post subject: |
|
|
Apprentice
Joined: 18 Mar 2013
Posts: 26
Location: Cosmos
|
Any thoughts or inputs to the above scenario/questions.
Please let me know if some portion of the question requires clarification.
_________________
Learning to unlearn, re-learn |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Feb 11, 2014 11:29 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| brokerguy wrote: |
| Please let me know if some portion of the question requires clarification. |
Question's fine. Just not used caching that much to usefully comment. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| iShakir |
| Posted: Wed Feb 12, 2014 1:56 am Post subject: |
|
|
Apprentice
Joined: 07 Mar 2013
Posts: 47
|
Your question / point (2) is a little unclear. When you say "communicate with the container servers" do you mean as part of a message flow doing map operations, or admin tasks such as starting the container servers?
If you mean map operations, the egs don't communicate with the container servers. The communicate as a client to the cache. This in general (as I understand) involves communication with the catalog servers, which may well be on a different broker on a different machine, I doubt JNI is involved.
If you're worried about inter-process communication then just look to scope back what can see the information you want to cache. If the data never changes like in (A), then perhaps you do just want to keep a copy per eg (assuming you can afford to keep multiple copies!)
(B) clearly requires the global cache, as you need data to be available to all message flows, and continue to be available as you scale horizontally.
(C) is an interesting one, if you don't wish to use a global cache, then perhaps you will find you will have an even bigger hit to performance if you have multiple caches that need updating.
EDIT: That's a little muddled essentially the trade off is this:
1.) Scope caching back: Works well on a small scale, as you scale horizontally memory and CPU utilization (for any changes necessary) will scale linearly with number of egs.
2.) Cache globally: Cache can remain static (enough) once you find a HA configuration of catalog and container server that works for you. Then scaling horizontally is simple, as you're simply adding more clients (a little more network traffic), your cache itself should remain using the same amount of resources. |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Feb 12, 2014 2:52 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
You can also use User Defined Configurable services. These are loaded into memory and available to any flow (using JCN to access).
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Feb 12, 2014 6:07 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
In general, with performance questions, you need to accept that you're going to have to run your own tests to verify how things run in your environment with your code.
For your scenarios, for A I would always use User Defined Properties or a User Defined Configurable Service, depending on how complicated the data structures are and what I am trying to access them from. Either way, I would tend to create getter functions to access them, so that I can change my mind about how the data is stored later without affecting the consumers.
For B and C, I would use GlobalCache. I would specifically use GlobalCache for C because I would want to be able to access stored responses across all EGs, regardless of which EG originally processed the request.
If your own performance tests show that the MbGlobalCache functions do not give you the level of responsiveness and control that meets your requirements, you can look at using the eXtremeScale client API java libraries directly within IIB instead. These give you additional control and may give you better performance in some cases.
There *may* be licensing issues with this, so it is worth clarifying with your IBM sales representative that you can use the eXtremeScale client libraries without having a license for eXtremeScale itself - you would still presumably be using IIB as the cache engine. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
