| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| 8.0.0.1 Global Cache - limits, settings, performance, serial |
« View previous topic :: View next topic » |
| Author |
Message
|
| t603 |
 Posted: Tue Dec 11, 2012 10:08 am Post subject: 8.0.0.1 Global Cache - limits, settings, performance, serial |
 |
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
Hello,
I am investigating Global Cache in WSMB 8.0.0.1 on my computer. After several reading of InfoCenter, MQSeries.net I am still confused about some
behavior of Global Cache. Is there someone, who have experiences with Global Cache and want to help me with questions below? All the questions are asked from the point of view comparing Global Cache to the relational or XML database.
Thank You in advance for Your answers. Stepan
1) How to store ResultSet (row), XMLDocument (InputRoot), Properties, Environments... in Global Cache?
According to the [GCOV], only (using Java language) primitive types (byte, short, int, long, float, double, char, boolean) and String can be used for values of both key and value. But if Global Cache could and should supersede storing configuration data and time-limited message properties among message flowsin relational databases (if i understand [GCOV] right), how to store data like "database table with multiple rows with multiple columns" or MQMD of async processed messages in Global Cache? Is there (and if yes, what is recommended) preferred approach to store multidimensional data of objects (like database tables of messages)? Serialization in Java (afaik Global Cache uses Java HashMap)? Or own tree logic like: value is the key of value, which is the key of (final) value) - but this could be dangerous regarding something like "read-commited transaction isolation level" - mentioned in point 6.
2) Are there any guidance of planning, configuring and automatic monitoring of Global Cache?
I read [PEC]. But this is configuration of standalone WebSphere eXtreme Scale, version 7.1.1. Also there is mqsicacheadmin command. But configuration of eXtreme Scale is much more large than using mqsicacheadmin command. How to allocate CPU, RAM, number of JVMs? Or is it necessary - maybe Broker can do this work by itself.
4) Are there any size limits of Global Cache?
Like number of records, size of one record, size of the whole Global Cache...
3) What is the size limit of String stored in value (and key)?
I did not find such information about this in InfoCenter. Especially in case of storing messages and database tables in serialized form this could matter.
4) Are there performance consideration of Global Cache?
I mean something like indexing of keys or etc. like databases have. If I can store really many records ~ dump configuration tables during start of Broker from database to Global Cache. Does Global Cache slow down linear or exponentially while adding records.
5) Parallel reading from Global Cache
How does Global Cache handles with parallel reading the same record by different message flows? Let say I will have some records, which will be almost permanently READED by many flows "in one time". How this Global Cache handle? In parallel? Can this parallelism be somehow supported by admin or developer (some settings)?
6) Global Cache and ACID transaction like approach
Let say, that one node will read tables from database and store data into Global Cache. During writing or reading (during globalMap.put() cycle) something could crash. What is the atomicity of transaction?Each globalMap.put()? Or some begin-end ~ begin-commit-rollback? Is there any wait-till-something-to-make-data-readable mechanism? Does record behave like HashMap or ConcurrentHashMap?
7) Is each globalMap.get() real-time or is cached within node?
Is each globalMap.get() unique or is cached within node like CURRENT_TIMESTAMP ESQL function?
8) What is "null" value of key or value
Byte 0, short 0, int 0, long 0L, float 0.0f, double 0.0d, char '\u0000', String null, boolean false? Or anything else?
References:
[GCJCN] = http://publib.boulder.ibm.com/infocenter/wmbhelp/v8r0m0/topic/com.ibm.etools.mft.doc/bc23770_.htm
[GCOV] = http://publib.boulder.ibm.com/infocenter/wmbhelp/v8r0m0/topic/com.ibm.etools.mft.doc/bn23730_.htm
[HM] = http://docs.oracle.com/javase/6/docs/api/java/util/HashMap.html
[CHM] = http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ConcurrentHashMap.html
[PEC] = http://publib.boulder.ibm.com/infocenter/wxsinfo/v7r1m1/index.jsp?topic=%2Fcom.ibm.websphere.extremescale.doc%2Fcxscapplan.html |
|
| Back to top |
|
 |
| jhart |
| Posted: Wed Dec 12, 2012 4:57 am Post subject: |
|
|
Novice
Joined: 12 Dec 2012
Posts: 19
Location: IBM Hursley, UK
|
Hello Stepan - Although I don't have answers for all of your queries, hopefully the following helps:
1) There's no recommended or preferred approach for storing multi-dimensional objects in the cache. However, at the moment, although keys must be Java primitive types or Strings, values can be any Java serializable object. So it is quite feasible to use your own POJOs, as long as they are available to all Execution Groups that might be working with them.
Regarding storage of MQMDs, if you take a look at the Coordinated Request Reply sample in the product, this has been updated in v8.0.0.1 to demonstrate use of the Global Cache. This sample stores an MQMD as a byte array, having used the MbElement.toBitstream() method to serialize it.
2/3/4) The "Frequently Asked Question" below will hopefully be available as part of a DeveloperWorks article in the near future:
Q: Are there any implications for the size of execution group processes when using the global cache?
A: With a catalog or container server enabled, the heap size of the JVM within an execution group will increase, by at least 30-40MB. There is no impact on the performance of flows within that execution group as a result of hosting a cache component. However, all the data in the global cache will be hosted within your execution group processes, and the size of these processes will grow accordingly. If you create flows that continually write to the cache, and never remove data, the size of your processes will grow, eventually leading to out-of-memory errors. You should consider how much data is likely to be placed in the cache, and refer to the WXS product documentation for guidance on JVM heap size settings.
4/5) The Global Cache capability is provided via embedding components from WebSphere eXtreme Scale within the Execution Group JVMs. WebSphere eXtreme Scale handles concurrency and scaling very effectively. Reads from multiple flows are done in parallel.
6) Another "Frequently Asked Question" type of response here:
Q: Are interactions with the global cache transactional?
A: Each interaction with the global cache is a transaction in its own right. WebSphere eXtreme Scale pessimistic locking is used during each action, and control is only returned to the user once that action has been committed to the primary and replica versions of the data. But the global cache interactions are not integrated with the Message Flow transaction. If a message flow rolls back, after some data has been put into the cache, the data will not be removed. For this reason, you are advised to consider this when designing message flows, and to make cache puts/updates/removes occur as late in the flow as possible. The Coordinated Request Reply sample in the product demonstrates this technique – only storing correlation information into the cache once the flow's MQOutput node has successfully put a message.
7) Each get() is unique.
 Not sure I fully understand this one. You can't put an entry into the cache with a null key. If you want to use a null value, then use the Java null object. i.e. myMap.put(key, null);
James |
|
| Back to top |
|
|
| Y75 |
| Posted: Tue Aug 06, 2013 8:42 am Post subject: |
|
|
Apprentice
Joined: 29 Jul 2013
Posts: 32
|
I am doing research on embedded global cache. Have few questions.
1) What configurations are available in global cache beyond cache policy which I believe is used to configure topology?
2) Can I specify templates for dynamic maps?
3) Can I configure CopyMode? If no, by default what copy mode is used by broker? |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Aug 06, 2013 8:55 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| Y75 |
| Posted: Tue Aug 06, 2013 11:11 am Post subject: |
|
|
Apprentice
Joined: 29 Jul 2013
Posts: 32
|
I am guessing, to specify templates for global cache, I would be looking to provide object grid.xml. In broker api, I am dealing with MbGlobal Map.
I see no methods in MbGlobalMap where I can specify object grid.xml or even set copy mode in manner similar to what is given below
| Code: |
BackingMap bm = dg.defineMap("PERSON");
// use COPY_ON_WRITE for this Map with
// IPerson as the valueProxyInfo Class
bm.setCopyMode(CopyMode.COPY_ON_WRITE,IPerson.class); |
Pardon my ignorance, I am very new to broker.
[/code] |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Aug 06, 2013 11:19 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| Y75 wrote: |
I am guessing, to specify templates for global cache, I would be looking to provide object grid.xml. In broker api, I am dealing with MbGlobal Map.
I see no methods in MbGlobalMap where I can specify object grid.xml or even set copy mode in manner similar to what is given below
| Code: |
BackingMap bm = dg.defineMap("PERSON");
// use COPY_ON_WRITE for this Map with
// IPerson as the valueProxyInfo Class
bm.setCopyMode(CopyMode.COPY_ON_WRITE,IPerson.class); |
Pardon my ignorance, I am very new to broker.
[/code] |
eXtremeScale is available for IIB and WMB V8. If you are needing access to those extended functions, there is a developer package you can download to investigate it.
The point is - especially if you are new to developing message flows - you must be trained. If you are not trained, asking questions like, "is it possible" or "how do I do ..." is frowned upon here because it shows you have not done your own work to find out what you need or you have not attended the required training.
| lancelotlinc wrote: |
| Quote: |
When contacting IBM Software Support it is important to understand what support is available through IBM: Specific product questions: If you have been trained on the product, you can submit specific questions or concerns.
Requests that are not within the scope of IBM Software Support and require you to contact either your Sales or Professional Services representative: Support for untrained customers: Requests for step by step IBM help but do not have the required IBM product training. IBM Education offers customer training if additional staff training is needed. Please check the IBM Web site for specific information about available courses.
Your responsibilities: You need to be aware of your responsibilities when working with an IBM support center. If you do not have the required skill or are unwilling to do the work, you can engage a services provider such as IBM Global Services (IGS) or a business partner to assist you, for an additional fee.
|
Have you attended the required nine days of training?
Here is the roadmap:
https://www-304.ibm.com/jct03001c/services/learning/us/pdfs/roadmaps/wmb_v8_dev.pdf |
The required training will not answer all your queries, nevertheless it provides a foundation upon which people can converse. A common vernacular and vocabulary which is helpful when dialoguing about highly technical issues. Without this commonality between technical people, its very difficult to communicate.
You seem knowledgable about Java and you seem to know what you want. Extend this by achieving the training objectives and downloading the eXtremeScale developer's kit. By comparing what is in eXtremeScale and what MBGlobalCache provides, and discovering these differences, you will gain comprehension over and above most other developers about this specialty.
Hopefully jhart will notice this post and provide direct answers to your questions, as he seems to be the product's author with regard to MbGlobalCache.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Last edited by lancelotlinc on Tue Aug 06, 2013 11:57 am; edited 2 times in total |
|
| Back to top |
|
|
| Y75 |
| Posted: Tue Aug 06, 2013 11:41 am Post subject: |
|
|
Apprentice
Joined: 29 Jul 2013
Posts: 32
|
Thanks for the reply!
Though I have not done training yet but I will be doing in next few days. Rest assured, I am doing my homework before asking questions and have read lot of material on info center.
Now to question at hand, may be we can use extreme scale and get to those functions but is there any documentation on what defaults IIB has used in embedded global cache. Is the copymode COPY_ON READ and WRITE? |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Aug 06, 2013 11:53 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| Y75 wrote: |
Thanks for the reply!
Though I have not done training yet but I will be doing in next few days. Rest assured, I am doing my homework before asking questions and have read lot of material on info center.
Now to question at hand, may be we can use extreme scale and get to those functions but is there any documentation on what defaults IIB has used in embedded global cache. Is the copymode COPY_ON READ and WRITE? |
You're welcome. The copyMode is a backingMap function. When you obtain a MbGlobalCache reference, you are only talking to the shard that resides on the local EG. If the backingMap that connects your shard to the rest of the glass has been configured to use pessimistic locking, the MbGlobalCache reference will reserve a lock on the glass for that particular data item. If not overridden, the default backingMap copyMode is COPY_ON_READ_AND_COMMIT .
Again, its rather pointless to go much further with the discussion unless you actually do this work yourself. Download the eXtremeScale developer's kit and configure the glass in accordance with your business requirement.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Aug 06, 2013 11:59 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| jhart |
| Posted: Mon Aug 12, 2013 6:50 am Post subject: |
|
|
Novice
Joined: 12 Dec 2012
Posts: 19
Location: IBM Hursley, UK
|
Hello - just catching up on this thread, I'll attempt to add some value..
- The WXS grid within WMB (/IIB) has configuration xml files, much like any other WXS grid. However, these are really part of WMB internals at the moment, and should not be modified.
- These files define one backingMap template, with PESSIMISTIC locking, and no CopyMode (which will default to COPY_ON_READ_AND_COMMIT as stated by lancelotlinc).
- It is not currently possible to define your own backing map templates for the embedded grid.
- One way of viewing this is that the embedded grid provided inside WMB has a locked down configuration, and is optimized for use in, and by, WMB. If there's a clear use case to extend this, the development team welcome requirements! (http://www.ibm.com/developerworks/rfe/)
- Of course, if you create your own WXS grid outside of WMB (whether standalone, or in WAS, or in an XC10 appliance), you can configure that however you like. As mqjeff says, you can now connect to such an external grid from WMB/IIB using a configurable service. |
|
| Back to top |
|
|
| Y75 |
| Posted: Mon Aug 12, 2013 7:21 am Post subject: |
|
|
Apprentice
Joined: 29 Jul 2013
Posts: 32
|
| Thanks Jhart, mqjeff and lancelotlinc. Your replies are much appreciated! |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
