| Author |
Message
|
| smdavies99 |
 Posted: Sat Sep 07, 2013 6:02 am Post subject: Global Cache - Catalogue Server |
 |
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Broker 8.0.0.2 Windows.
I've created a broker with the global cache defined.
I was looking at how I could make fewer single points of failure in the Broker setup. For example, If the EG where the Catalogue Server is running abends then you have lost your cache.
It seems that you can only have one catalog server running in a broker.
I've created two Catalog Servers but only one ever starts. The other fails and the EG keeps reporting BIP11113 errors.
Is this correct?
If so, then what is the downside (if any) of creating an EG that runs ONLY the Catalogue Server?
That way, because no flows run in the EG it can't abend and wipe out your cache.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Mon Sep 09, 2013 3:33 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Sep 09, 2013 4:25 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
I'm still a bit confused. do you mean that two catalogue servers is onlt supported in the full version of eXtreme?
Is there any problem with creating a EG that only runs the Catalogue Server?
Having the Catalogue Server running configure with an active EG is a point of failure that I'd like to avoid if possible.
The full version of eXtreme costs money on top of broker and therefore that option is not possible here (my customers despite spending billions on a a new 'toy' are really stingy when it comes to critical operating software.)
Even getting them to stump up for a full Enterprise Broker license is really hard work.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Sep 09, 2013 4:32 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
You can do what you are suggesting. There is no reason why you can't.
The embedded GlobalCache configuration with WMB/IIB is not currently modifiable as you could with the eXtremeScale product's poilcy.xml file.
One question I would challenge you with is, why do you need to run a separate EG for the catalogue server ? The catalogue server is a stand-alone executable and does not need to be run inside a WMB EG. It can be run by your system startup scripts and be totally independent of WMB.
If your EGs are that unstable, I would look at your WMB code rather than try a band-aid approach .
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Sep 09, 2013 5:07 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| jhart |
| Posted: Mon Sep 09, 2013 5:59 am Post subject: |
|
|
Novice
Joined: 12 Dec 2012
Posts: 19
Location: IBM Hursley, UK
|
Yes, you can absolutely run with multiple catalog servers - either within a single broker or across multiple brokers.
The simplest way to achieve this, with a single broker, is to use the sample\globalcache\policy_one_broker_ha.xml file under your product install.
- Make a copy of this file somewhere convenient.
- Modify this copy to reflect your broker name and hostname.
- Change your broker-level cache policy property to be the fully-qualified location of the new policy file. (also note that the listener host specified in the file must match the broker-level Global Cache listenerHost property)
- Restart.
- As long as you have more than one EG, your cache will have two catalogs.
When running with two catalogs, the startup time is a little longer, due to handshaking etc. But you do eliminate that single point of failure (either of those catalogs could go away, and the cache would still be available).
Global Cache is intended (and thoroughly tested!) to share data between execution groups - either in a single broker or between multiple brokers.
The following article explains some of the concepts and more advanced configuration available:
http://www.ibm.com/developerworks/websphere/library/techarticles/1212_hart/1212_hart.html
Where I was quoted below:
| Quote: |
the embedded grid provided inside WMB has a locked down configuration, and is optimized for use in, and by, WMB.
|
This relates to the underlying WebSphere eXtreme Scale configuration. For example, number of partitions, locking strategy, replication settings, backing map definitions.
But you do get to control the topology of your cache - how many brokers it covers, how many catalog servers it contains etc. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Sep 09, 2013 6:06 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| jhart wrote: |
Where I was quoted below:
| Quote: |
the embedded grid provided inside WMB has a locked down configuration, and is optimized for use in, and by, WMB.
|
This relates to the underlying WebSphere eXtreme Scale configuration. For example, number of partitions, locking strategy, replication settings, backing map definitions.
But you do get to control the topology of your cache - how many brokers it covers, how many catalog servers it contains etc. |
@jhart - so the line is blurry between what is permitted and what is not. Perhaps a more important question is why?
The embedded GlobalCache functionality is limited due to what reason ? Would it not just be easier to provide a fully functioning cache product ? Is the limitation due to technical hurdles or business hurdles ?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Sep 09, 2013 7:04 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Thanks. I'll look at using the policy feature.
At the moment I'm getting lots of these errors
| Code: |
( DEVBRKR.EG_0 ) This execution group has failed to establish a connection to the broker global cache. The attempt to connect to the remote server components has now been running for ''91100'' seconds. The connection is being made using the connection string ''192.168.1.76:2809''.
|
I can see the problem. Here's how I create the broker (some bits missing)
| Code: |
mqsicreatebroker DEVBRKR -i TestUser -a MyPassword1000 -q DEVBRKR
mqsichangebroker DEVBRKR -b none -r 12809-12830
mqsichangebroker DEVBRKR -f all
mqsichangeproperties DEVBRKR -e EG_CACHE -o ComIbmCacheManager -n enableCatalogService -v true
mqsichangeproperties DEVBRKR -e EG_CACHE -o ComIbmCacheManager -n enableContainerService -v true
mqsichangeproperties DEVBRKR -e EG_CACHE -o ComIbmCacheManager -n enableJMX -v true
mqsireportproperties DEVBRKR -e EG_CACHE -o ComIbmCacheManager -r
mqsichangeproperties DEVBRKR -e EG_0 -o ComIbmCacheManager -n enableCatalogService -v true
mqsichangeproperties DEVBRKR -e EG_0 -o ComIbmCacheManager -n enableContainerService -v true
mqsichangeproperties DEVBRKR -e EG_0 -o ComIbmCacheManager -n enableJMX -v true
mqsireportproperties DEVBRKR -e EG_0 -o ComIbmCacheManager -r
|
The EG_0 config needs to be told that the Ports are different from the default.
I'll fix that and look at the policy file options.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| jhart |
| Posted: Mon Sep 09, 2013 7:29 am Post subject: |
|
|
Novice
Joined: 12 Dec 2012
Posts: 19
Location: IBM Hursley, UK
|
@smdavies99 - I would advise not using "-b none" to begin with. i.e. something like (again, bits missing) :
| Code: |
mqsicreatebroker DEVBRKR -i TestUser -a MyPassword1000 -q DEVBRKR
mqsichangebroker DEVBRKR -b <path to policy file>
mqsichangebroker DEVBRKR -f all
mqsistart DEVBRKR
mqsireportproperties DEVBRKR -e EG_CACHE -o ComIbmCacheManager -r
mqsireportproperties DEVBRKR -e EG_0 -o ComIbmCacheManager -r |
You can then switch to "none" and tweak some of the settings. The reason I suggest this is that a few of your ComIbmCacheManager properties are likely to be wrong at the moment (things like catalogClusterEndPoints). If you start with a policy and report the properties, you'll probably see what I mean.
@lancelotlinc - Fair questions.
You are right in saying that there are a few restrictions, but the the function provided is designed to meet the needs and requirements of the majority of Broker users - in a way that's intuitive and consistent with other parts of the product. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Sep 11, 2013 7:24 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
I switched to using a policy file and now have Three EG's configured as follows
| Code: |
C:\scripts\wbiscripts\devbrkr>mqsireportproperties DEVBRKR -e EG_LOGS -o ComIbmC
acheManager -r
ComIbmCacheManager
uuid='ComIbmCacheManager'
userTraceLevel='none'
traceLevel='none'
userTraceFilter='none'
traceFilter='none'
overrideTraceSpec=''
enableCatalogService='true'
enableContainerService='true'
enableJMX='true'
listenerPort='12809'
listenerHost='192.168.1.84'
haManagerPort='12810'
jmxServicePort='12811'
connectionEndPoints='192.168.1.84:12809,192.168.1.84:12813'
catalogClusterEndPoints='DEVBRKR_192.168.1.84_12809:192.168.1.84:12812:12810,D
EVBRKR_192.168.1.84_12813:192.168.1.84:12816:12814'
domainName='WMB_DEVBRKR_192.168.1.84_12809_DEVBRKR_192.168.1.84_12813'
clientsDefaultToSSL=''
sslProtocol=''
sslAlias=''
status='both'
BIP8071I: Successful command completion.
|
How can I check that I do actually have two Catalogue servers running?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| jhart |
| Posted: Wed Sep 11, 2013 7:53 am Post subject: |
|
|
Novice
Joined: 12 Dec 2012
Posts: 19
Location: IBM Hursley, UK
|
You'll have two BIP7162 messages in your event log (one for each catalog).
Or look at the ActivityLog for the GlobalCache resource manager in each EG.
All of them should have a BIP11109I (connected to the cache).
If you only have 3 EGs, then they should all also have a BIP11114I (container server started).
Two of them will also have BIP11111I (catalog server started).
Also in the mqsireportproperties output above, you'll see :
This means that EG is hosting catalog + container. Moreover, both the catalog and container have started ok. (Otherwise it would say "failed").
Finally, if the cache is available and working, then both catalogs must be up. It will not work at all until a majority of catalog servers are available. When running with 2 catalogs - this means both of them! |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Sep 11, 2013 8:39 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Thanks.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Sep 11, 2013 10:58 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
I've noticed that it takes a relatively long time to get the EG connected to the Catalogue Server. for example
| Code: |
This log was generated on: Thu Sep 12 07:52:42 BST 2013
Message Number,Timestamp,Message Summary,
BIP11112W,12-Sep-2013 07:42:01.000 BST, Retrying start of cache components,
BIP11112W,12-Sep-2013 07:42:31.000 BST, Retrying start of cache components,
BIP11111I,12-Sep-2013 07:42:34.000 BST, Started catalog server,
BIP11114I,12-Sep-2013 07:42:34.000 BST, Started container server,
BIP11109I,12-Sep-2013 07:42:35.000 BST, Connected to cache 'WMB',
|
Ok, this may only be 30 seconds but as the saying goes, 'Every Second counts'.
some of our customers are demanding an SLA with a max downtime of 5 mins a year but will only pay for an active-passive HA solution.
That would give us time for only TWO failovers a year.
time for some re-negociation methinks.
The question is though, is the EG able to process messages during this 30 second or so window? I really hope not especially if the flows are dependent upon the Cache availability to operate.
I'll be doing some more tests later today but a heads up would be nice.
finally, it appears that using the policy file configuration method you can't fix where the catalogue server run. Is this correct or do I need to dig into the Websphere eXtreme docs to find it?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Sep 12, 2013 3:08 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
30 seconds is typical when running more than one shard of glass due to the handshaking that occurs during startup. In some cases it can be as long as 90 seconds or more. You may like to add a footnote in your SLA document.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
|
|
