| Author |
Message
|
| sebastia |
 Posted: Wed Dec 24, 2008 3:40 pm Post subject: Tool to save CM domain configuration |
 |
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Merry Xmas to you all !
We have a brand new MB domain, version 6.1
And we are preparing the backup + recovery procedure.
What we need is quite simple (at first sight) ...
A script that has to ...
a) scan all the domain of a given Configuration Manager,
and for every Message Broker in it ...
b) save all the Broker data required to restore the domain
The output of the script has to be a text file
that helps to restore the domain in case of recovery ...
... maybe a script, why not ?
My question is : DOES SUCH A TOOL ALREADY EXIST ?
We are thinking on quite a simple data structure :
cm-name
|
[broker-name]*n
|
[execution-group-name]*n
|
[bar-file-name]*n
And we dont want to re-invent the weel, do we ? |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed Dec 24, 2008 6:51 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Wouldn't you want to take a backup of the config mgr's DB for that. For the procedure see the chapter about moving a config mgr from one box to the other.
Have fun. 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| sebastia |
| Posted: Thu Dec 25, 2008 2:39 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Thanks, mr Saper.
I have read the thread where mr Peter Potkay asks himself
"Is it worth backing up the Config Manager?"
and this is NOT deifinitely the tool for me.
I was thinking on a schell that does
a) detect the configuration of the domain,
and
b) generates a (magic) script that contains
... one "mqsicreateconfigmgr" sentence
... few "mqsicreatebroker" sentences
... few "mqsicreateexecutiongroup" sentences
... few "mqsideploy" sentences
so, after running this magic script,
we shall have the domain re-build from scratch.
What do you think ?
( ) |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Dec 25, 2008 6:31 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| sebastia wrote: |
Thanks, mr Saper.
I have read the thread where mr Peter Potkay asks himself
"Is it worth backing up the Config Manager?"
and this is NOT deifinitely the tool for me.
I was thinking on a shell that does
a) detect the configuration of the domain,
and
b) generates a (magic) script that contains
... one "mqsicreateconfigmgr" sentence
... few "mqsicreatebroker" sentences
... few "mqsicreateexecutiongroup" sentences
... few "mqsideploy" sentences
so, after running this magic script,
we shall have the domain re-build from scratch.
What do you think ?
() |
This is more than just rebuilding the domain. This is rebuilding the whole environment. And it should be done from stuff that is in the version control system.
I would suggest that as long as you're at it you also use MS03 and recreate the qmgrs.
So using the config manager proxy you can certainly determine the brokers, and execution groups. Looking at the flows and deployed objects in the egs you could further determine the bar file name.
Now you will need to program this so that the output can be used in multiple scripts. Remember that mqsicreatebroker needs to be run on the broker machine etc...
So your output is:
- a series of scripts generated by MS03 to rebuild the qmgrs.
- From version control a list of changes/additions to the relevant qm.ini
- A list of brokers (broker name) source config mgr proxy / version control
- A list of execution groups (names) per broker: source config mgr proxy/version control
- A list of bar files per execution group/per broker: source config mgr proxy / version control
- A list of collectives with their members : source config mgr proxy / version control
- A list of topics : source config mgr proxy / version control
- A list of durable subscriptions per broker : source config mgr proxy/ version control
Your list of operations for recreating the environment would be:
- ready the hardware with OS and Software (MQ, Message Broker, DB)
- Create the qmgrs (should be from version control)
- rebuild the qmgrs (MS03 from version control) + changes (qm.ini)
- restore permissions on mq running the setmqaut scripts from version control (source:MS03)
- recreate the brokers and config mgr using scripts from version control
- add the brokers to the domain using scripts from version control
- recreate the execution groups using the config mgr proxy and scripts (version control)
- recreate the permissions on the config mgr using scripts from version control (mqsicreateaclentry)
- redeploy the bar files using the config mgr proxy and scripts from version control
- redeploy the topics using the config mgr proxy and scripts from version control
- recreate and redeploy the collectives using the config mgr proxy and scripts from version control
- restoring the durable subscriptions
This is best done using a subscription flow on the broker
You can pass the subscription information in xml format in a message.
The flow will then move the relevant part of the xml to the RFH header and send the message to the SYSTEM.BROKER.CONTROL.QUEUE.
It is best to make the message a request/reply type with full report.
You can then examine the reply queue and verify that the subscription was succesful
Of course you realize that in order to do all this you will need to do a lot of programing using the config mgr proxy...
a) to generate load scripts
b) to be able to use the load scripts in an automated way
There are also a number of scripts that you should place under version control if they aren't already there, like the ones calling mqsicreatebroker, crtmqm. etc...
Hope this answers your query.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| sebastia |
| Posted: Thu Dec 25, 2008 1:00 pm Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
EXCELENT, mr Saper !
I read an implied answer : there is no tool for me ....
Thanks a lot - time to sit and think on it !
Seb. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Dec 26, 2008 5:27 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
We need an MS03-type utility for WMB.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Dec 26, 2008 7:22 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| PeterPotkay wrote: |
| We need an MS03-type utility for WMB. |
It's not actually possible. There's no reasonable mechanism for recreating a BAR file from a running broker.
Even if mqmatt added a "recover bar file" operation to the configmgr proxy API, it's not clear that the situation would be much better.
Right now you can just reinstall WMB somewhere, create a broker of the same name, and then restore the broker database over the "new" broker database.
And of course, recreate all of your site specific customizations of mqsiprofile and shared-classes and jplugin and etc. etc. etc. etc. etc...
I expect that there will continue to be product supplied means to back up and recover a broker runtime environment.
I do not expect that to ever replace entirely the need to keep your broker development artifacts (flows, sets,schemas,etc) under source control. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Dec 26, 2008 8:53 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
I can't help think that some of the thoughts expressed here are a bit arse about faced. By this I mean that instead of thinking about backing up a CM and everything it deploys, it is better to be able to rebuild it from the ground up in a reliable and consistent manner is a better solution.
Surely by using scripts to create the
- QM's
- Broker DB
- CM & Broker
- EG's
And then
- Deploying the Bar files via a script(s)
and with everything in stored SVN or some other Versioning system the recreation of a complete environment should be done within an hour or so with little or no chance of 'human error'
At the moment most of the above can be scripted and GUI interaction can be kept to a minimum. Setting individual QM's to autostart in Windows is AFAIK the only thing I can't automate.
Repeatability & Auditability are key measures when desiging these sorts of systems. If you get these right then problems can be recreated elsewhere knowing that you are using the same 'stuff' that went into building the original systems.
You can create logs of many parts of the above steps. What I think is missing is for the CM to log the deploy/delete requests in detail.
This way the CM logs can be archived. Then any rebuilds can have their CM logs compared with the originals.
By detail I mean everything about the deploy. Who, what including any version data from the Bar files, when and to which broker the operation was done.
To me this is of far more use in recreating an environment than any ability to back a CM on its own.
Some of the data is probably already recorded but the ability to run a 'report' on the deployed system woul be of great interest to me.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| sebastia |
| Posted: Fri Dec 26, 2008 9:56 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Peter - MS03-kind-of-tool is EXACTLY what I was requeting !
()
Sure I am very novice to MB, so superior minds opinions are required
Mr Davies99 - when you say ... "rebuild it from the ground" ...
... what source/origin are you pretending to use ?
ClearCase would be a good / natural offering, am I wrong ?
Let's have a nice 2009 we all ! |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Dec 26, 2008 11:54 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Any of the more common (and even not so common) Version control Systems can be used.
Personally, I prefer Subversion or 'git' (I'm a Linux Dev in my spare time) but Clearcase can easily be used instead to hold the scripts.
All you need is to understand the check in/out commands from the command line. Then you can check out the sets of scripts for the individual components. eg Db2 Database, WMQ QMgrs, Broker Creation, Broker Deploy etc.
I have a set of template scripts for windows, Linux & Unix which I can quickly modify to provide the basis of a set of creation & deploy scripts for a broker project. These are part of my 'toolkit' so to speak.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| sebastia |
| Posted: Fri Dec 26, 2008 12:05 pm Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
To say the truth, if I am new to MB ...
... to ClearCase I am ... a novice (or a null-ity)
mr davies : is there any chance you can share
( an initial version of )
your scripts ?
>>> campdefabes@gmail.com
Just to have a clue of the direction to follow ...
Dont worry to say "not, impossible" - I can understand.
( ) |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Dec 26, 2008 2:49 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
The scripts are part of my Toolkit as I indicated. As such, I use them in my Consulting Jobs so I really can't give them away for free even at this time of year.
You will undoubetdly learn more by creating your own set.
That said, a few guidelines.
- Make the commonly changes things script variables. eg the path to the WMQ - Log Files & the number/size of the log files.
- Separate the different components into separate directory trees. egy a directory tree for WMQ scripts with a separate directory for each QMGR.
- Don't be afraid to call subscripts from main ones. eg, to deploy a bar file.
- Keep them simple and well commented.
- On Windows, use setlocal & endlocal to avoid scoping problems with deployment operations.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| sebastia |
| Posted: Sat Dec 27, 2008 3:40 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Thanks for the guidelines, mr Davies !
I really appreciate your time and patience ... |
|
| Back to top |
|
|
|
|
