| Author |
Message
|
| orman |
 Posted: Mon May 04, 2015 8:32 pm Post subject: Broker automation and scale out |
 |
|
Apprentice
Joined: 08 Aug 2013
Posts: 40
|
Hello all
Lately, we started investigating automation and scale out for the broker environment
What means we want to add another or more than one broker faster and easier as we can by "click"
So as I see it I got two ways to go:
1. Have a good scripting tools for deploy and set a full configuration for my new broker
2. Using the mqsirestore and mqsibackup so in some commands I get new broker running
But, the problem with this one is
Brokers as to have same name
Maintain getting difficult
Cannot use it for broker with another name
Cloning the "master" ha to stop the others
#We still have to deal with:
Deploying new bars
Moving the new version or architecture like multi instance make us to redeploy all the source
*today we do not use Application, which make the flows maintain much better so no deal with spread sources
So, what do think about it? |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue May 05, 2015 4:40 am Post subject: Re: Broker automation and scale out |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| orman wrote: |
| So, what do think about it? |
Option 1.
There's a wealth of material for Chef, Puppet and the rest of that space to build a new, unqiuely named broker with the proverbial single button click. The same technology combined with ANT, MAVEN and all the usual suspects will deal with keeping the code properly synced - there's no particular reason the same code has to be on all your brokers. You could have real time code on some, batch on others.
Or have the same code base on them all. The cat skins in a number of ways and there's no "right" answer.
But Option 2 is road to pain & misery, for at least the reasons you mention.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue May 05, 2015 4:57 am Post subject: Re: Broker automation and scale out |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| Vitor wrote: |
| orman wrote: |
| So, what do think about it? |
Option 1.
There's a wealth of material for Chef, Puppet and the rest of that space to build a new, unqiuely named broker with the proverbial single button click. The same technology combined with ANT, MAVEN and all the usual suspects will deal with keeping the code properly synced - there's no particular reason the same code has to be on all your brokers. You could have real time code on some, batch on others.
Or have the same code base on them all. The cat skins in a number of ways and there's no "right" answer.
|
What amazes me is the number of posts here that indicate that indicate the lack of scripted builds at sites. This includes queue managers as well as brokers.
for that alone you deserve 
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue May 05, 2015 5:04 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
You can write an absolutely dead simple shell script to run a series of mqsi commands to create and provision a broker.
If you don't know the mqsi commands to create and administer brokers, then it's well past the time you should have learned - so get to it.
If you want to get fancier, you can use Ant to execute those same commands.
If you want to get fancier, you can build a docker container that will run a series of customizations based on incoming environment variables - again, using the correct mqsi commands to create and provision a broker.
If you want to get fancier you can use Chef or Puppet or etc. to create a more automated way to deploy, customize, and decommission docker containers.
There are also a number of options with plain Virtual Machines.
If you've done none of this before, start with shell scripting. |
|
| Back to top |
|
|
| orman |
| Posted: Tue May 05, 2015 12:13 pm Post subject: |
|
|
Apprentice
Joined: 08 Aug 2013
Posts: 40
|
So you recommend maintain all source and using BAR files which includes the right configuration as we want on the Production environment
and in addition the version so if i want to make a new environment - i need to have all the pre-version as a base. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue May 05, 2015 12:29 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| orman wrote: |
| So you recommend maintain all source and using BAR files which includes the right configuration as we want on the Production environment |
| orman wrote: |
| and in addition the version so if i want to make a new environment - i need to have all the pre-version as a base. |
Pre-requistite rather than pre-version. You need to store what version of IIB, how it's installed, what execution groups you have and other such information so when you make a new environment it's correctly set up for the bar files you're about to drop into it.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue May 05, 2015 12:53 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| orman wrote: |
So you recommend maintain all source and using BAR files which includes the right configuration as we want on the Production environment
and in addition the version so if i want to make a new environment - i need to have all the pre-version as a base. |
So that when you have a PROD outage you can quickly create a COMPLETE working replica of the PROD in Minutes knowing that it will work just as well as the defunct/disabled prod system.
Makes sense?
If so then you know what to do.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| orman |
| Posted: Tue May 05, 2015 8:53 pm Post subject: |
|
|
Apprentice
Joined: 08 Aug 2013
Posts: 40
|
| mqjeff wrote: |
You can write an absolutely dead simple shell script to run a series of mqsi commands to create and provision a broker.
If you don't know the mqsi commands to create and administer brokers, then it's well past the time you should have learned - so get to it.
If you want to get fancier, you can use Ant to execute those same commands.
If you want to get fancier, you can build a docker container that will run a series of customizations based on incoming environment variables - again, using the correct mqsi commands to create and provision a broker.
If you want to get fancier you can use Chef or Puppet or etc. to create a more automated way to deploy, customize, and decommission docker containers.
There are also a number of options with plain Virtual Machines.
If you've done none of this before, start with shell scripting. |
Yes I know all the mqsi commands and we got a script which deploy all bar files
But It does not work in versioning so it is not that good for me
Today when we deploy code it made without install kit what can make mistake and no future automation for deploy |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue May 05, 2015 9:39 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| orman wrote: |
But It does not work in versioning so it is not that good for me
|
would you care to explain what you mean by 'not work in versioning'?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed May 06, 2015 4:55 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| smdavies99 wrote: |
| orman wrote: |
But It does not work in versioning so it is not that good for me
|
would you care to explain what you mean by 'not work in versioning'? |
Case in point:
This site has a system (with, in the interests of full disclosure, some custom and proprietory Java) that performs the sort of task you're describing. We use it not to create new PROD systems in the event of an outage, but to ensure the bar files deployed to our Prod system are also deployed to the spare ("warm") Prod system and also to build new Prod systems for scaling purposes. This involves spinning up a new virtual server, installing WMQ, IIB and the other odd componentes, configuring the broker and pushing the correct version of the needed bar files. This is all done with script, ANY with the custom Java mostly in the "who needs to approve all of this" workflow area.
My 2 cents.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
