| Author |
Message
|
| mqjeff |
 Posted: Thu Feb 26, 2015 7:31 am Post subject: DevOps/Administrative Automation Questions |
 |
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
How many of you are doing things that you actually consider "DevOps", rather than "development" or "administratoin"?
When you automate things as part of this, what kind of things do you use to do that? - Scripting?
- Programming?
- Configuration for an automation tool?
- Does it depend on the infrastructure you're working with?
What do you prefer to use?- shell-level scripting?
- perl/ruby level scripting?
- java level development?
- Rest/HTTP interfaces?
Even if you're doing "pure" administration, how do you automate things? |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Feb 26, 2015 10:24 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
My 2 cents:
We use shell scripting to automate administrative tasks (creating queues, creating configurable services, creating ODBC data sources, etc) for both WMQ & IIB. In lower environments these scripts are available to developers so they can perform their own administration in an otherwise controlled environment. We've scripted much of the install of the software and once we've got the last of the manual steps out of it (which is tasks performed by other teams like OS install & SAN allocation) the intention is to use this to provide stand up / tear down development environments.
We use script because it's quick and everything is currently Linux. The medium term goal is to use Chef or Python for the install process, and move the administrative functions into a web page, probably exploiting Java under the covers to operate the software (e.g. CMP for IIB).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Feb 26, 2015 10:28 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Vitor wrote: |
| We use shell scripting to automate administrative tasks (creating queues, creating configurable services, creating ODBC data sources, etc) for both WMQ & IIB. |
Is that relatively consistent across other teams that do product admin/operations at your site?
If you had a brand new product that you were responsible for, how would you prefer to administer it? |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 26, 2015 10:49 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| Vitor wrote: |
| We use shell scripting to automate administrative tasks (creating queues, creating configurable services, creating ODBC data sources, etc) for both WMQ & IIB. |
Is that relatively consistent across other teams that do product admin/operations at your site? |
It's broadly consistent, and is becoming more consistent as time passes; we're moving to a new data center model and manual processes are heavily frowned upon by people high enough up the management chain for their frown to matter. Where we still struggle is,as I alluded to above, where we cross responsibility boundaries so even if all the tasks are scripted, we can't invoke them in a single pass.
We're leading the charge because we've been using shell script for this kind of automation for years (i.e. since not long after I started here) so we have more stuff already in place.
| mqjeff wrote: |
| If you had a brand new product that you were responsible for, how would you prefer to administer it? |
Well I do have a new product - we're installing IIBv9 clean in the new data center with no reference to the previous install. I would prefer to be using Chef / Puppet / something for the whole build and have a web based administration tool for developers fully integrated with our automatic deploy process but we lack time & skills, as well as political will to force all the stakeholders into a room. So for the time being developers have to include administrative items in with code and the deployment process works it out, with a better day in the 3-5 year plan.
If you want a more detailed discussion, including details I'm disinclined to post on a public forum, you know how to reach me. 
(At the office. Never call me at home; my wife may become suspicious.....)
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Feb 26, 2015 11:04 am Post subject: The Lowest Common Denominator |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
for example, ed (and vi) are present on just about every Unix/Linux system.
Not every system has Ruby (or even Java).
If you create a configuration process that relies on the presence of other tools then what happens when you encounter a site where tools like Puppet, Ruby etc are not deployed or even worse Not allowed to be deployed.
If you are working in a single company environment (like Vitor) then the above may well not be so important. If you take your skills from company to company then they are vitally important.
so...
1) use shell script for everything.
a) standardise on KSH if you use Unix because BASH is sometimes and option
b) Use plain DOS shell commands for Windows.
c) capture the command window/consoe output if possible with usilities lke 'screen' if allowed.
2) Use template scripts wherever possible.
My templates have all the bits you change in a separate script that is called by the main script.
3) Document everything. Some sites need to go through this with a fine toothcomb. Then you have the answer to the thorny question 'what happens if you get run over by a Bus'
4) Put everything into Source control.
a) have scripts that put it in and get it all out of different source control systems.
5)be pedantic and bloody minded. The procedures must be right first time and everytime.
Finally, keep it simple stupid.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Feb 27, 2015 5:47 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| I'm really hoping for a wide set of replies. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Feb 27, 2015 5:52 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
There is no real standardization in little shops that have less than 15 queue managers... It will all depend on the MQAdmin and the number of scripts that have been setup, and whether or not the scripts will even be followed or whether the MQExplorer will be used to create the new qmgr...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Feb 27, 2015 5:54 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Jeff, I am not a fan of the continual build-up and tear-down approach.
Any more than I do with my house. 
However don't overestimate the technical capabilities of admin people.
I don't want to code anything more complex than shell scripts. Absolutely not Java.
Possibly simple Perl is OK, but IBM must provides lots of working examples and not just publish the API and walk away.
A HTTP/Rest might be OK, if a fully working example was provided that I can easily modify to add/remove some fields.
It's a lot easier to tweak something that is ready to use, than to start from nothing.
No compiled languages please.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Feb 27, 2015 5:58 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| zpat wrote: |
| However don't overestimate the technical capabilities of admin people. |
I assure that one of the points of this question is to make sure that other people aren't making this overestimation. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Feb 27, 2015 6:55 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| zpat wrote: |
Jeff, I am not a fan of the continual build-up and tear-down approach.
|
IMHO, this is just what you must do in order to validate your build process before it gets anywhere near Production.
From my days as a student, the Maths Lecturers used to call it 'going back to first principles'.
With the current fixation for all things 'Cloudy' and 'spinning up a new instance of XXXX then being able to know that the new instance is built properly is mandatory before you can bring it online.
I go to a new site with a release package. In it is everything I need to configure MQ and IIB, deploy all the Applications (flows etc) and have utilities such as rfhutil, q etc just a click away.
By standardising on the whole thing, I can create my own versions of the any of the environments (from all over the world) we support within minutes.
It also means that others(who are usually less skilled in MQ etc) can do the work knowing that it has all been proven beforehand.
I know that this is a very different approach to those who have very stable environments. There is no one fix that is a panacea for all ills.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Feb 27, 2015 8:04 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
We have relatively few, highly centralised, very highly available QMs and Brokers with HA/CMP and SRDF data replication.
This saves license fees. It offers the necessary availability for an environment with a lot of dosh at stake.
This sort of stuff cannot be torn-down and rebuilt willy-nilly. It runs 24x7 etc etc.
Companies that are more distributed in nature (e.g. branches everywhere) may prefer ease of environment duplication.
We only build new QMs or brokers once every few years, new EGs maybe once or twice a year.
Arguably it's more of a mainframe mindset - but that is my background.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Feb 27, 2015 9:37 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Please don't feel shy about giving your opinions. This isn't a trick question at all, nor anything that I'm only asking of long standing members.
If you've got an opinion or something you'd like to share on this topic in general, please respond. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Mar 04, 2015 6:53 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| mqjeff wrote: |
Please don't feel shy about giving your opinions. This isn't a trick question at all, nor anything that I'm only asking of long standing members.
If you've got an opinion or something you'd like to share on this topic in general, please respond. |
  |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Mar 05, 2015 4:15 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
Jenkins,
svn/Git,
Maven,
IBM Integration API |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Thu Mar 12, 2015 9:22 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
I am here using Shell Scripts and Java codes for more details analysis
Java is mainly used to detail out the flows and understanding what exactly switched on and all (can do the same with the help of shell)
I do work on Apps Server too, so python/jython for that,
For DB check and all (First hand Check) - using WebInterface - developed using javascripts and all
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
|
|
