| Author |
Message
|
| ruimadaleno |
 Posted: Tue Jul 19, 2016 1:32 am Post subject: Dependency Management in broker/iib |
 |
|
Master
Joined: 08 May 2014
Posts: 274
|
Hi all,
i'm trying to understand how to properly manage message flows dependencies on broker/iib environments.
from an operator/sysadmin perspective when a deploy takes place every dependency should be documented and shipped in deploy package. But then time goes by and a new server is added, or the system is upgraded and then this dependency info is somehow lost/broken
If dependency is a Library this can easily be checked by broker web console/broker explorer (maybe using a command like mqsilist - not shure) but when we are talking about a JAR file, used by a java compute node in the message flow, that must be copied to the Shared-Classes folder it gets tricky to manage 
so my question here is , how do you folks manage dependencies in a broker environment ?
i'd like to ear both developer and sysadmin perspectives on this matter
best regards
Rui Madaleno
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Tue Jul 19, 2016 2:46 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
We have a master file that containes details of the Flows and what EG's that are needed,
We can generate a set of scripts that will :-
1) Create the broker and provision all the static items
2) Create the configurable services related to the flows
3) deploy all the flows to a new broker.
4) create delta deployment scripts for new/updated flows etc
These scripts, the bar files, a backup of the broker before and after deployments (And the master file) are also saved in a Git Repo as a record of the build of that broker. We can return to ANY version of any Broker at will.
Takes some time to put together but worth it.
We handle the MQ QMGRS in the same way. Everything is scripted.
Others will have different approaches. Each will work for that organisation and may not be suitable for others.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| ruimadaleno |
| Posted: Tue Jul 19, 2016 3:29 am Post subject: |
|
|
Master
Joined: 08 May 2014
Posts: 274
|
| smdavies99 wrote: |
We have a master file that containes details of the Flows and what EG's that are needed,
We can generate a set of scripts that will :-
1) Create the broker and provision all the static items
2) Create the configurable services related to the flows
3) deploy all the flows to a new broker.
4) create delta deployment scripts for new/updated flows etc
These scripts, the bar files, a backup of the broker before and after deployments (And the master file) are also saved in a Git Repo as a record of the build of that broker. We can return to ANY version of any Broker at will.
Takes some time to put together but worth it.
We handle the MQ QMGRS in the same way. Everything is scripted.
Others will have different approaches. Each will work for that organisation and may not be suitable for others. |
Hi smdavies99
whooaa heavy organization in place ! 
just curious, what is the format of the "master file" you mention ? are you using some form of xml structured file or something more "usual" as an excel file ?
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Jul 19, 2016 5:01 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
If you think that is 'heavy organisation' then as the saying goes 'you aint seen nothing yet'.
as for the file... it is in XML format and we have some java that does the parsing of it and produces the various 'mqsi' commands.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jul 19, 2016 5:07 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| smdavies99 wrote: |
| If you think that is 'heavy organisation' then as the saying goes 'you aint seen nothing yet'. |
This site spreads the IIB development across 200+ development teams with all the developed objects deployed to one of four Production environments mirrored across two data centers.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jul 19, 2016 5:42 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Building this kind of infrastructure is handled these days by people known as 'devops'...
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Jul 19, 2016 6:06 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| mqjeff wrote: |
| Building this kind of infrastructure is handled these days by people known as 'devops'... |
A
If that is the case, can I have
- My current Salary as an architect
Plus
- the going rate for that mythical thing called a 'DevOps' practicioner?
Plus
- The going rate for a Developer
Oh silly me, I'm expected to do all of the above already.
Aren't small orgs wonderful.
but I'm glad to be retiring soon. I can't take much more SnakeOil.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jul 19, 2016 6:29 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| smdavies99 wrote: |
If that is the case, can I have
- My current Salary as an architect
Plus
- the going rate for that mythical thing called a 'DevOps' practitioner?
Plus
- The going rate for a Developer |
Yes, yes you can. 
Mine is currently paying my wife's medical bills.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jul 19, 2016 6:34 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I've had good luck with "I expect this job pays...", not "How much does this job pay"...
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| ruimadaleno |
| Posted: Tue Jul 19, 2016 8:30 am Post subject: |
|
|
Master
Joined: 08 May 2014
Posts: 274
|
And how do you enforce the Master File usage ? how do you guarantee that this master file is up to date ? what is the governance model (if one is in place , maybe it's a single person effort to get the job done  )
every sysadmin that updates some config in broker environment (example: increase additional instances) must update the master file ?
every developer must update Master File when he/she delivers a new message flow ?
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Jul 19, 2016 8:42 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Every site has their own release procedures. How strictly they are adhered to is down to the people involved and what happens to those who mess things up by not following the procdures.
It is up to you to develop your own policies and procedures
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jul 19, 2016 8:58 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ruimadaleno wrote: |
| what is the governance model |
We use a clearly defined set of documentation and a central repository that are scrutinized at defined points in the development process and enforced (ultimately) by our change control process.
| ruimadaleno wrote: |
| every sysadmin that updates some config in broker environment (example: increase additional instances) must update the master file ? |
Yes
| ruimadaleno wrote: |
| every developer must update Master File when he/she delivers a new message flow ? |
Yes
| smdavies99 wrote: |
| How strictly they are adhered to is down to the people involved and what happens to those who mess things up by not following the procedures. |
In addition to the above, we have procedures to detect unauthorized changes. Depending on where you are on the curve between "did the change and retroactively filled out the change request" and "didn't bother with a change request because it's a really small change unconnected to anything else and a change request is a waste of my time", penalties range from an awkward conversation with the CTO (who take the change process seriously) to a final conversation with HR.
Penalties are doubled if you manage to break something. The CTO is paged for all production outages and gets grumpy. Bad if you have a CR, worse if you don't.
  
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Jul 19, 2016 11:20 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
The size and scope of the policies and procedures depends entirely upon what already exists in your organisation. Some have really strong processes. Others don't.
Don't try to re-invent the wheel whatever you do.
Otherwise you will be hung out to dry.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| ruimadaleno |
| Posted: Sat Jul 23, 2016 2:41 am Post subject: |
|
|
Master
Joined: 08 May 2014
Posts: 274
|
so, from what i can understand you have developed a xml schema format where the caracteristics/properties, deployed msgflows/apps are listed. This xml file is kept in some version control system to allow retrieve/versioning and the "secret ingredient" is somekind of policy in place (stronger or ... not ... it depends on the company "environment" on change).
one question: why developing an xml schema format ? why not using a Pom/maven file ? or Ant ?
_________________
Best regards
Rui Madaleno |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jul 25, 2016 4:52 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ruimadaleno wrote: |
| one question: why developing an xml schema format ? why not using a Pom/maven file ? or Ant ? |
I'm not using an XML format so I'll let my worthy associate speak to his solution. I'm using a set of Oracle tables and some apps.
We don't use maven/ant/similar technologies for prod implementations because it's the view of this site that they'd sooner have the bar file that was tested in the lower environments implemented in production with the necessary bar file overrides to change environmental details. I know some sites use those tools to build one bar file per environment from a single source, and I'm not saying that's a bad solution or that any solution which differs from this site's solution is a bad solution. It's just our solution. As my worthy associate says, it's what works for your organization.
What we do have is a tightly controlled, automated promotion process that prevents changes that have not been through the correct governance and approval process.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
