| Author |
Message
|
| PeterPotkay |
 Posted: Mon Nov 10, 2014 6:48 pm Post subject: Where am I? |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
What do you guys use to allow the exact same BAR file to be deployed to your different Brokers, and have conditional ESQL code execute based on the environment the flow was deployed to?
For example, if the flow was deployed to the QA Broker, the code would execute some logic specific to the QA environment, but if it was deployed to a Prod Broker, it does Prod specific stuff. 99.99% of the flow would be the same in all environments, but simple stuff like a URL to post to or Reply To Q name to use might be different for that flow between environments.
You can't convince me User Defined Properties that you change on the bar file as it gets deployed to different Brokers is the same unaltered bar file.
This list shows which properties are available to ESQL:
http://www-01.ibm.com/support/knowledgecenter/SSKM8N_8.0.0/com.ibm.etools.mft.doc/ak04897_.htm?lang=en
While we have a strong naming convention for our Broker names, I don't want this logic based on broker names. If we code it for specific names, we will have hundreds of flows to change if we ever want to scale horizontally and add one more Broker. We could rely on the naming pattern (if the 3rd character of the broker name is a "P" its Production), but I don't want to tie myself to that naming convention for eternity.
What do you guys use to answer a message flow if it asks "Where am I?"
I was hoping ESQL could query an environmental variable that is set at build time, as in SET MQSI_ENVIRONMENT=INTEGRATION
I do not want to use JCNs.
I guess we could use a File and stick a File Input Node in each flow - ugh. Would that even work at Broker startup when all the flows fire up at once? Don't wanna go there.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Mon Nov 10, 2014 7:40 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
What's wrong with a UDP?
Using mqsiapplybaroverride will change the value. So you start from the exact same bar file but the deployable is targetted for a specific environment.
The environment name would then be supplied in the bar override file.
Is this too far fetched? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Nov 10, 2014 11:18 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
We use a table with one row for each flow. The flow reads it on startup and then it is cached in a shared variable. Thus any changes mean just a flow restart.
I'm using this at the moment to deploy some flows that have some functionality disabled. When the destination has their equivalent functionality ready to go I can use change the column in the table and restart the flow. If things go wrong then disabling it is easy.
We hold all sorts of params in different table columns. There has been a push to replace it all with a simpler table using key/value pairs but it is not as easy to use as what we have now. The only advantage is expandability but as we have not had to change the table structure in more than 2 years, I'd say that it is working well.
That said, we will probably add some extra columns for the next big project but it will always be backwards compatible.
We don't allow any Barfile overrides or UDP's. The barfile that goes through test is what is deployed. I know this is very 'old school' but we know for sure what has been deployed.
We tried Implementing Overrides once and the override file that was shipped to Japan was for a system in Indonesia. Trying to run the flow with Indonesia queue names was not a pleasant experience. Yes I know thjat this can be overcome but there was a lot of 'egg on face' and in a society like Japan this is very important to avoid if possible.
Broker allows us to do things in many different ways. We should use the way that works for you. no one way is correct.
Just my 2p's worth.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Nov 11, 2014 5:06 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| fjb_saper wrote: |
What's wrong with a UDP?
Using mqsiapplybaroverride will change the value. So you start from the exact same bar file but the deployable is targetted for a specific environment.
The environment name would then be supplied in the bar override file.
Is this too far fetched? |
Its a change. You are not deploying the same exact bar file to PROD as your are in QA. I understand the concept, I appreciate the flexibility UDPs offer, I get that its WAY better then cracking open the source code to change a parm or two and then re-bar it. But its involves a human getting involved to set all those UDPs each and every time the message flow is deployed for its lifetime, multiplied by hundreds of bar files.
I want to get to the point where the only decision is which bar file do I pull from the Source Repository, what time does the deploy fire and to which EG on which Broker - no other changes, no other thinking required by the deployers.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Nov 11, 2014 5:20 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| smdavies99 wrote: |
We use a table with one row for each flow. The flow reads it on startup and then it is cached in a shared variable. Thus any changes mean just a flow restart.
|
Where is the table? In a DB on a non Broker server, and you rely on DSNs to get your flow on the QA server to refer to the QA database? On the PROD server to refer to the PROD database?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Nov 11, 2014 6:15 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Nov 11, 2014 6:59 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
That's moved the changes at deploy time from the bar file to the broker. It still involves someone being involved and doing something different between QA and PROD.
The only thing I want to be done differently between the QA deploy and the PROD deploy is:
Date/Time of deploy
Broker being deployed to
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Nov 11, 2014 7:08 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
mqsiapplybaroverrides doesn't have to involve someone either.
I understand your point about 'the same', and I'm sure you're trying to meet an audit requirement.
But basically, what you actually need to do is use a properties system of *some kind*. This can be UDPs, this can be a database table, this can be a flat file that sits on the broker, this can be global cache, this can be a web service that gets called, this can be nearly anything.
But UDPs are about the only thing that don't require custom coding in each and every flow.
Even aside from all of that, there are still a large number of things that don't require any change. DSNs, queue names, anything that is based on a configurable service, etc. These things all get resolved by the deployed environment to the correct location for that environment. |
|
| Back to top |
|
|
| nelson |
| Posted: Tue Nov 11, 2014 3:12 pm Post subject: |
|
|
Partisan
Joined: 02 Oct 2012
Posts: 313
|
| smdavies99 wrote: |
| We use a table with one row for each flow. The flow reads it on startup and then it is cached in a shared variable. Thus any changes mean just a flow restart. |
Very interesting discussion. How do I say a message flow to do something on startup? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Nov 11, 2014 8:57 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| PeterPotkay wrote: |
That's moved the changes at deploy time from the bar file to the broker. It still involves someone being involved and doing something different between QA and PROD.
The only thing I want to be done differently between the QA deploy and the PROD deploy is:
Date/Time of deploy
Broker being deployed to |
No it doesn't. You have the override properties for each environment stored in source control. Your deployment process is scripted and the apply bar override is part of the deployment process.
The only involvement is the setup of the properties in source control. It is flexible because when the developer changes the flow he has to adjust the override properties to match the new flow...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Nov 11, 2014 11:20 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| nelson wrote: |
Very interesting discussion. How do I say a message flow to do something on startup? |
A Timer node will fire when the flow is started. Then with a bit of logic you can sunc the read. We also re-read the values every 24 hours.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Nov 12, 2014 6:39 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| nelson wrote: |
| smdavies99 wrote: |
| We use a table with one row for each flow. The flow reads it on startup and then it is cached in a shared variable. Thus any changes mean just a flow restart. |
Very interesting discussion. How do I say a message flow to do something on startup? |
ESQL shared variables are not persistent across restarts. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Nov 12, 2014 6:45 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Hmm, ESQL can be used to query the Execution Group Label, which is the friendly name of the Execution Group.
http://www-01.ibm.com/support/knowledgecenter/SSKM8N_8.0.0/com.ibm.etools.mft.doc/ak04897_.htm?lang=en
If I had a suffix appending to the end of the execution group, the ESQL could identify what EG its running in and thus automagically understand what environment it was in.
MY_EG_INT
MY_EG_QA
MY_EG_PROD
MY_EG_PERF
MY_EG_DEV
This would allow multiple logical environments to exist in parallel on the same Broker, as in create 2 execution groups on the same broker called MY_EG_INT and MY_EG_DEV.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Wed Nov 12, 2014 9:01 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| mqjeff wrote: |
ESQL shared variables are not persistent across restarts. |
Quite correct.
That is why we use the Timer Node. This fires off a message when the flow is started/restarted. The ESQL (in an Atomic block) checks for the existence of one of the colums in the Row variable. If it is blank then it reads the data for that flow and only that flow (with no locking naturally)
The Atomic block makes sure that flows with multiple instances work properly.
This technique was first developed for Broker 6.0.0.0 circa 2006/7.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Nov 12, 2014 11:50 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
| If I had a suffix appending to the end of the execution group, the ESQL could identify what EG its running in and thus automagically understand what environment it was in. |
And if I had wheels, I'd be a wagon.
Accepting that as has been previously commented you're trying to meet an audit requirement, it's unrealistic to consider a UDP a "change' to the deployed code. I'm also worried conceptually about this kind of "if I'm here I must do this" being in code rather than a UDP.
Consider this: you have logic that checks for transactions over the last 30 days in Prod but (to help with test data) only 10 days in QA and 1 day in test. By whatever mechanism, you determine where you are and apply the correct limit. The business then says 90 days not 30 days - you need a code change! If the number of days was a UDP, it's just a redeploy.
Also if a UDP changes the code, where do you stand on promotable properties?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
