| Author |
Message
|
| brokernewbie |
 Posted: Thu Jan 15, 2015 2:11 pm Post subject: broker.xml update for a given flow |
 |
|
Newbie
Joined: 15 Jan 2015
Posts: 7
|
1) I build BAR B1 using a script with 10 flows. It has 10 .cmf and 1 broker.xml which has information about these 10 flows.
2) I compile for the same BAR again - BAR B2 with a single new flow. So this has 1.cmf and 1 broker.xml which has information about only 1 flow. This is a NEW flow - not the same as the 10 exiting ones.
3) I want to take the newly built flow from the BAR B2 (11th .cmf) to the BAR B1 which I can do successfully using a script.
4) Now I have BAR B1 with 11 .cmf files and 1 broker.xml which has information about only 10 flows.
Problem : How do I update the broker.xml in B1 to have information about the 11.cmf.
In short for those understand the lingo - We are using a CI system + scripting to automate full build (point 1), delta build (point 2) , delta package update (point 3+4). Problem is 3+4 is done with respect to code, but not metadata (broker.xml). Using a hacky solution to do some text processing to merge the 2 xmls is not a preference if it can be helped.
And yes, I tried mqsipackagebar, but it does not update the original broker.xml from B1, just adds 11.cmf to B1 |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Fri Jan 16, 2015 6:17 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Clearly you need to edit the broker.xml.
It is, thankfully, an XML file.
There are no presupplied tools with Broker that can combined bar files.
The usual approach is to build a new bar file that contains all of the necessary resources from source control directly.
You could attempt to use mqsiapplybaroverride to set properties on the new flow, to populate parts of the broker.xml.
But there's no guarantee that will work, and you're already unzipping and rezipping the bar file, so you might as well edit the broker.xml file while you're at it. |
|
| Back to top |
|
|
| brokernewbie |
| Posted: Fri Jan 16, 2015 7:01 am Post subject: |
|
|
Newbie
Joined: 15 Jan 2015
Posts: 7
|
mqjeff thanks for the quick response,but
| Quote: |
The usual approach is to build a new bar file that contains all of the necessary resources from source control directly.
|
Understand, but we have a clear defined requirement to do a "delta/partial build and deploy". Primary reason - if we build a whole BAR, then the time stamps of all flows get changed. In case of code issues after deployment, what they want to be able to do is quickly find only the flows which have changed and focus on fixing only them.
| Quote: |
You could attempt to use mqsiapplybaroverride to set properties on the new flow, to populate parts of the broker.xml.
|
We went that route, but all this command does is "override" or update existing values. So if the original vanilla values don't exist, it will not "add" them to the xml.
| Quote: |
| you're already unzipping and rezipping the bar file |
No, we are not un(zipping), we use jar -uf to update/add .cmf files. But I get the point you are trying to make - 1 hack to 2 hacks - same thing. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Jan 16, 2015 7:06 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| brokernewbie wrote: |
| Quote: |
The usual approach is to build a new bar file that contains all of the necessary resources from source control directly.
|
Understand, but we have a clear defined requirement to do a "delta/partial build and deploy". Primary reason - if we build a whole BAR, then the time stamps of all flows get changed. In case of code issues after deployment, what they want to be able to do is quickly find only the flows which have changed and focus on fixing only them. |
Well, again, the way you would do that is to build a bar file that only contains the new flows.
Also, be aware that when you move to v8 or later and start using applications and libraries, that you can't do updates of individual flows.
For reasonably good reasons.
| brokernewbie wrote: |
| Quote: |
| you're already unzipping and rezipping the bar file |
No, we are not un(zipping), we use jar -uf to update/add .cmf files. But I get the point you are trying to make - 1 hack to 2 hacks - same thing. |
And hack is the right word. You should make sure your change team understands that you're making modifications to deployable resources, after those resources have been compiled and tested, and you're doing those changes outside of supported tools and IBM supported methods. |
|
| Back to top |
|
|
| brokernewbie |
| Posted: Sun Jan 18, 2015 1:38 pm Post subject: |
|
|
Newbie
Joined: 15 Jan 2015
Posts: 7
|
mqjeff
| Quote: |
| Well, again, the way you would do that is to build a bar file that only contains the new flows. |
That is what we are doing. give user an interface to select 1 or more flows they want to build >> mqsicreatebar >> build only those which user selected.
| Quote: |
| Also, be aware that when you move to v8 or later and start using applications and libraries, that you can't do updates of individual flows. |
We are using 8.0.0.1/2 and hence able to use mqsipackagerbar. And I can still update individual flows using above technique and use either a jar-uf or mqsipackagebar to push them into the existing bar. But I know what you are talking about and probably does not apply here.
Overall this is evolving into a discussion on good/bad practices and not really helping me with a trick to solve the problem. Still appreciate your inputs. I will keep this open in case someone else has another idea. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Jan 19, 2015 6:12 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| brokernewbie wrote: |
mqjeff
| Quote: |
| Well, again, the way you would do that is to build a bar file that only contains the new flows. |
That is what we are doing. give user an interface to select 1 or more flows they want to build >> mqsicreatebar >> build only those which user selected. |
Ok. Then why do you need to merge two bar files? If you're building a bar file that contains only the changed resources.. then when you deploy it, only the changed resources will get deployed and only those resources will get their time stamps updated?
I have seen previous requirements to merge bar files because it takes less time to deploy one really large bar file rather than deploy two or three large bar files. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jan 19, 2015 10:08 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY
|
Also when deploying your delta make sure you don't use the option to remove all deployed artifacts before doing the deploy. 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| brokernewbie |
| Posted: Tue Jan 20, 2015 10:30 am Post subject: |
|
|
Newbie
Joined: 15 Jan 2015
Posts: 7
|
| Quote: |
| Ok. Then why do you need to merge two bar files? If you're building a bar file that contains only the changed resources.. then when you deploy it, only the changed resources will get deployed and only those resources will get their time stamps updated? |
Idempotency. I need to be able to have a complete set of BARs at all times so I can
- Deploy the same code to another environment by just overriding some values
- Rebuild an environment to a given state if it crashes. |
|
| Back to top |
|
|
| brokernewbie |
| Posted: Tue Jan 20, 2015 10:32 am Post subject: |
|
|
Newbie
Joined: 15 Jan 2015
Posts: 7
|
| Quote: |
| Also when deploying your delta make sure you don't use the option to remove all deployed artifacts before doing the deploy |
I am presuming this is an option available in the toolkit.
We use Jenkins to call custom written API Java code and automatically deploy a set of BARs to a given environment, so there is no option of (not) doing that. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jan 20, 2015 10:47 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| brokernewbie wrote: |
| I am presuming this is an option available in the toolkit. |
You presume wrong. It's an option on the deploy.
| brokernewbie wrote: |
| We use Jenkins to call custom written API Java code and automatically deploy a set of BARs to a given environment, so there is no option of (not) doing that. |
Yes there is.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| brokernewbie |
| Posted: Tue Jan 20, 2015 12:37 pm Post subject: |
|
|
Newbie
Joined: 15 Jan 2015
Posts: 7
|
| Vitor wrote: |
You presume wrong. It's an option on the deploy.
|
Cool, very rarely it feels nice to be called wrong. So I looked up the documentation:
- mqsideploy with the switch -d deployedObjects (not discussing as I not using this to deploy)
- Java API has this
| Code: |
public DeployResult deploy(InputStream barStream,
String barFileLabel,
boolean isIncremental,
long timeToWaitMs)
isIncremental - If true, the contents of the Bar file will add or modify what has already been deployed on the execution group. If false, the contents of the execution group will be completely cleared before deployment (also known as a "complete" deploy). |
This is the one I think I might be interested in. So if I deploy my partially updated BAR using the isIncremental param do we know the exact behavior of this ?
Q: BAR B1 has 10 flows which are already there on the broker and I update just 1 flow. Will it go and only deploy that 1 flow and not touch the rest 9 ?? |
|
| Back to top |
|
|
| brokernewbie |
| Posted: Tue Jan 20, 2015 12:40 pm Post subject: |
|
|
Newbie
Joined: 15 Jan 2015
Posts: 7
|
I would have tested but it will be some effort to update the Java code and test, so if someone knows how the mqsideploy behaves with the -d switch, it would be nice to know :
Q1: BAR B1 has 10 flows which are already there on the broker and I update just 1 flow in the BAR and not touch the rest 9. Will it go and only deploy that 1 flow and not the rest 9 ??
Q2: What happens if my broker.xml contains information only about the 1 flow .
Q3: What happens if my broker.xml contains information only about the rest 9 flows . |
|
| Back to top |
|
|
| joebuckeye |
| Posted: Tue Jan 20, 2015 1:00 pm Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 364
Location: Columbus, OH
|
You seem to be trying to force an outside methodology onto a broker framework that does not follow that methodology.
At one point we thought about having one bar file per execution group (we have ~15 exec grps) but it was too much hassle to get all the builds to work correctly. Now we just have the builds/bars around certain project/team groupings.
So if one message flow changes, we rebuild that bar and deploy the bar. Don't confuse deployed time with edited time. You can use mqsilist with the '-d 2' argument to see both of them for every deployed object in the exec group. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jan 20, 2015 1:18 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Ok.
If you want to update 1 flow, then you should create a bar file that contains 1 flow.
If you want to update 9 flows, then you should create a bar file that contains 9 flows.
If you want to update 10 flows, and you have a bar file with 1 flow in it and another bar file with 9 flows in it, you should build a new bar file that has 10 flows in it.
That's the fundamental point that you seem to be missing.
If you deploy a flow containing k flows to an EG containing n flows (for all k<n), then only the date stamps on the k flows will be updated.
Separately, as I said, if you have two bar files with LARGE NUBMERS of flows in them, then deploying one bar file with BOTH sets of flows in it can take less time than deploying each bar file separately. But that's LARGE numbers, not 10. Or really really really large individual flows . |
|
| Back to top |
|
|
|
|
