| Author |
Message
|
| t603 |
 Posted: Tue Nov 12, 2013 7:21 am Post subject: Parallel deployment of BARs to different EGs on one BK? |
 |
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
Good afternoon,
is it possible to run parallel deployment (mqsideploy, CMP API) of different BARs, MSETs... to different EGs on one BK in IBM WebSphere Message Broker 7.0.0.6 or IBM Integration Bus 9.0.0.0 on IBM AIX?
I need to speed up deployment of average 200+ bars (1 bar = 1 message flow) and then message sets and then ... to our 5 brokers, each broker with about 40 execution groups. It looks like within one broker BARs are deployed one by one. Am I right? Or are bars deployed parallel by default and only my observation of deployment proces through log and Toolkit is wrong?
Since I guess, that bars (message flows) are in different execution groups on one broker independent, I wish to deploy bars in parallel. Is it possible? Multiple admin channels or something else?
I searched documentation:
1) Deploying resources on "http://publib.boulder.ibm.com/infocenter/wmbhelp/v7r0m0/topic/com.ibm.etools.mft.doc/af35100_.htm"
2) Deploying a broker archive file on "http://publib.boulder.ibm.com/infocenter/wmbhelp/v7r0m0/topic/com.ibm.etools.mft.doc/af03890_.htm"
3) CMP API on "http://publib.boulder.ibm.com/infocenter/wmbhelp/v7r0m0/topic/com.ibm.etools.mft.cmp.doc/com/ibm/broker/config/proxy/ExecutionGroupProxy.html#deploy(java.lang.String,%20boolean,%20long)"
and MQSeries.net, but no luck.
Thank You in advance for answer. Stepan |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Nov 12, 2013 7:25 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
AFAIK each deployment is a request message on the broker's command queue. So no matter how many admin channels you use, that's just going to put stuff on the queue faster not change how fast the command processor goes through them.
Unless a friendly passing IBMer confirms or denies that the command processor maintains the affinity of messages you need a PMR
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Tibor |
| Posted: Tue Nov 12, 2013 7:29 am Post subject: |
|
|
Grand Master
Joined: 20 May 2001
Posts: 1033
Location: Hungary
|
| In my observation, deployment package is a message in the SYSTEM.BROKER... queue, and all the EGs are listening on it. So parallel processing should be an option, but you have to know, deployment is a performance-hungry task, so I am not sure you can gain a lot... |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Nov 12, 2013 7:31 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| It can significantly improve deployment time if you put more than one flow in a BAR file. |
|
| Back to top |
|
|
| t603 |
| Posted: Tue Nov 12, 2013 7:39 am Post subject: |
|
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
| Quote: |
| ...deployment package is a message in the SYSTEM.BROKER... queue, and all the EGs are listening on it... |
...Yes, OK, I suppose something like that. So my question is, if it is possible to multiply such queue ~ allocate queues with "deployment" capability to each EG or several EGs. To say to each EG the name of its deployment queue. Because to have one queue for all EGs is too little, I guess.
| Quote: |
| It can significantly improve deployment time if you put more than one flow in a BAR file. |
...It was some decision of our development team to have 1 bar = 1 message flow. I suppose due maintenance of changes of shared message flows and message set. But I do not it know exactly.
Last edited by t603 on Tue Nov 12, 2013 7:46 am; edited 1 time in total |
|
| Back to top |
|
|
| Tibor |
| Posted: Tue Nov 12, 2013 7:42 am Post subject: |
|
|
Grand Master
Joined: 20 May 2001
Posts: 1033
Location: Hungary
|
| hi mqjeff, if the deployment policy is strict there out, there is no chance for creating a huge BAR file, which contains everything. |
|
| Back to top |
|
|
| Tibor |
| Posted: Tue Nov 12, 2013 7:47 am Post subject: |
|
|
Grand Master
Joined: 20 May 2001
Posts: 1033
Location: Hungary
|
| t603 wrote: |
| Because to have one queue for all EGs is too little, I guess. |
Don't think so, you can easily increase your queue capacity with MaxMsgLen and MaxDepth parameters. |
|
| Back to top |
|
|
| t603 |
| Posted: Tue Nov 12, 2013 7:53 am Post subject: |
|
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
| Tibor wrote: |
| t603 wrote: |
| Because to have one queue for all EGs is too little, I guess. |
Don't think so, you can easily increase your queue capacity with MaxMsgLen and MaxDepth parameters. |
...I do not solve number of BARs already sent to be deployed or maximum size of one or all BARs, but the speed of deployment.
Last week before release we deploy all bars, msets etc. to our deployment environment (then backup broker and restore it to production in DAY D evening), but we need to deploy all on deployment environment several times, since changes after deadlines are unfortunately usual. And this is time consuming now. |
|
| Back to top |
|
|
| Tibor |
| Posted: Tue Nov 12, 2013 8:03 am Post subject: |
|
|
Grand Master
Joined: 20 May 2001
Posts: 1033
Location: Hungary
|
| Have you already tried to launch mqsideploy commands parallelly? Of course, you should increase timeout settings before, but I think, it might work. |
|
| Back to top |
|
|
| bielesibub |
| Posted: Tue Nov 12, 2013 9:26 am Post subject: |
|
|
Apprentice
Joined: 02 Jul 2008
Posts: 40
Location: Hampshire, UK
|
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Nov 12, 2013 10:46 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
At the risk of being called a  and having been in the software dev business for 40 years I prefer small deployments to one bar file with many flows inside.
I'm old school in that the Bar file is the lowest common deployable object. Restricting one file contains exactly one object. Us old fogies like to test a bar file through all the different domains (test/uat/pre-prod) without changes. Naturally this precludes barfile-overrides but IMHO once built the bar file must not be edited. Each testable object is tested and deployed.
Also, I check into Source control the bar file (versioned naturally). This way we can rebuild any level of system and to any defined set of patches/revisions for each object.
I know that many will disagree with me on some of these practices but I have posted them here just to show why I prefer small deployments of individual objects. We really don't need to go down a rat-hole discussing my methods. They have been discussed before.
  etc etc
Besides, so what if it takes 10 mins to deploy everything to a broker if you do it in bulk as oppsed to 15 mins doing it individually? How often are you going do do this? IMHO not very often.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| t603 |
| Posted: Tue Nov 12, 2013 1:21 pm Post subject: |
|
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
Above mentioned article from mqmatt (?) contains following statements:
| Quote: |
| This procedure lets the user continue working in the Toolkit while the broker is finishing any existing work items and applying the new configuration, and even lets you send multiple deployment requests simultaneously, although each broker can only have one deployment in progress at any one time. |
and
| Quote: |
| Each broker can only have one deployment in progress at any one time, and so when firing off multiple deployments to the same broker, second and subsequent deployments to the same broker will fail unless they are synchronised. |
Unfortunately I understand above statements as "There is no way, even using Java CMP API, to force broker to deploy more bars in one time." Oh, oh, bad news. I do not understand such limitation, but I can not to do with this anything.
| smdavies99 wrote: |
| Besides, so what if it takes 10 mins to deploy everything to a broker if you do it in bulk as oppsed to 15 mins doing it individually? How often are you going do do this? |
Well, believe me, 10 or 15 minutes I would not definitely solve here on MQseries.net or even in my mind  I am solving this problem, because it takes several hours. Tomorow, when I come to my work, I will post more precise timing - I did not know it, because I do not perform it personally but my colleagues and I want to give them better chance to catch and repair last final errors before release day. So I would have a chance to re-run deployment within pre-release week several times. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Nov 12, 2013 1:31 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
I've only seen deployments taking several hours once before and that was for V2.0.1 on AIX more than 10 years ago.
I did a test in a VM earlier. This is on a laptop running Windows and the VM is Windows Server 2008/Broker 7.0.0.4
20 deployments (14 flows and 6 message sets) took 1minute 8 seconds.
Repeated 6 times with a newly started broker and freshly created Execution Groups the test averaged 1min 12 seconds.
Can you tell what is actually taking all the time? How many flows etc?
Being realistic though, how many times are you going to do this once the broker is fully configured? So in the long run, does it really matter? Obviously it would if you were going to tear down your broker every few days...
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 12, 2013 1:48 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| t603 wrote: |
| Well, believe me, 10 or 15 minutes I would not definitely solve here on MQseries.net or even in my mind I am solving this problem, because it takes several hours. |
If an individual deployment (i.e. a single bar file containing one or two deployable objects) is taking more than 5 or so minutes then there's a serious issue with the broker you should investigate. 2 or 3 is nearer the mark.
Clearly if (as in your case) you have 200+ bar files each going to 40+ execution groups then that still gives you a significant time overhead to work through all of them, which I suspect is where your question stems from.
| t603 wrote: |
| I do not understand such limitation |
You need to turn this on it's head; why does this limitation exist? If there was a serious groundswell of user opinion towards doing multiple deployments then IBM would have removed this limitation and engineered such a facility into the product. So why is this a problem for you and not for anyone else? A couple of possible points suggest themselves:
- your deploys really are running much too slowly and you should investigate. If you have 40 EGs I would verify you have sufficient if not plentiful resources on the server
- there's a problem with your procedure. I'm a charter member of the 1 flow = 1 bar school of thought, but with this number of deployments that gives you a huge packaging overhead. As my most worthy associate points out earlier, you'll see significant improvements repackaging into a smaller number of bar files. This in no way prevents you reverting to single bars later. Note that if all you're doing is backing up the development broker and restoring it to production as you describe, you're negating almost of the benefits of single bar deployment anyway.
- there's another problem with your proceduure. It's crackers to deploy every single flow to development, back that up, restore it and call that a production deploy. Simply deploy what's changed to production. It doesn't take long and is I suspect the key reason the rest of us have not pressured IBM to allow multiple deployments. All you need are proper deployment controls & there are any number of proven methods available.
If you deploy by backup & restore because that's "faster", I put it to you this thread proves conclusively it's not!
Outside of that, I think you're probably doomed to be sitting drinking a lot of coffee watching your deployment process run. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| t603 |
| Posted: Mon Nov 18, 2013 2:55 am Post subject: |
|
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
Well, You are right, if I understand Your messages right, I should investigate, why deployment of my BARs takes so tool long rather to asking for parallel deployment to one broker.
So I passed several tests to try investigate it. Below described tests I passed several times since Thursday to today (Monday) with similar results.
I created the most simple flow I can imagine: MQ Input (message of type BLOB) -> MQ Output. Deployments with trace ON and OFF takes both 02:30 minutes:
mqsideploy BK1 -e debug_TZI_SRY -a testA.bar -w 600 -v testA.log.txt
EG contains only one another message flow. HW was not utilised to its limits, afaik. I would like to share excerpt from log of deployment, but it is still too long for post and attachments are not enabled here, afaik. Based on this log, parsing and sending of BAR is very fast within seconds, but then about 95 % of total time of deployment consists from repeated sequence of two lines of "AdministeredObjectPool":
| Code: |
2013-11-18 10:54:24.0972 CMPMQReceiver.. d[3]: Polling response queue...
2013-11-18 10:54:25.0717 main........... d[2]: AdministeredObjectPool deregistered a thread that is no longer waiting for a 'actionresponse': main
2013-11-18 10:54:25.0718 main........... d[2]: AdministeredObjectPool registered a waiting for 'actionresponse' thread: main |
I suppose, that this is waiting from Broker for its answer. Does anybody know, how to debug Broker's part, which is behind these two "AdministeredObjectPool" lines?
Stepan |
|
| Back to top |
|
|
|
|
