| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| The fastest way to redeploy whole WSMB 7.0.x with 830 flows? |
« View previous topic :: View next topic » |
| Author |
Message
|
| t603 |
 Posted: Thu Oct 18, 2012 5:43 am Post subject: The fastest way to redeploy whole WSMB 7.0.x with 830 flows? |
 |
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
Hello,
can I ask someone, what is the fastest way to redeploy whole WSMB 7.0.x, which contains 830 flows in 31 execution groups and two brokers? Overwrite existing flows? Delete all flows and deploy them as new ones? Any strategy about the number of flows within one BAR? Are there any public documentation, tips, case studies about such migrations?
I am running WSMB 7.0.x on AIX 6.1. It should be done fast, because it looks like, now, that I can not do it incremental (flow by flow while other flows will be accessible by consumers of flows), because the change is driven by switching underlying infrastructure database. We are deploying offline (ksh), not online (Toolkit).
WSMB techniques (like those in the first paragraph) are preferred to HW solutions (like borrow temporary additional CPU and RAM from another LPAR or create copy of environment, deploy, then switch).
Thank You, Stepan |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Thu Oct 18, 2012 5:45 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Oct 18, 2012 5:57 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Who is planning this migration? Sounds like a sense of urgency is overriding prudent planning.
Why can you not use your configuration management system to control the deployment of the bars in your repository?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| t603 |
| Posted: Thu Oct 18, 2012 5:59 am Post subject: |
|
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
|
| Back to top |
|
|
| t603 |
| Posted: Thu Oct 18, 2012 6:16 am Post subject: |
|
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
| lancelotlinc wrote: |
| Who is planning this migration? Sounds like a sense of urgency is overriding prudent planning. |
...You could be right. But we are now in the phase of feasibility study. Migration is caused by switching from DB2 to Oracle. So we are investigating impact of this change to the environment. Migration unfortunately results to the need of change almost all of SQL command in ESQL nodes (caused by things like '' = null in Oracle, incompatible names of columns, using AS keyword in table aliasing etc.). Then the problem with connection pooling, redeploying flows. Certainly, we will pass several migrations on development environment and will see, how it will be sucessfull.
| Quote: |
| Why can you not use your configuration management system to control the deployment of the bars in your repository? |
..I am sorry, I do not understand this question. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Oct 18, 2012 7:15 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| t603 wrote: |
| lancelotlinc wrote: |
| Who is planning this migration? Sounds like a sense of urgency is overriding prudent planning. |
...You could be right. But we are now in the phase of feasibility study. Migration is caused by switching from DB2 to Oracle. So we are investigating impact of this change to the environment. Migration unfortunately results to the need of change almost all of SQL command in ESQL nodes (caused by things like '' = null in Oracle, incompatible names of columns, using AS keyword in table aliasing etc.). Then the problem with connection pooling, redeploying flows. Certainly, we will pass several migrations on development environment and will see, how it will be sucessfull.
| Quote: |
| Why can you not use your configuration management system to control the deployment of the bars in your repository? |
..I am sorry, I do not understand this question. |
A well-groomed engineering practice has common traits.
http://en.wikipedia.org/wiki/Software_engineering
One is a source code control system that you check all your source code into.
http://en.wikipedia.org/wiki/Revision_control
A second is a configuration management system that you check all your deployable artifacts into.
http://en.wikipedia.org/wiki/Configuration_management
A third is a continuous integration system that tests all the artifacts against defined use cases.
http://en.wikipedia.org/wiki/Continuous_integration
When people skip some or all of these practices, things usually don't work right and the organization manages crises' all the time rather than planning for success.
A well-known software engineer once told a manager: "A lack of planning on your part does not constitute an emergency on mine."
| Quote: |
Crises:
1. a. A crucial or decisive point or situation; a turning point.
b. An unstable condition, as in political, social, or economic affairs, involving an impending abrupt or decisive change.
2. A sudden change in the course of a disease or fever, toward either improvement or deterioration.
3. An emotionally stressful event or traumatic change in a person's life.
4. A point in a story or drama when a conflict reaches its highest tension and must be resolved. |
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| t603 |
| Posted: Thu Oct 18, 2012 7:29 am Post subject: |
|
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
| lancelotlinc wrote: |
| When people skip some or all of these practices... |
...So I understood well to Your last question in Your previous post right  I can't to say anything. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Oct 18, 2012 5:11 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Why do you need to redeploy at all?
Current deployment should be sufficient. After all, all you need to do is change the DSNs in the broker profile to point to oracle instead of db2??
Minimum downtime, no redeployment 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| t603 |
| Posted: Thu Oct 18, 2012 11:07 pm Post subject: |
|
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
| fjb_saper wrote: |
| Why do you need to redeploy at all? |
...Because of incompatible syntax and features of DB2 and Oracle. Unfortunately, some of these are big surprise for me. We are interacting database in SQL using the ESQL command PASSTHRU, THE and direct SELECT, INSERT, UPDATE, DELETE. For example:
1) Oracle says, that zero length string is null (and stores even '' as null in table), which DB2, MS SQL Server and other databases distinguish. So all myVariable = '' in SQL commands and moreover following ESQL command has to be manually checked and possibly changed.
2) Incompatible names of columns. Like UID and others. Even if we could enforce Oracle to accept UID as column name, we would have to escape UID by quote character in all our ESQL.
3) Table aliasing using keyword AS. While DB2 allows (and we use it) "from MyTable as MT", Oracle allows just only "from MyTable MT" - without AS keyword. I did not find, if it is possible to switch Oracle to any "compatibility mode" to allow using AS in table aliasing. But because of points 1 and 2, I do not care about this change.
Looks like terrible works - find ALL SQL command in ALL ESQL (good regular expression exercise, but luckily done), walk through ALL SQL command, change where necessary, commit, build BARs, redeploy, test it (the worst task - to catch some tester). Hundreds of changes, redeploy of all 830 flows (some of 560 flows are in two ouf our brokers), because these changes are in infrastructure shared ESQL libraries 
I even can not imagine, how to build all 560 flows, because I have 32bit Toolkit 7.0.0.x and due maximum -Xmx of about 1.8 MB RAM, I am able to have workspace just across small piece of our codebase. And also manually click-click BARS (we have in our SVN repository source codes, not BARs), so all BARs have to be done manually after the change of ESQL code. I tried to use "mqsicreatebar", but it ended unusable - to build one small BAR takes too long, results also in tons of "linking" errors not related to builded flow and even it looks like even it is command line tool, it runs silently Toolkit in background - but I guess, it is my wrong feeling.
And to explain all of these things - because original task was: instal db, transfer data, switch definition in ODBC, done
Uff, I have a strange feeling about this migration. t603[/list] |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Oct 19, 2012 1:43 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
There should be scripts for recompling the flows into bar files, and scripts for deploying.
That is, if there *aren't* scripts, you should *write* them.
And, yes, what you have experienced is the hazard of using PASSTHRU. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Oct 19, 2012 2:50 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| fjb_saper wrote: |
Why do you need to redeploy at all?
Current deployment should be sufficient. After all, all you need to do is change the DSNs in the broker profile to point to oracle instead of db2??
Minimum downtime, no redeployment |
As thesong lyric goes...
It Ain't Necessarily So
There are some differences between the SQL used by Oracle and that of other DB vendors.
We have to maintain TWO sets of SQL commands (dynamic joins esp where you can't use a stored proc). One for ORacle and one for SQLServer. I also have code that works on DB2 but does not work on Oracle.
You do need to retest every flow. Yes you might be lucky and get away with it but... The company concerned here is spending a lot of money on Oracle. It could event be more than the equvalent DB2 license....  It would be prudent to test the bejabbers off the application before switching over.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| t603 |
| Posted: Fri Oct 19, 2012 3:26 am Post subject: |
|
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
| mqjeff wrote: |
| There should be scripts for recompling the flows into bar files, and scripts for deploying. |
...Which command would You recommend me to be the heart of "BARing" script?
We do not have script for creating BARs. We thought about mqsicreatebar, but we were totally unsuccessful, because mqsicreatebar it gave unusable results (terrible slow, more than Toolkit, lot of unrealted errors) to everyone from us. I do not know another way (this and GUI Toolkit) to create bar.
...We have script for deploying, The heart of this script is mqsideploy command, not WSMB Java API (but some tries were done by us already with Java API). It works fine, but sequentialy and 1 BAR = 1 flow or dictionary, so for common it is suitable, but for my purpose its nonparalel base is unusable. I will investigate mqsibackupbroker, which sounds very promising, although I will have to ask for another environment. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Oct 19, 2012 4:03 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
mqsicreatebar is not "slower" than Toolkit. It *runs* Toolkit. So it is only "slower" because you have to wait for Toolkit to start up.
In v8, you can use mqsipackagebar instead. This doesn't run Toolkit, because it doesn't need to compile anything, because v8 lets you compile things at deploy time rather than build time.
Outside of that, there is no way to create BAR files.
If you are having issues with mqsicreatebar failing, it is because you are not properly identifying all of the needed resources for a given build, or the needed resources are not completely error free.
It should be a goal of all development teams that all resources added to version control and marked as "ready" should indeed be free of errors. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Oct 19, 2012 4:10 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| t603 wrote: |
| We do not have script for creating BARs. We thought about mqsicreatebar, but we were totally unsuccessful, because mqsicreatebar it gave unusable results (terrible slow, more than Toolkit, lot of unrealted errors) to everyone from us. |
The mqsicreatebar command is the way to go. There have been any number of posts about the best way to incorporate it into a script. It's true that if you just throw 830 flows at it in a huge heap and say "figure it out" it may well not go that well. I bet you don't have all 830 flows in the Toolkit when it "works"; you need to understand that all mqsicreatebar is doing is creating a headless version of Toolkit and running the same build process that you do through the GUI. So anything you can build through the Toolkit builds through mqsicreatebar.
| t603 wrote: |
| It works fine, but sequentialy and 1 BAR = 1 flow or dictionary, so for common it is suitable, but for my purpose its nonparalel base is unusable. I will investigate mqsibackupbroker, which sounds very promising, although I will have to ask for another environment. |
If you have 1 bar file = 1 flow or dictionary then that's a choice you've made. I accept that maybe 830 deploys is faster that 1 deploy of a bar with 830 flows in it due to the reduction of overhead, but 1 artifact to a deploy is a good model and even for this bulk move you should be able to bulk up a little.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| t603 |
| Posted: Fri Oct 19, 2012 4:23 am Post subject: |
|
|
Voyager
Joined: 16 Oct 2012
Posts: 88
Location: Prague, the Czech Republic, Europe
|
| mqjeff wrote: |
| mqsicreatebar is not "slower" than Toolkit. It *runs* Toolkit. So it is only "slower" because you have to wait for Toolkit to start up. |
...Yes, You are right. But I wanted to have automated build from SVN sources using mqsicreate within operation system batch script. And then silent invoking of "Toolkit" for each call of mqsicreatebar killed this approach. I did not find any possibility to run mqsicreatebar with some configuration ~ build.xml file as inpur parameter. Maybe some automation using Java API within Apache Ant would be a chance, but it is too uncertain, so I did not tried it yet.
I do not waste Your valuable time by slowly moving off-topic (although it is and would/will be beneficial to me). I am thanking You very much for all Your (from all) hints.
Stepan |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2 Next |
Page 1 of 2 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
