| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Creative Discussion: WMB ESB Team Structure |
« View previous topic :: View next topic » |
| Author |
Message
|
| lancelotlinc |
 Posted: Thu Dec 09, 2010 6:02 am Post subject: Creative Discussion: WMB ESB Team Structure |
 |
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
I would like to invite a discussion on how to organize a WMB development team. From recent posts, it seems the ideal is not attainable to many people who read this board.
To achieve a closer reality to an ideal, could it be possible we discuss what the obstacles are and ways to address them?
For example, some say our company cannot afford to hire more people. To this I say, if properly approached, what manager would say no to 1+1=3 combination?
If a development team has five people on it, and five people seem to be getting no where fast because the development and qa environments are all whacked up with multiple execution groups running the same flows, producing unexpected results --- that to me is a huge waste of energy, resources, and money.
The return-on-investment for a Build Engineer would be the fact that five people are ten times more productive because their efforts are not stepping all over each other.
Therefore, its less an issue that a company cannot afford something and more of an issue that the Build Engineer is not properly demonstrated as valuable addition to the team. In fact, rather than new hire a Build Engineer, designate an existing developer on the team of five as the "Build Engineer". Not a hard thing to do, and costs no more money.
http://java.sun.com/developer/technicalArticles/Interviews/cattell_qa.html
The Things I Wish I Learned in Engineering School: A Conversation with Sun Microsystems Distinguished Engineer Rick Cattell
"if you want to see your ideas used in the real world, you have to think about more than just the engineering." You need to think outside the box, be a people person, have soft skills that make a difference.
"I also didn't realize that good technology is only 10% of success. If your management doesn't know how to manage a successful engineering project, or your marketing department doesn't know how to access the customers, or doesn't tell you what the customer wants, or if your lawyers don't handle your intellectual property correctly, or if the chief architect doesn't have the ability to create a consistent and simple architecture, then your work can be for naught, and you can spend years building things that never see the light of day."
This means embarking on a campaign to change the status quo. Tactfully pushing your ideas to the top and keeping the discussion there for all to critique and discuss.
Please comment.
Lance
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Dec 09, 2010 6:29 am Post subject: Re: Creative Discussion: WMB ESB Team Structure |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| This means embarking on a campaign to change the status quo. Tactfully pushing your ideas to the top and keeping the discussion there for all to critique and discuss. |
There's nothing wrong with any of the above. Apart from a lack of realization that the status quo has become the status quo for a reason. 
Allow me to play Devil's Advocate here for a moment. This site has a team of 5 developers getting nowhere fast, with the environments in confusion (and there being no environments between Dev & QA where "stable" development can be user or business tested for function rather than quality).
There's going to be considerable resistance to dropping one of the developers out to become a "Build Engineer"; taking the team from 5 to 4 will make MSoft Project move the timeline out. Saying that this will improve efficientcy and increase the development speed by making the team work smarter not harder is likely to elicit the response, "that's the kind of sound bite we use on you. Don't expect us to fall for it".
Increasing effort by removing people is counter-intuitive, at least to management. Depending on site, it can also be consider interfering with management decision making (resourcing being a management playpen).
It's this that makes the ideal unattainable. Not the facts.
Further discussion welcomed of course.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Dec 09, 2010 6:37 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
@Vitor
Ok, so we recognize the problem. And I think you and I are in agreement that use of tools like Hudson, Maven, Ant are all good on balance.
And it seems this discussion is not unlike the global SOA issue: change the hearts and minds of management first, then SOA will make sense to them.
So how do we acheive our goal of (slowly) moving towards a more mature build environment. What do you see as the steps to take?
BAR files can be built via the command line. Thats an easy thing. In your environment are you using the GUI toolkit to build bar files or the command line?
Could we inch towards this more mature build environment without impacting timelines?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Dec 09, 2010 7:06 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
BuildEngineer is a hat, not necessarily a full time job.
With a team of 5 people responsible for the development side of running a Broker environment, it would seem sensible to have two of these people put on the build engineer hat for deployments to production.
Particularly with things like SOXLEY and PCI compliance requiring a significant effort at separation of duties.
But I've rarely seen Broker shops with as many as five people full time on broker development. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Dec 09, 2010 7:27 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| BAR files can be built via the command line. Thats an easy thing. In your environment are you using the GUI toolkit to build bar files or the command line? |
Always with GUI toolkit aside from one noteable site where we got Hudson going. The number of posts on this forum on mqsicreatebar & ant (and indeed proper use of source control) attest to the non-straightforward nature of this endevour.
I've had some success with automating mqsiapplybaroverrides to deploy the files correctly modified for the environment they're going into.
Accepting that Build Engineer can be a hat, and more than 5 fully engaged broker developers is plausible, the fact remains that you're disengaging a full time development resource, even if only to make him part time as they develop when they're not wearing the hat.
(And mqjeff is correct when he says many shops don't run to as many as 5 people)
It is a hearts and minds issue. The hearts & minds of management tend to be focused on delivery and consequent fire fighting. It's a hard sell this better world. On the current site I can't even get backing for the automated deployment model I mention above.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Dec 09, 2010 7:32 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Again, you may find PCI compliance or Sarbanes-Oxley legislation to be a powerful motivating force for mgmt to create a build system for production deployments. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Dec 09, 2010 7:42 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
I had to smile at the '5' developers number.
You should really caveat that by outlining the role you expect said developers to perform.
In our agile scrum based setup, we have 6 devs but need at least 8. Why?
Because the ESB(aka Broker) is the piggy in the middle. The Broker Devs have to do a lot more besides cutting code.
They have to ensure that the interfaces (WSDL's BAPI, XSD's etc) are all properly defined with the external parties.
Oh, and then there is the Oracle Stored Proce, packages and stuff that the ESB devs are expected to do. (because you know SQL don't you and the PHB's have just laid off 50% of the Oracle Code hackers, or R3, CRM etc)
Then the ESB devs have to work with the Testers to define the TDD setup. Sometimes they even get involved with the data modelling.
Then there are the peer reviews for other projects to be done.
Finally, you get to shove something over the wall into the release process. Don't for get all the release notes perfected so that the release team don't screw up and deploy the same WebService into 3 different Execution Groups.
Then, take a depp breath and roll your sleeves up an fix all the 'P1' bugs that should have been done last week while you were trying to put a release to bed.
So what is expected of your broker devs? This will to some extent dictate the structure and the skills needed.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Dec 09, 2010 8:14 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
In my experience...
Broker development resources should write cradle-to-grave all necessary source code, message flows, command line scripts, runmqsc sripts, Junit tests, Ant make files, mqsicreatebar & mqsiapplybaroverride scripts, and Maven dependency scripts.
Build Engineer takes these artifacts, and creates the Hudson operations to pull the source code, pull the dependencies, execute the mqsicratebar, execute the bar override for the target environment, and push the resulting bar to the correct execution group on the correct system.
We typically have five environments, four of which are sanitized from human hands. Sandbox, Dev, QA, Perf, and Prod. Prod has multiple geographically dispersed identical environments, which all attach to the same cluster, managed by cluster channel priority. Xi50 appliances perform DMZ and preliminary structure work. The DMZ appliances then route to the main ESB on AIX.
Developers can make whatever changes they want to the Sandbox. After that, the only way any of the other environments change is through Hudson. Humans are prohibited from touching any of the environments and this avoid the issue of putting to bed a new deployment and cuts down on P1 issues since all changes are scripted and tested before hand.
If a deployment is needed, a ticket is created in the help desk system to generate the work order to the Build Engineer. This is true for Dev, QA, Perf, and Prod. Build Engineer cannot change any environment without a work order. Developers don't even have log ons to any environment besides Sandbox.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Dec 09, 2010 8:19 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| In my experience... |
As I've said before, you've had an interesting and fortunate experience.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bsiggers |
| Posted: Thu Dec 09, 2010 11:02 am Post subject: industrialized solutions |
|
|
Acolyte
Joined: 09 Dec 2010
Posts: 53
Location: Vancouver, BC
|
Hello, just to add by two cents worth to this interesting discussion, coming from a large consulting/outsourcing company that has had to deal extensively with SOX/PCI issues.
What we generally had was a "three-ring circus" for *all* project delivery and support, not just ESB.
- Developers (do design and build work)
- Change Management (do all deployments to TEST, PERF, PROD)
- Production Support (Support and Administration in all environments)
1. Developers make the code in DEV, where they have quite extensive access levels. They have essentially only read-only access in TEST and PERF. When code needs to be migrated to another environment, the developers package their change together and get in contact with Change Management team.
2. Change management team is responsible for migrating code (and surrounding artifacts like DB objects, etc.) to all other environments (TEST, PERF, PROD). They're not experts in the code itself or what the change actually is, so the developers have to document *exactly* what they're changing.
3. Production support provides administration and supports the infrastructure (ie. Broker, MQ, etc.). They have admin-level rights in all environments, but do not develop code, and do not do migrations.
As you can probably imagine, there is a healthy level of conflict between all the above groups, especially at the architecture / best practices level, and requires good communication and relationships between all teams. But you've got clear seperation of duties, which makes getting through SOX/PCI much simpler. You've also got to have a certain scale to make this system work, and there's no question that things can be done more efficiently. But it works pretty well, and the same system can be used across *all* technologies (ETLs, Broker, J2EE, PLSQL, etc.) |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Dec 09, 2010 11:07 am Post subject: Re: industrialized solutions |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| bsiggers wrote: |
| What we generally had was a "three-ring circus" for *all* project delivery and support, not just ESB. |
This is a key point.
Where I have seen things like Build Engineers and separation of duties in general, it has been in places that have this for everything and not something that was put in place for Broker or the ESB.
There's not a lot to do to help an existing Build Engineer who knows what they're doing with their tools get up to speed with applying these to Broker. |
|
| Back to top |
|
|
| harish_td |
| Posted: Thu Dec 09, 2010 7:50 pm Post subject: |
|
|
Master
Joined: 13 Feb 2006
Posts: 236
|
| smdavies99 wrote: |
I had to smile at the '5' developers number.
You should really caveat that by outlining the role you expect said developers to perform.
In our agile scrum based setup, we have 6 devs but need at least 8. Why?
Because the ESB(aka Broker) is the piggy in the middle. The Broker Devs have to do a lot more besides cutting code.
They have to ensure that the interfaces (WSDL's BAPI, XSD's etc) are all properly defined with the external parties.
Oh, and then there is the Oracle Stored Proce, packages and stuff that the ESB devs are expected to do. (because you know SQL don't you and the PHB's have just laid off 50% of the Oracle Code hackers, or R3, CRM etc)
Then the ESB devs have to work with the Testers to define the TDD setup. Sometimes they even get involved with the data modelling.
Then there are the peer reviews for other projects to be done.
Finally, you get to shove something over the wall into the release process. Don't for get all the release notes perfected so that the release team don't screw up and deploy the same WebService into 3 different Execution Groups.
Then, take a depp breath and roll your sleeves up an fix all the 'P1' bugs that should have been done last week while you were trying to put a release to bed.
So what is expected of your broker devs? This will to some extent dictate the structure and the skills needed. |
The story of our lives |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Dec 10, 2010 5:17 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| harish_td wrote: |
| The story of our lives |
This was the point of my inquiry. If our lives are so impacted by circumstances, what shall we do to influence the betterment of our lives going forward?
We can sit and do nothing. Then nothing will change.
We can advocate for a better process.
If you read my earlier post on the other thread, Country Club members who pay large sums of money in annual Country Club dues do not need the US Treasury to gift them a free golf cart at the expense of the tax payers and pensioners.
The similar conclusion can be made for corporations who spent the money to purchase the hardware, software, and on-going maintenence for the ESB. The funds to do the needful are present. I don't "buy" the line that we have no money. (Pardon the pun.)
I see the central issue a lack of motivation on the people that have the power. That be you and me. We have the power to make things better. Why are we not motivated to do so?
The technology to accomplish a better process is available to us, most of these tools are public domain. Why are we stopping ourselves from making steady progress going forward?
As was once said of Steve Austin: "We have the technology, we can rebuild him."
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Dec 10, 2010 5:40 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| lancelotlinc wrote: |
| Why are we stopping ourselves from making steady progress going forward? |
To touch back on my previous point, we are not stopping ourselves doing anything. The majority of us work for someone. And while the tools are public domain, the resources to implement them cost money. Even if it's only management play money sliding round on bits of paper between various budgets.
To touch back on your point:
| lancelotlinc wrote: |
| what shall we do to influence the betterment of our lives going forward? |
I'm open to ideas. Advocating a better process isn't working for me personally.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Dec 10, 2010 5:48 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| Vitor wrote: |
| I'm open to ideas. |
Here is an idea. Or rather a CHALLENGE.
The first step making a better process is easy.
Take a bar file you are working on currently and construct a command line batch file to build it rather than use the GUI toolkit to build it.
One bar file, one command line.
Who will accept my challenge?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2, 3 Next |
Page 1 of 3 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
