| Author |
Message
|
| hopsala |
 Posted: Wed May 18, 2011 4:19 am Post subject: WMB 7 in production |
 |
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
Pretty much every site I know are on the move towards V7, from either 6.1 or, tragically, 6.1; but no one has gotten there yet. Anyone out there already using WMB V7 in production?
Would love to hear user experiences concerning:
- Reliability
- Performance
- Clustering
And if anyone is using monitoring events (what IBM calls 'business monitoring' although I think is perfect for operational monitoring) that's a definite bonus.
Cheers! |
|
| Back to top |
|
 |
| lancelotlinc |
| Posted: Wed May 18, 2011 4:31 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
As you indicated, "business monitoring" is not "system monitoring". You need Tivoli or BMC Patrol, or some other system-level monitor for your production system tickets. Business monitoring is useful for business analysts to see what types of transactions are flowing. For example, a bank had 57% deposit transactions vs. 43% withdrawal transactions.
Many customers have V7 in production. It is stable, reliable and trustworthy. After FP1, the memory leaks came under control also. The best platform to run WMB V7 on is AIX Power7 for a number of reasons. AIX Power7 can process identical transaction for three times fewer system resources and twice as fast than other heavy iron. It is possible to acheive 500,000 TPS on AIX Power7.
For an inexpensive WMB platform, RHEL wins an award. It is possible to achieve 10,000 TPS on RHEL 5.5/WMB 7.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| joebuckeye |
| Posted: Wed May 18, 2011 4:45 am Post subject: |
|
|
Partisan
Joined: 24 Aug 2007
Posts: 365
Location: Columbus, OH
|
We have been running most of our broker flows on v7 since January and the ones using the WTX node since March (had to wait for a patch for WTX nodes).
We migrated from v6.0 to v7.0 starting in August last year and were finished by December (held off moving into production due to the end of the year holidays).
We only have 150 or so flows and we only did what was needed to get the v6 flows working under v7. Adding monitoring events and any new functionality that was rolled out in v7 was not part of our migration process.
Now IBM will tell you that v6 flows will work under v7 but we found that most plugin nodes either did not work under v7 or changed how they were configured under v7.
The biggest changes we had to make were in the flows using the old SendMail node and the old WTX node.
Both of those nodes were brought "into the family" in v7 and their configuration changed between v6 and v7.
But if the flows didn't use WTX or SendMail the migration was very smooth. |
|
| Back to top |
|
|
| hopsala |
| Posted: Sun May 22, 2011 4:48 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| lancelotlinc wrote: |
| As you indicated, "business monitoring" is not "system monitoring". You need Tivoli or BMC Patrol, or some other system-level monitor for your production system tickets. Business monitoring is useful for business analysts to see what types of transactions are flowing. For example, a bank had 57% deposit transactions vs. 43% withdrawal transactions. |
I respectfuly disagree. Although IBM defined these as 'business' events, the fact is that they are perfectly usable as monitoring events. Transaction time, payload, failure events, are all both operational and business events. I see no reason not to use these for the normal purposes of logging, monitoring etc, by writing my own subscribing app to collect these events. An operational event is an operational event by any other name..
| joebuckeye wrote: |
| Now IBM will tell you that v6 flows will work under v7 but we found that most plugin nodes either did not work under v7 or changed how they were configured under v7. |
We've actually had a few rewriting to do while migrating the code to V7. A lot of ESQL and Java code was deprecated. It wasn't too much trouble, but still required manual migration.
Did you guys work with a full clustering solution in production? |
|
| Back to top |
|
|
| hopsala |
| Posted: Wed May 25, 2011 1:23 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
bump.
Any further comments on v7 in production or on what's written above are very welcome. |
|
| Back to top |
|
|
| zpat |
| Posted: Wed May 25, 2011 2:23 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| How did you detect the deprecated ESQL code? When we rebuilt and redeployed our WMB 6.0 bar files under WMB 7 - there were no apparent problems. |
|
| Back to top |
|
|
| hopsala |
| Posted: Thu May 26, 2011 3:15 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| Well, we didn't have to detect it, the code just didn't compile. I was only partially involved, but I remember something about ESQL no longer being able to accept functions as parameters in some cases, and a subtle different in REFERENCE variable behavior.. |
|
| Back to top |
|
|
| bsiggers |
| Posted: Thu May 26, 2011 10:16 am Post subject: |
|
|
Acolyte
Joined: 09 Dec 2010
Posts: 53
Location: Vancouver, BC
|
Hello -
Am using both WMB7 in production, and using some event monitoring events (for logging purposes, since the code itself did not contain any logging). Works great, in all respects, but make sure you've got FixPack 2 in there, otherwise we had some weirdness with the FileInput nodes, etc.
Am not currently using a clustered Broker solution - however we are using clustering at the MQ level. Production platform is a virtualized Windows Server 2008. We do have a fairly significant amount of monitoring setup (EGs, Flows, etc.) that was implemented using PowerShell scripts feeding information back into a centralized Nagios monitoring system.
Hope this helps - if you're interested in specifics let me know. |
|
| Back to top |
|
|
| hopsala |
| Posted: Sun May 29, 2011 2:11 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
Thanks bsiggers for your reply,
I'd love to hear more specifics about both the monitoring events and your monitoring scripts. Which events did you switch on? How do you subscribe to them? Do you have a monitoring profile you share across flows, or do you configure events in design time?
About the monitoring scripts, how do you know that a flow is no longer working - whether a web-service flow or an mq one? Did you try to subscribe to the broker's operational publications? |
|
| Back to top |
|
|
| zpat |
| Posted: Sun May 29, 2011 10:51 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| hopsala wrote: |
| Well, we didn't have to detect it, the code just didn't compile. I was only partially involved, but I remember something about ESQL no longer being able to accept functions as parameters in some cases, and a subtle different in REFERENCE variable behavior.. |
Subtle differences worry me. That could make it into production and then cause problems. Do IBM provide any documentation on this? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon May 30, 2011 3:20 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| zpat wrote: |
| hopsala wrote: |
| Well, we didn't have to detect it, the code just didn't compile. I was only partially involved, but I remember something about ESQL no longer being able to accept functions as parameters in some cases, and a subtle different in REFERENCE variable behavior.. |
Subtle differences worry me. That could make it into production and then cause problems. Do IBM provide any documentation on this? |
Not to my knowledge. We're moving from V6.0.0.10 to V7.0.0.2 and seeing different behavior on the MRM parser in the toolkit and as well in the runtime. (opened PMRS)
We are trying to use some workarounds but I'm sure we haven't kicked all the tires yet, and some differences we may never see, not having the flows to reveal them.
Remember your mileage may vary. 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bsiggers |
| Posted: Mon May 30, 2011 11:56 am Post subject: |
|
|
Acolyte
Joined: 09 Dec 2010
Posts: 53
Location: Vancouver, BC
|
Hello,
A great base to start from with these monitoring/auditing events is this IBM document - it is very good, and the basis for everything that we've done.
http://www.ibm.com/developerworks/websphere/library/techarticles/0911_fan/0911_fan.html
| Quote: |
| Which events did you switch on? How do you subscribe to them? Do you have a monitoring profile you share across flows, or do you configure events in design time? |
What I'm interested in is basically flowstart/flowend/flowrollback events, and also the terminal.in entries for various nodes within the flows themselves. I have the option to capture the payload, which I've chosen not to do at the moment.
I have one monitoring profile per flow, and have automated the process of deploying, activating, etc. the monitoring profiles using a manually created script. The actual events that I'm capturing I've defined in XML files and these get applied to the monitoring profile using the same script. There is probably a very smart, automated way to do this, but unfortunately I don't have the time or energy to do so  I don't configure events at designtime - everything is applied without even a redeployment of the flow.
In the back end, all of these events go to a topic to which I've created a subscriber queue, and all the events are processed by a 'audit' message flow, which basically just appends the slightly-formatted events to one file per flow. This provides me with the primitive logging capability that we were looking for.
| Quote: |
| About the monitoring scripts, how do you know that a flow is no longer working - whether a web-service flow or an mq one? Did you try to subscribe to the broker's operational publications? |
My monitoring is very simple - it basically parses the output of 'mqsilist <broker> -e <execution group> looking for BIP1288I (flow ok) and BPI1289I (flow stopped) messages, counting them up. I also have other monitoring checking tons of stuff at the MQ level (inhibited queues, do the required queues have listeners, is there backlog anywhere, are messages being dequeued, etc.). I've also got external probes checking the web services from the outside making sure that we're getting valid http responses back. I've tried to avoid relying on broker internals and operational publications, as to be honest I haven't used them enough to trust them as a basis for monitoring. I've got more faith in the command-line tools at this point.
All this is a very simple mechanism for monitoring - as we learn more about the system, we're constantly expanding our monitoring, adding new probes, checks, etc. It's a learning process.
Hope this is useful. Let me know if you'd like more information. |
|
| Back to top |
|
|
| zpat |
| Posted: Mon May 30, 2011 12:29 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| fjb_saper wrote: |
| zpat wrote: |
| hopsala wrote: |
| Well, we didn't have to detect it, the code just didn't compile. I was only partially involved, but I remember something about ESQL no longer being able to accept functions as parameters in some cases, and a subtle different in REFERENCE variable behavior.. |
Subtle differences worry me. That could make it into production and then cause problems. Do IBM provide any documentation on this? |
Not to my knowledge. We're moving from V6.0.0.10 to V7.0.0.2 and seeing different behavior on the MRM parser in the toolkit and as well in the runtime. (opened PMRS)
We are trying to use some workarounds but I'm sure we haven't kicked all the tires yet, and some differences we may never see, not having the flows to reveal them.
Remember your mileage may vary. |
Please post here when you get an answer on your PMRs. |
|
| Back to top |
|
|
| hopsala |
| Posted: Sun Jun 12, 2011 12:34 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| bsiggers wrote: |
| I've also got external probes checking the web services from the outside making sure that we're getting valid http responses back |
First off, thanks for the detailed reply!
About the external probes - did you program your flows so that they accept such 'ping' requests, or did you use some other means?
I've lately learned about the http HEAD request which looks like the best way to go about it http://www.mqseries.net/phpBB2/viewtopic.php?t=57681, though I haven't tried it yet. |
|
| Back to top |
|
|
| bsiggers |
| Posted: Mon Jun 13, 2011 9:28 am Post subject: |
|
|
Acolyte
Joined: 09 Dec 2010
Posts: 53
Location: Vancouver, BC
|
| Based on the requirements of most of our webservices, most of them have a kind of query/lookup basis. So what we could do is just perform a query/lookup against a known good part number, for example, and just keep repeating and monitoring response times and return codes. This approach probably would not work so well with other webservices, but as you say you could probably work in some kind of 'test mode' or something like this, but like you mentioned this would involve some development effort. |
|
| Back to top |
|
|
|
|
