| Author |
Message
|
| hal |
 Posted: Tue Nov 06, 2012 3:30 pm Post subject: WebSphere Message Broker 8.0.0.1 defunct processes |
 |
|
Acolyte
Joined: 07 Dec 2005
Posts: 67
Location: New York City, New York
|
Defunct processes appear after the Broker starts up in WebSphere Message Broker Multi-Platfoms Version 8.0.0.1
IBM has two fixes that worked for me on my Red Hat Enterpise Linux OS. If you are running WebSphere MQ 7.1 or 7.5, ask IBM for both IC86231 and IC87190. With WebSphere MQ 7, IC86231 by itself may do the trick.
From http://www-01.ibm.com/support/docview.wss?uid=swg1IC86231
| Quote: |
When using WMB 8.0.0.1, defunct mqsifindmqpath processes are
created on UNIX platforms by the bipbroker and DataFlowEngine
processes. These defunct processes remain until the broker is
stopped, and reappear when the broker is started again. |
From http://www-01.ibm.com/support/docview.wss?uid=swg1IC87190
| Quote: |
When using WMB 8.0.0.1 on a system which has WMQ 7.1 or 7.5
installed, defunct processes are created on UNIX platforms by
the bipbroker and DataFlowEngine processes. These defunct
processes remain until the broker is stopped, and reappear when
the broker is started again. |
Hal |
|
| Back to top |
|
 |
| rekarm01 |
| Posted: Wed Nov 07, 2012 12:59 am Post subject: Re: WebSphere Message Broker 8.0.0.1 defunct processes |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| hal wrote: |
| Defunct processes appear after the Broker starts up in WebSphere Message Broker Multi-Platfoms Version 8.0.0.1 |
Aside from cluttering up the "ps" output, are the defunct processes creating other problems? |
|
| Back to top |
|
|
| hal |
| Posted: Thu Nov 15, 2012 12:20 pm Post subject: |
|
|
Acolyte
Joined: 07 Dec 2005
Posts: 67
Location: New York City, New York
|
| Hard to say because my WebSphere Message Broker was very lightly used until I put the fixes in. Before that I detected three defunct processes for each execution group. Perhaps a UNIX system administrator can explain any possible side-effects of defunct/zombie processes. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Nov 15, 2012 12:40 pm Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| hal wrote: |
| Hard to say because my WebSphere Message Broker was very lightly used until I put the fixes in. Before that I detected three defunct processes for each execution group. Perhaps a UNIX system administrator can explain any possible side-effects of defunct/zombie processes. |
Zombie processes are sometimes caused by using mqm as the service Id. mqm should never be used as the Message Broker service Id.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| hal |
| Posted: Thu Nov 15, 2012 2:34 pm Post subject: |
|
|
Acolyte
Joined: 07 Dec 2005
Posts: 67
Location: New York City, New York
|
Just read your "Why mqm should not be the Broker Service Id" topic. It is unfortunate that IBM did not clearly document a unique service Id requirement.
I started with the NEON (New Era Of Networks) message broker and converted that to MQSI 2.1. Way back then I used separate service Ids for the MQSI and MQSeries products. When I upgraded from MQSI 2.1 to WMB and from MQSeries to WMQ, I explictly asked IBM if I could safely combine my two Ids into one mqm Id and they answered me in the affirmative. (One service Id simplifies Middleware administration.)
Over the years I encountered situations when the Broker would not start until I deleted all files from the /var/mqsi/common/locks directory and/or removed all IPCs owned by mqm via the ipcrm/amqiclen commands. Your explanation sheds some light about what happens under the covers when using a single service Id. But it is going to be a hassle for me to split it now, especially in production environments. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Nov 16, 2012 6:02 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| hal wrote: |
Just read your "Why mqm should not be the Broker Service Id" topic. It is unfortunate that IBM did not clearly document a unique service Id requirement.
I started with the NEON (New Era Of Networks) message broker and converted that to MQSI 2.1. Way back then I used separate service Ids for the MQSI and MQSeries products. When I upgraded from MQSI 2.1 to WMB and from MQSeries to WMQ, I explictly asked IBM if I could safely combine my two Ids into one mqm Id and they answered me in the affirmative. (One service Id simplifies Middleware administration.)
Over the years I encountered situations when the Broker would not start until I deleted all files from the /var/mqsi/common/locks directory and/or removed all IPCs owned by mqm via the ipcrm/amqiclen commands. Your explanation sheds some light about what happens under the covers when using a single service Id. But it is going to be a hassle for me to split it now, especially in production environments. |
Better late than never. IBM has acknowledged the problem in several ways: technote and email.
While it may appear to be difficult, its not as hard as it may seem. Simply add a sudo command right before Broker starts. Also remember to sudo to the Broker service Id before executing any mqsi commands in your runtime server (ie. don't run mqsi commands directly from your personal id).
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Nov 16, 2012 6:51 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| hal wrote: |
| I explictly asked IBM if I could safely combine my two Ids into one mqm Id and they answered me in the affirmative. |
It takes the pain of zombie processing to realize you cannot use mqm Id.
While I do appreciate the skill of IBM support technicians, some of them may not have field experience to understand your question. On the surface, it would seem logical that the WMB runtime can use any Id, as long as the Id was a member of the mqbrkrs group (which we know is not correct).
It's not until you fully comprehend the implications of ' primary group membership ' versus ' secondary group membership ' that you realize the fact that the mqm id cannot be used for the Broker service Id.
In order to release WMB internal locks , the primary group membership must be mqbrkrs not mqm. This is why you must sudo to an Id whose primary group membership is mqbrkrs in order to properly run mqsi commands.
If you mqsideploy a bar file under an Id who is not primary group membership of mqbrkrs, your Id is locking those internal WMB entities with a pgid other than mqbrkrs. Later, when WMB runtime does housekeeping, it cannot gain an exclusive lock on those internal entities and results in a core dump / zombie process. The reason it cannot lock the semaphore is because it is trying to lock with pgid of mqbrkrs but you hold the lock with a different pgid. In some cases you can get away with this because in the few minutes you deploy the bar, there is no contention. In other cases when WMB tries to intervene while your bar deployment is happening, is when this conflict occurs.
The Broker service Id is not enough simply to have group membership in mqbrkrs. Broker service Id and the Id you use to run mqsi commands MUST have PRIMARY group membership of mqbrkrs.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sat Nov 17, 2012 2:07 pm Post subject: Re: WebSphere Message Broker 8.0.0.1 defunct processes |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| hal wrote: |
| Perhaps a UNIX system administrator can explain any possible side-effects of defunct/zombie processes. |
Unless the number of zombie processes increases without limit, and fills up the process table, the performance impact should be negligible. Zombie processes do not consume CPU or memory resources. For WMB 8.0.0.1, the number of zombie processes seems to be bounded for each DataFlowEngine and bipbroker process. A fix for these issues is scheduled for WMB 8.0.0.2; is there a compelling need to apply a fix before then?
| lancelotlinc wrote: |
| It takes the pain of zombie processing to realize you cannot use mqm Id. |
While there are no doubt pains associated with using mqm as the broker service id, are any of these pains due to "zombie processing" itself, or are they due to other issues (e.g. semaphore/shared memory cleanup)? |
|
| Back to top |
|
|
| DSPS |
| Posted: Thu Feb 14, 2013 2:58 am Post subject: defunct processes still there after creating new user. |
|
|
Apprentice
Joined: 11 Sep 2008
Posts: 44
Location: Southern England
|
Thank you very much for the pointer to the fix(es) - which I'll try and get hold of. I've tried creating a new user for broker that is distinct from the MQ administrative user, new user has the broker group as their primary group, and I'm still getting the defunct/zombie processes.
I'll try and find out from my current customer what our arrangements are for getting fixes from IBM - since fixpack 8.0.0.2 is just showing as being available sometime in Q1 this year. Well, today is nearly half way through Q1....but I'm not sure my customer would want to wait up to six weeks for this to be tidied up.
So, I'll try to get hold of the specific fixes IC86231 and IC87190.
Thanks again.
_________________
Distributed Systems Professional Services
WebSphere and MQ Consulting and Training |
|
| Back to top |
|
|
|
|
