| Author |
Message
|
| Bni |
 Posted: Wed Aug 09, 2006 7:11 am Post subject: Shared Memory and semaphore |
 |
|
Apprentice
Joined: 09 Aug 2006
Posts: 29
|
Several weeks ago we are trying to install a patch (CDS08) to one of our production system with MQ version 5.2( no patches at all) on HP-UX. According to the installation instruction, shared memory and semaphore, if any, should be cleaned out before applying the patch. And my personal experience on the development server is that if we cannot clean the shared memory, patches cannot be installed. We cleaned the shared memory in production during that weekend attempt after we shutdown mq, but right after that, they just pop up.
I got a suggestion from another board that we need to disable the port listening service for MQM, and we did, but the shared memeory and semaphore comes right back up after we clean them up( after around 30 seconds).
Our sys admins cannot figure out or track where those processes are from. But sure those are created by mqm. They are still researching how to track those processes. If you run ipcs -m -p , we can get the LPID and CIPD for shared memory segments, but if you ps -p <LPID> or ps -p <CIPD>, you cannot get anything, it seems it orphaned. ipcs -s -p command cannot get any process id which creates semaphores. So there is not a way to even get a process which creates semapore. Now we are kind of stuck.
Any experts out there have any idea what is going on?
Thanks
Brian |
|
| Back to top |
|
 |
| mvic |
| Posted: Wed Aug 09, 2006 7:38 am Post subject: Re: Shared Memory and semaphore |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| Bni wrote: |
| Several weeks ago we are trying to install a patch (CDS08) to one of our production system with MQ version 5.2 |
5.2 has long been out of marketing, and out of support. However...
I think the MQ programs running might be /opt/mqm/bin/amqcrsta, being started by inetd in response to incoming channel requests.
You could try commenting out the MQ lines in inetd.conf, and restarting inetd (see manual pages for inetd and inetd.conf). Then install MQ maintenance. Then uncomment the MQ lines in inetd.conf and restart inetd. Restart queue managers etc.
Or maybe restart the queue managers before re-instating MQ lines inetd.conf - this might be a bit cleaner. But it's up to you. |
|
| Back to top |
|
|
| Bni |
| Posted: Wed Aug 09, 2006 7:44 am Post subject: |
|
|
Apprentice
Joined: 09 Aug 2006
Posts: 29
|
| Thanks, that is the advise I got from another board and it does not resolve the situation. I know 5.2 is out of support, but that is not our oldest, we have 5.0 running in our more critical production environment. |
|
| Back to top |
|
|
| xxx |
| Posted: Wed Aug 09, 2006 8:19 am Post subject: |
|
|
Centurion
Joined: 13 Oct 2003
Posts: 137
|
Stop listner, ( i.e comment it out in the inetd.conf ) so that there will be no connections ,
Stop MQ,
check if there are any amq process hanging , using ps -ef | grep amq
ipcs -a | grep -v mq , clean with the help of root id if needed,
Now you should be able to install the patch ! Else
reboot the server , making sure that MQ is not started automatically and also the lister is not started .
Finally go to supported 5.3 or 6.0 |
|
| Back to top |
|
|
| bbburson |
| Posted: Wed Aug 09, 2006 9:34 am Post subject: |
|
|
Partisan
Joined: 06 Jan 2004
Posts: 378
Location: Nowhere near a queue manager
|
| xxx wrote: |
| ipcs -a | grep -v mq , clean with the help of root id if needed, |
??? grep -v will give you everything EXCEPT the mq resources. Did you mean grep -i perhaps? |
|
| Back to top |
|
|
| xxx |
| Posted: Wed Aug 09, 2006 9:43 am Post subject: |
|
|
Centurion
Joined: 13 Oct 2003
Posts: 137
|
I am sorry,
grep -i , yep that will be a disaster if used with v |
|
| Back to top |
|
|
| Bni |
| Posted: Wed Aug 09, 2006 9:45 am Post subject: |
|
|
Apprentice
Joined: 09 Aug 2006
Posts: 29
|
| I believe Bruce is right, grep -v mq will not display anything related to MQ. I don't quite understand this whole thing. That is exactly what we did. Perhaps, I shall do a little bit different: let sys admin stop the listerning services and then I will bring down MQ and then clean shared memory and semaphore if there is any. Last time what we did was I first brought down MQ manager, then sys admin stop the listening services, then we clean the shared memory and semaphore, we even stop the service guard, then right after that, shared memeory and semaphore came back after around 30 seconds. Every advice I got points to the listerning services, which might contribute to this issue. maybe our sys admin did not stop the listening services completely? I doubt that, but I will try again with our sys admins. |
|
| Back to top |
|
|
| mvic |
| Posted: Wed Aug 09, 2006 4:36 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| Bni wrote: |
| sys admin stop the listening services |
If you mean inetd, it will help us if you say so precisely.
If the admins commented out the MQ lines and restarted inetd, you should have no problem from amqcrsta programs. So maybe it's worth looking elsewhere - eg. cron jobs.
If you can't track it down via inetd or reviewing the cron jobs on the machine, perhaps try saying
| Code: |
| strmqtrc -e -t all -t detail |
(run this as an mqm-group user, not root) and waiting for 5 minutes. Then
and take a look in /var/mqm/trace. Use dspmqtrc to format the trace files, and see if they tell you what processes were running against MQ. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Aug 09, 2006 6:06 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I'd thought the pid was part of the name of the trace file.
Or am I confusing that with the FDC?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Aug 10, 2006 2:38 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| jefflowrey wrote: |
I'd thought the pid was part of the name of the trace file.
Or am I confusing that with the FDC? |
The PID is indeed part of the trace file name, but the program filename (eg. /home/me/bin/myprogram) is not - that's the information I thought might be useful. |
|
| Back to top |
|
|
| Bni |
| Posted: Thu Aug 10, 2006 5:16 am Post subject: |
|
|
Apprentice
Joined: 09 Aug 2006
Posts: 29
|
| Thank you very much guys. I will definitely take a look at it. I am new to MQ and not trained at all on MQ except start and stop mq. I am learning bascailly by myself. I will see what I can find out to do a trace on it. |
|
| Back to top |
|
|
| Bni |
| Posted: Sat Aug 19, 2006 2:38 pm Post subject: I just tested trace file |
|
|
Apprentice
Joined: 09 Aug 2006
Posts: 29
|
Our sys admin stopped inetd for mqm and I stopped MQ application and then I clean the share memory and semaphore, I then started the tracefile on the server. After a while the share memory and semaphore came back, but there is nothing in trace directory. I waited for at least 10 minuets and i also cleaned the shared memory and semaphore twice, but nothing in the trace file while share memory and semaphore popped up from nowhere.
I am done with this problem, so the next step is to reboot the server and install the patch while everything is down.
Thanks for all the suggests and inputs. but If you still have any ideas, Iwould love to hear from you.
Again thank you guys. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Aug 19, 2006 5:09 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Have you thought about stopping as well any monitoring software such as Tivoli for MQ (omegamon for MQ) or QPasa etc...?
This too might have shared memory/ semaphores pop up in the "mqm" space.
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
