| Author |
Message
|
| RogerLacroix |
 Posted: Thu Aug 30, 2012 1:59 pm Post subject: What UserID started a queue manager on Linux? |
 |
|
Jedi Knight
Joined: 15 May 2001
Posts: 3254
Location: London, ON Canada
|
All,
I'm have a really weird problem (or just a bad week). It is a LD_LIBRARY_PATH environment variable issue on a Linux server. On 1 server everything is fine but on an identical server, the LD_LIBRARY_PATH environment variable is always NULL when a security exit is invoked by the queue manager.
The mqm account is using korn shell (/bin/ksh) as the default shell and the .profile file has the following in it:
| Code: |
LD_LIBRARY_PATH=/var/mqm/exits64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH |
When the customer does "echo $LD_LIBRARY_PATH" command, it shows that LD_LIBRARY_PATH is set correctly. The customer says that they are starting the queue manager from the mqm account. But when the security exit issue getenv("LD_LIBRARY_PATH") on this particular server, getenv returns NULL.
I'm totally stumped. Any Unix/Linux gurus out there know what goofy thing is biting me in the posterior?
A second question, is there anyway to tell if a user is starting a queue manager with a user account (not mqm) that is in the mqm group? If I do "ps -ef | grep mqm", it shows that the queue manager is running as mqm but that does not mean it was started by mqm (if you know what I mean).
Help.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Thu Aug 30, 2012 7:01 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20700
Location: LI,NY
|
This looks like foreground vs background problem.
When they have the problem is the sudo or su done right?
| Quote: |
| su -c mqstartscript.ksh |
Maybe the script shoudl source the mqm profile (justo be sure)?
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Mon Sep 03, 2012 3:21 pm Post subject: Re: What UserID started a queue manager on Linux? |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| RogerLacroix wrote: |
| I'm totally stumped. Any Unix/Linux gurus out there know what goofy thing is biting me in the posterior? |
One possibility is that the Korn shell is running in privileged mode (i.e. the effective user/group id does not match the real user/group id). In this case, the Korn shell does not process $HOME/.profile (or $ENV). Check whether any suspect scripts or executable files have the setuid/setgid permissions enabled:
| Code: |
$ ls -l file
-rwsr-sr-x ... file
^ ^
| +-- setgid
+-- setuid |
|
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Tue Sep 04, 2012 3:05 pm Post subject: Re: What UserID started a queue manager on Linux? |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3254
Location: London, ON Canada
|
| rekarm01 wrote: |
| One possibility is that the Korn shell is running in privileged mode (i.e. the effective user/group id does not match the real user/group id). In this case, the Korn shell does not process $HOME/.profile (or $ENV). Check whether any suspect scripts or executable files have the setuid/setgid permissions enabled: |
They checked and permissions are fine.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Tue Sep 04, 2012 3:06 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3254
Location: London, ON Canada
|
All,
Ahhhh. This is driving me crazy. We have checked and checked and everything looks correct on the second server. Here is a screen capture I just received (no startup script just a simple strmqm command):
| Quote: |
$ endmqm TESTQM
Quiesce request accepted. The queue manager will stop when all outstanding work is complete.
$ dspmq
QMNAME(TESTQM) STATUS(Ended normally)
$ echo $LD_LIBRARY_PATH
/var/mqm/exits64:
$ strmqm TESTQM
WebSphere MQ queue manager 'TESTQM' starting.
The queue manager is associated with installation 'Installation1'.
11 log records accessed on queue manager 'TESTQM' during the log replay phase.
Log replay for queue manager 'TESTQM' complete.
Transaction manager state recovered for queue manager 'TESTQM'.
WebSphere MQ queue manager 'TESTQM' started using V7.1.0.1.
$ dspmq
QMNAME(TESTQM) STATUS(Running) |
And when a connection comes in, the security exit shows that LD_LIBRARY_PATH is NULL. Ahhhhhh.
But there is a new clue. I did not know that they were using WMQ v7.1.0.1 on their servers.
Since, crtmqenv and setmqenv commands set up environments variables which include LD_LIBRARY_PATH, does anyone know where this information is stored. I'm thinking that LD_LIBRARY_PATH has been corrupted. I can see /var/mqm/mqs.ini has the stanzas with "InstallationName=Installation1" but where is the information related to Installation1 stored on Linux?
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| nathanw |
| Posted: Wed Sep 05, 2012 12:30 am Post subject: |
|
|
Knight
Joined: 14 Jul 2004
Posts: 550
|
Just out of curiosity has the QM been cold stated in the past? The old trick of creating a similar QM and copying certain files across?
Because I am wondering if the new files did not have the correct information or had a different link for certain info hence why you do not see it.
Just a thought
_________________
Who is General Failure and why is he reading my hard drive?
Artificial Intelligence stands no chance against Natural Stupidity.
Only the User Trace Speaks The Truth  |
|
| Back to top |
|
|
| Philip Morten |
| Posted: Thu Sep 06, 2012 6:07 am Post subject: |
|
|
Master
Joined: 07 Mar 2002
Posts: 230
Location: Hursley Park
|
| RogerLacroix wrote: |
And when a connection comes in, the security exit shows that LD_LIBRARY_PATH is NULL. Ahhhhhh.
|
Is this not because LD_LIBRARY_PATH is ignored for setuid/setgid programs?
_________________
Philip Morten
The postings on this site are my own and do not necessarily represent IBM's positions, strategies or opinions. |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Thu Sep 06, 2012 11:25 am Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3254
Location: London, ON Canada
|
| nathanw wrote: |
Just out of curiosity has the QM been cold stated in the past? The old trick of creating a similar QM and copying certain files across?
Because I am wondering if the new files did not have the correct information or had a different link for certain info hence why you do not see it. |
This is not an HA or mult-instance setup. Just 2 different servers with 2 different queue managers where 1 works correctly and the other does not.
Yes, a cold start was done as noted above with endmqm and strmqm.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Thu Sep 06, 2012 11:32 am Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3254
Location: London, ON Canada
|
All,
On the MQ ListServer, Andy Hicksen of IBM posted:
| Andy Hicksen wrote: |
| In MQ V7.1 there are some cases where the queue manager will internally reset LIBPATH when creating a new process. |
I just happened to ask the customer for an MQ trace - very simple, start QMgr, start trace, have the client connect, stop the trace and here is the results:
| Code: |
Trace Data
===============
Thread stack (from libmqmcs_r.so)
-> InitProcessInitialisation
{ InitProcessInitialisation
This library has dependencies on (libmqmcs_r.so)
-{ xcsGetIFIXData
--{ xcsQueryValue
id: 14
0x0000: 2f6f7074 2f6d716d |/opt/mqm |
length: 8
--} xcsQueryValue rc=OK
Patch files is zero length
-} xcsGetIFIXData rc=OK
-{ xcsIsEnvironment
xcsIsEnvironment[AMQ_NO_CS_RELOAD] = FALSE
-} xcsIsEnvironment rc=OK
-{ xcsLoadFunction
LibName(libmqmcs_r.so) LoadType(2097200)
General, comms, CS, OAM, or WAS
--{ xcsQueryValue
id: 13
0x0000: 2f6f7074 2f6d716d 2f6c6962 3634 |/opt/mqm/lib64 |
length: 14
--} xcsQueryValue rc=OK
--{ xcsGetEnv
xcsGetEnv[LD_LIBRARY_PATH] = NULL
--} xcsGetEnv rc=OK
FullPathLibName(/opt/mqm/lib64/libmqmcs_r.so) loaded with dlopen |
I'm biting my tongue right now, as I have wasted so many hours on this that it is ridiculous. The 24th line, basically the start of the MQ trace clearly shows that the LD_LIBRARY_PATH environment variable is NULL (well before it ever loaded the security exit).
And the best part - other MQ trace files show LD_LIBRARY_PATH environment variable being set properly. So YES, that quirk Andy Hicksen described is really the problem.
i.e.
| Code: |
--{ xcsGetEnv
xcsGetEnv[LD_LIBRARY_PATH] = '/var/mqm/exits64:'
--} xcsGetEnv rc=OK |
So, I have asked the customer to open a PMR with IBM to fix this quirk (bug).
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Sep 06, 2012 11:46 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
It's also possible that some programs are being started in a way that causes the ksh to evaluate the .profile, and in some cases some programs are being started in a way that causes ksh to evaluate .kshrc.
http://osr507doc.sco.com/en/OSUserG/_The_Korn_shell_profile_and_kshrc.html
So whilst you're sure you've set it in .profile, you might not have double-checked that it's set in .kshrc. |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Thu Sep 06, 2012 1:38 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3254
Location: London, ON Canada
|
| mqjeff wrote: |
It's also possible that some programs are being started in a way that causes the ksh to evaluate the .profile, and in some cases some programs are being started in a way that causes ksh to evaluate .kshrc.
http://osr507doc.sco.com/en/OSUserG/_The_Korn_shell_profile_and_kshrc.html
So whilst you're sure you've set it in .profile, you might not have double-checked that it's set in .kshrc. |
There was only 1 program being started (strmqm) and the MQ trace shows that all but 1 MQ trace file had LD_LIBRARY_PATH set correctly, so I think the issue is what Andy Hicksen stated.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Sep 06, 2012 5:30 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
strmqm starts a large set of processes. And then some of those processes start other processes, like the listener process that's spawning the thread that receives the client connection and loads your exit.
The issue Andy Hickson likely is the cause of the problem, but it's really really worth taking two minutes to double-check .kshrc. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Fri Sep 07, 2012 1:16 am Post subject: Re: What UserID started a queue manager on Linux? |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| RogerLacroix wrote: |
And when a connection comes in, the security exit shows that LD_LIBRARY_PATH is NULL. Ahhhhhh.
But there is a new clue. I did not know that they were using WMQ v7.1.0.1 on their servers. |
Since strmqm has the setuid/setgid bits set, it's entirely likely that LD_LIBRARY_PATH has always been NULL. Before WMQ v7.1, this wouldn't necessarily be a problem, as only one instance of WMQ could be installed at a time, and symbolic links in /usr would point to the installation, resolving any library references.
But WMQ v7.1 allows for multiple installations, a primary installation, and non-primary installations. For Linux, the primary installation must be set manually, using the setmqinst command. For the primary installation, exits can still work as-is. For non-primary installations, exits can be modified to use function pointers instead of referring to WMQ libraries directly. |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Fri Sep 07, 2012 2:46 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3254
Location: London, ON Canada
|
| mqjeff wrote: |
| but it's really really worth taking two minutes to double-check .kshrc. |
I have asked the customer to put LD_LIBRARY_PATH in .kshrc and give it another try.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Fri Sep 07, 2012 2:57 pm Post subject: Re: What UserID started a queue manager on Linux? |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3254
Location: London, ON Canada
|
I really wish people would fully read a person's posting before making comments. None of what you posted is applicable.
| rekarm01 wrote: |
| Since strmqm has the setuid/setgid bits set, it's entirely likely that LD_LIBRARY_PATH has always been NULL. Before WMQ v7.1, this wouldn't necessarily be a problem, as only one instance of WMQ could be installed at a time, and symbolic links in /usr would point to the installation, resolving any library references. |
Did you read my posting regarding 1 MQ trace file showing LD_LIBRARY_PATH being set correctly and another where it is NULL (from the same trace). 
| rekarm01 wrote: |
| But WMQ v7.1 allows for multiple installations, a primary installation, and non-primary installations. For Linux, the primary installation must be set manually, using the setmqinst command. For the primary installation, exits can still work as-is. For non-primary installations, exits can be modified to use function pointers instead of referring to WMQ libraries directly. |
I never said anything about the customer having multiple versions of MQ installed. All I said was that they are running WMQ v7.1.0.1 on both servers. 1 server works perfectly, the other has this problem. Secondly, I never said anything about the security exit having issues with referring to WMQ libraries.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
|
|
