| Author |
Message
|
| RaviKrG |
 Posted: Wed Jan 05, 2011 3:25 am Post subject: Production broker down and not starting |
 |
|
Master
Joined: 07 Sep 2008
Posts: 240
|
Hi. when we try to start the broker it starts and we see all the bip and the Data processes but even afetr waiting for 30 min we do not see the queues getting the ipprocs . when we check the broker logs we get as
]BIP2080E: Execution group *******failed to return an internal configuration Start message response within a 60 second time period. : **********.agent: /build/S610_P/src/AdminAgent/ImbAdminAgent.cpp: 8263: ImbAdminAgent::getXMLResponse: :
We are every time time getting this exception on the broker logs.
env is broker 6.1 on solaris.
any help will be highly appreciated |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Wed Jan 05, 2011 3:28 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
There should be more information listed in the logs.
Do you see any abend files being created?
Otherwise, take a startup trace and see.
Is the Broker database available? |
|
| Back to top |
|
|
| RaviKrG |
| Posted: Wed Jan 05, 2011 3:37 am Post subject: |
|
|
Master
Joined: 07 Sep 2008
Posts: 240
|
| Yes there was some abend files getting generated but could not get head and tail out of it. and yes the db2 is currently running file ... is there any other alternatives which could really bring up the services.. in the mean time we can have a look at the start up trace also. and yes there are no other things in the logs .. the same things is getting updated again and again. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jan 05, 2011 4:20 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
The top block of the abend file should be relatively readable.
The other messages in the log should say *something* about why the EGs are failing. The one you posted just said that the EG didn't respond when asked to start up.
EDIT: Sorry, did I miss read your post? There are *no* other messages in the log, other than normal ones that the EG is being started and etc?
If this is a previously working system, and you made a change, and now it is a non-working system, you should examine in great detail EVERYTHING about the change. |
|
| Back to top |
|
|
| RaviKrG |
| Posted: Wed Jan 05, 2011 10:53 pm Post subject: |
|
|
Master
Joined: 07 Sep 2008
Posts: 240
|
| Thanks all for the reply. We did see that while starting the broker it was taking lot of semaphores. we have cleared the semaphores and now have increased the default value from 256. and after a fresh reboot of the server, we did a manual restart of the broker, qmgr and db2 and till now it has come up OK. but still we have opened a pmr and trying to know what really has hapened and whot could be the best fix for this. will keep you posted. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Jan 06, 2011 12:05 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Looking at the IPCS data is SOP IMHO.
Products like WMQ and Oracle and many others make extensive use of shared memory and semaphores to sync access to resources.
WMQ even has recommendations in the Install Manual for what setting to use as a minimum on Linux. Solaris etc.
Do your kernel settings meet with the WMQ Minimum? If not then you won't get much truck from IBM support here.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| RaviKrG |
| Posted: Thu Jan 06, 2011 12:22 am Post subject: |
|
|
Master
Joined: 07 Sep 2008
Posts: 240
|
| the values for mqm for sem-ids and shm-ids is 1024 and shm-memory is4294967296. and mq was fine untill we restarted the broker and we found that the db2 error logs complains of the semaphore issue. we incresed the values for the mb to sem-ids 8192 and shm-ids= 10240 and then we started the broker but still no use. hence after a reboot of the server all looked OK. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Jan 06, 2011 1:34 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Sometimes Semaphores and Shared memory segments get left behind after an app has shut down. If there is a crash then this is very common.
the ipcs command can be used to remove unused artefacts in these cases.
The alternative is as you found out, to reboot... 
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
|
|
