| Author |
Message
|
| paranoid221 |
 Posted: Wed Jun 26, 2024 10:31 am Post subject: Simulate a queue manager crash RDQM |
 |
|
Centurion
Joined: 03 Apr 2006
Posts: 101
Location: USA
|
Hi,I have a need to simulate a scenario where the queue manager crashes or becomes unresponsive. I need to demonstrate how we recover from such a scenario.
Anyone has ideas how to simulate this.
Thanks in advance.
_________________
LIFE is a series of complex calculations, somewhere multiplied by ZERO. |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Wed Jun 26, 2024 1:07 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Sure. Roboot the sever o/s, or pull the server power cord, or kill the LPAR where the qmgr runs or ?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| paranoid221 |
| Posted: Thu Jun 27, 2024 12:06 am Post subject: |
|
|
Centurion
Joined: 03 Apr 2006
Posts: 101
Location: USA
|
Appreciate the response.
There is no power cord to pull since this is a VM. Rebooting or power-cycling the VM is more or less a graceful shutdown of all services than a crash scenario.
Any other innovative ideas?
_________________
LIFE is a series of complex calculations, somewhere multiplied by ZERO. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Jun 28, 2024 3:50 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| paranoid221 wrote: |
Appreciate the response.
There is no power cord to pull since this is a VM. Rebooting or power-cycling the VM is more or less a graceful shutdown of all services than a crash scenario.
Any other innovative ideas? |
Have you tried killing the queue manager processes ?
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| paranoid221 |
| Posted: Fri Jun 28, 2024 5:32 am Post subject: |
|
|
Centurion
Joined: 03 Apr 2006
Posts: 101
Location: USA
|
Yes. I tried killing the processes with kill -9. They came right back up in a few seconds without any intervention on the same machine. These are RDQMs in a HA cluster.
The scenario we are trying to reproduce is where the queue manager becomes unresponsive either from corruption or extreme load or whatever the case may be and crashes and doesn't start back up. In this instance, should or shouldn't it failover to the alternate node? I'm tasked with this scenario since one of my customers experienced a queue manager crash in the past (they do not remember the exact cause of the crash) and they had to rebuild the queue manager again from the ground up. I've already tried the traditional ways - powering off the VM, disabling the network interface and others. These use cases caused the queue manager to failover to the alternate node in the cluster as expected. The queue manager crash I haven't been able to reproduce like we did in the past (kill -9).
Looking for ideas here
_________________
LIFE is a series of complex calculations, somewhere multiplied by ZERO. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jun 28, 2024 7:58 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| paranoid221 wrote: |
Appreciate the response.
There is no power cord to pull since this is a VM. Rebooting or power-cycling the VM is more or less a graceful shutdown of all services than a crash scenario.
Any other innovative ideas? |
Was I not clear??
Kill the the VM without graceful shutdown. Kill the o/s without graceful shutdown. You will likely need to adjust your settings so that automation doesn't automatically restart.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sat Jun 29, 2024 6:08 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| paranoid221 wrote: |
| These are RDQMs in a HA cluster. |
Ah, RDQM! I updated your post to include RDQM in the title.
The R in RDQM means Replicated. Your RDQM qmgr is replicated across three images. Killing one of the three enables one of the other replicates to continue.
Was an outage observed? How precisely did you test for an outage? Was the qmgr idle? Was it doing any real work? Did the work complete?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| hughson |
| Posted: Mon Jul 08, 2024 9:07 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
| paranoid221 wrote: |
| There is no power cord to pull since this is a VM. |
What VM technology are you using? Can't you just click on the ethernet disconnect button in your VM environment? I believe that's what I did using VirtualBox to do the equivalent of pulling out the network cable.
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Jul 09, 2024 7:57 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| hughson wrote: |
| paranoid221 wrote: |
| There is no power cord to pull since this is a VM. |
What VM technology are you using? Can't you just click on the ethernet disconnect button in your VM environment? I believe that's what I did using VirtualBox to do the equivalent of pulling out the network cable.
Cheers,
Morag |
The VMs I've worked with have a power-off equivalent button - usually zero with a 1 in it - a non-graceful o/s image shutdown.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| tczielke |
| Posted: Wed Jul 24, 2024 1:29 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
I am not sure if this is exactly what you are asking, but you can corrupt the qm.ini.
For example,
cp -p qm.ini qm.ini.bkp
echo wieo5uy20u534rwlnsfklweh4i29342-3urakldhfkjshdiry2034yr203u4203423ouj > qm.ini
stop the queue manager
try and start it, it should error:
strmqm QM1
IBM MQ queue manager 'QM1' starting.
The queue manager is associated with installation 'Installation1'.
The queue manager ended for reason 2195, 'MQRC_UNEXPECTED_ERROR'.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
|
|
