| Author |
Message
|
| deepjyot |
 Posted: Wed Jul 29, 2009 11:22 pm Post subject: Messages missing in remote queue after failover (HACMP) |
 |
|
Novice
Joined: 29 Jul 2009
Posts: 11
|
Hi,
We have a HA IBM AIX cluster and a stand alone AIX server in our environment. The stand alone AIX server has a remote defintion of a local queue on one node of the cluster and one sender channel from stand alone server to active broker box in HA cluster with the receiver channel.
We are sending out 15000 messages from stand alone AIX server and expecting them to reach the remote queues physically present on the 1 of the AIX servers in HA cluster. But, if we failover to standby AIX server in HA cluster from the Active broker box in HA cluster while messages are being sent, some messages are lost and they do not reach the remote queue. All messages folowing the lost message can be found in the remote queue once the applications (Queue Manager, Broker, etc.) come up on the stand by AIX server in HA cluster.
Can anyone let me know if we need to set some property of any queue or channel that has been created for this intercommunication?
Thanks
Deepjyot Singh
Cell: +1-6024021879 |
|
| Back to top |
|
 |
| AkankshA |
| Posted: Wed Jul 29, 2009 11:30 pm Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
why RQD ?? which queue manager name are you specifying ??
we use an alias queue in a somewhat similar sceneio and never faced any msg loss..
_________________
Cheers |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Wed Jul 29, 2009 11:32 pm Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| Quote: |
All messages folowing the lost message can be found in the remote queue
|
How about the messages sent before the failover? Are these all lost or just the ones that were in transit when the QM went down? If they are missing do they reappear when you fail back and are you using persistent messages?
Have you checked DLQs on both QMs (sender and receiver) in case they went there for some reason? |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Jul 29, 2009 11:56 pm Post subject: Re: Messages missing in remote queue after failover (HACMP) |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| deepjyot wrote: |
| ...But, if we failover to standby AIX server in HA cluster from the Active broker box in HA cluster while messages are being sent, some messages are lost and they do not reach the remote queue. All messages folowing the lost message can be found in the remote queue once the applications (Queue Manager, Broker, etc.) come up on the stand by AIX server in HA cluster... |
Then there is a high probability that those messages are not persistent, the channel settings are NPMSPEED(FAST), and the messages are in transit down the channel when it abnormally ends - working as as advertised.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Jul 30, 2009 4:12 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Jul 30, 2009 5:56 am Post subject: Re: Messages missing in remote queue after failover (HACMP) |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| exerk wrote: |
| deepjyot wrote: |
| ...But, if we failover to standby AIX server in HA cluster from the Active broker box in HA cluster while messages are being sent, some messages are lost and they do not reach the remote queue. All messages folowing the lost message can be found in the remote queue once the applications (Queue Manager, Broker, etc.) come up on the stand by AIX server in HA cluster... |
Then there is a high probability that those messages are not persistent, the channel settings are NPMSPEED(FAST), and the messages are in transit down the channel when it abnormally ends - working as as advertised. |
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| deepjyot |
| Posted: Thu Jul 30, 2009 3:03 pm Post subject: |
|
|
Novice
Joined: 29 Jul 2009
Posts: 11
|
@Vitor: The message property has been set to persistent, also the queue property is also set as persistent. And yes, the messages are in transit when i failover.
@WMBDEV1: The DLQ's do have any messages and the messages continue to reach the remote queues even after failover but 1 or 2 messages are lost. Before the failover everything works perfectly fine. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Jul 30, 2009 7:56 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
When the qmgr came back up were there any channels in doubt and how were they resolved? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| deepjyot |
| Posted: Fri Jul 31, 2009 3:18 pm Post subject: |
|
|
Novice
Joined: 29 Jul 2009
Posts: 11
|
| The channels were running... |
|
| Back to top |
|
|
| deepjyot |
| Posted: Fri Jul 31, 2009 4:31 pm Post subject: |
|
|
Novice
Joined: 29 Jul 2009
Posts: 11
|
<ErrorMessage>
<DateOfError>2009-07-31</DateOfError>
<TimeOfError>16:13:02.643975</TimeOfError>
<ErrorNumber>5294</ErrorNumber>
<ErrorText>RM Worker Not Found.</ErrorText>
<ErrorLabel></ErrorLabel>
<FlowPath>
<ACKNAKMessage/>
<Status>buildErrorQueueMessage</Status>
<CommitReached>Y</CommitReached>
<FlowCategory>SVF</FlowCategory>
<BatchStatus>N</BatchStatus>
<HL7Validated>Y</HL7Validated>
<HL7>Y</HL7>
<NakStatus>defaultNAK</NakStatus>
<AckType>ACK</AckType>
<MsgContent></MsgContent>
<AllowRollback>N</AllowRollback>
<ErrorStatus>endWriteToMapping</ErrorStatus>
</FlowPath>
<Insert>
<ErrorInsert>10</ErrorInsert>
<ErrorInsert>rmLoadWorker</ErrorInsert>
<ErrorInsert></ErrorInsert>
<ErrorInsert>HL7</ErrorInsert>
</Insert>
<EntireErrorTree>
<RecoverableException>
<File>/build/S600_P/src/DataFlowEngine/ImbMqOutputNode.cpp</File>
<Line>793</Line>
<Function>ImbMqOutputNode::evaluate</Function>
<Type>ComIbmMQOutputNode</Type>
<Name>SBMC67_MARMIX1_VALID_2_MAP#FCMComposite_1_4</Name>
<Label>SBMC67_MARMIX1_VALID_2_MAP.SBMC67.MARMIX1.NTX.MF</Label>
<Catalog>BIPv600</Catalog>
<Severity>3</Severity>
<Number>2230</Number>
<Text>Caught exception and rethrowing</Text>
<ParserException>
<File>/build/S600_P/src/MTI/MTIforBroker/MtiImbParser2/MtiImbRMErrorMap.cpp</File>
<Line>170</Line>
<Function>MtiImbRMErrorMap::checkRC</Function>
<Type></Type>
<Name></Name>
<Label></Label>
<Catalog>BIPv600</Catalog>
<Severity>3</Severity>
<Number>5294</Number>
<Text>RM Worker Not Found.</Text>
<Insert>
<Type>2</Type>
<Text>10</Text>
</Insert>
<Insert>
<Type>5</Type>
<Text>rmLoadWorker</Text>
</Insert>
<Insert>
<Type>5</Type>
<Text></Text>
</Insert>
<Insert>
<Type>5</Type>
<Text>HL7</Text>
</Insert>
</ParserException>
</RecoverableException>
</EntireErrorTree>
</ErrorMessage> |
|
| Back to top |
|
|
| exerk |
| Posted: Sat Aug 01, 2009 3:25 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
THIS may (or may not) be relevant, but if I'm reading the error message code correctly (probably not, it's me after all):
| Code: |
.
.
<CommitReached>Y</CommitReached>
.
.
<AllowRollback>N</AllowRollback>
.
. |
So are you perhaps getting a few messages, and because they can't be processed, you are discarding them?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
|
|
