Author |
Message
|
tarunmq1 |
Posted: Mon Oct 06, 2008 12:31 pm Post subject: Unique problem..MDB just stops processing |
|
|
Newbie
Joined: 15 Oct 2007 Posts: 7
|
Hi All,
My situation
I have two MDBs running pointing to two different Q Managers.
Due to message affinity, I can only run one instance at a time.
Every now and then, one MDB just stops processing the messages (and its the same MDB every time) The other MDB keeps running fine. The messages start to pile up in this one queue and then when we bounce it, it works fine. The listener ports never get stopped.
I don't see any exceptions in the logs so can't really put a handle on what's going wrong where.
We thought may be the connections weren't enough so we increased the connection pool from 10 to 20 but that didn't help.
Is it because its only one session and it gets destroyed or something?
Is there a way I can enable tracing on the MDB level?
I am using MQ5.3 w/ WAS5.1.
In case of JMS exception, i am rolling back the transaction. Does that mean the message is put back on the Q?
I have a backout threshold property set to 2 and a backout Q defined. and the listener port retries property is to the max # so my understanding is that after 2 retries it should send the message to the backout Q.
It happens only on production. Never experienced it on QA or any other environment.
I am attaching the code of my MDB
===========================
public void onMessage(javax.jms.Message msg) {
String xmlMsg = null;
if(msg instanceof TextMessage) {
try {
xmlMsg = ((TextMessage) msg).getText();
MsgLogger.logSPSRecvMsg(xmlMsg);
XXXFactory.processMessage(xmlMsg);
} catch (XmlException e) {
MsgLogger.logRejectedMsg(xmlMsg);
XXXLogger.MQ.fatal("XmlException caught " + e);
e.printStackTrace();
} catch (JMSException e) {
XXXLogger.MQ.fatal("JMSException caught " + e);
fMessageDrivenCtx.setRollbackOnly();
e.printStackTrace();
}
catch (Throwable e) {
XXXLogger.MQ.fatal("Throwable caught " + e);
XXXLogger.MQ.fatal("FAILED TO PROCESS THIS MESSAGE " + xmlMsg);
e.printStackTrace();
}
}
else {
XXXLogger.MQ.fatal("FAILED TO PROCESS THIS MESSAGE -- WAS NOT OF TYPE TEXT MESSAGE " + xmlMsg);
}
}
========================================
The other MDB (which works fine) has pretty much the same code.
Please let me know if you need me to provide more info..
Please help!!!!!!!
Thanks in advance, |
|
Back to top |
|
 |
atheek |
Posted: Tue Oct 07, 2008 2:37 am Post subject: |
|
|
 Partisan
Joined: 01 Jun 2006 Posts: 327 Location: Sydney
|
What do you mean by "MDB stopped" here. Did you check:
1. IPPROCS for the queue..Is it still showing 1 ( since you mentioned you are using single instance). ?
2. Did you check the backout count for the messages ..Are you sure your mdb is not repeatedly retrying to process a poison message.
I dont have WAS experience ..so not sure whether you have configured MDB correctly to move messages to a backout queue. However, it should be noted that setting the backout count on the queue itself doesn't move messages to the backout Q. It has to be done by the application. Just wait for fjb_saper and others to come here for your rescue...
tarunmq1 wrote: |
In case of JMS exception, i am rolling back the transaction. Does that mean the message is put back on the Q? |
From your code it looks like you are using Container Managed Transaction (CMT) . If so the messages will rollback in case of JMSException as you are calling setRollback of the message driven context
tarunmq1 wrote: |
I am using MQ5.3 w/ WAS5.1. |
MQ 5.3 is out of support since long ago..Better migrate to a supported version |
|
Back to top |
|
 |
fjb_saper |
Posted: Tue Oct 07, 2008 3:11 am Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
OK the doc says that WAS does automagically move the message to the backout / DLQ. So setting the bo threshold should be sufficient.
However for the MDB to do this and go on processing you need to have :
MDB retry > bothresh.
The MDB retry is a WAS thing and configurable in WAS.
If you have MDB retry <= bothresh the MDB will just stop on a poison message.
Enjoy  _________________ MQ & Broker admin |
|
Back to top |
|
 |
tarunmq1 |
Posted: Tue Oct 07, 2008 4:59 am Post subject: |
|
|
Newbie
Joined: 15 Oct 2007 Posts: 7
|
atheek, we are in the process of migrating to 6.0...but i just want to make sure that this thing doesn't happen in 6.0
i didn't run the IPPROCS command..will run it and see..
As fjb_saper said, WAS does automatically move the Q to backout. I have retries # set to the max Integer value (2........) and the back out threshold is set to 2 and i have defined a backout Q.
Even if this was happening, the listener port would have stopped too but the port never stops. Just the messages stop getting processed from this one Q. During implementing this, I had read about the poison thing and the listener port stopping if the retries < threshold count so i had made sure that the bothreshold count is less than retries count. |
|
Back to top |
|
 |
fjb_saper |
Posted: Tue Oct 07, 2008 11:27 am Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
I have seen this choke on a malformed message (Bad RFH header).
Use rfhutil(c) to inspect the message...  _________________ MQ & Broker admin |
|
Back to top |
|
 |
atheek |
Posted: Tue Oct 07, 2008 4:56 pm Post subject: |
|
|
 Partisan
Joined: 01 Jun 2006 Posts: 327 Location: Sydney
|
tarunmq1 wrote: |
As fjb_saper said, WAS does automatically move the Q to backout |
Ahh..great. Heard one of our architects recently saying that IBM products mix well with their own products but not others's
tarunmq1 wrote: |
The messages start to pile up in this one queue and then when we bounce it, it works fine |
Assume you are bouncing WAS/MDB. If the issue is with a poison message, you should have still got the issue after the bounce
Have you checked IBM site for any APAR's for the versions of WMQ/WAS you are having |
|
Back to top |
|
 |
fjb_saper |
Posted: Tue Oct 07, 2008 7:39 pm Post subject: |
|
|
 Grand High Poobah
Joined: 18 Nov 2003 Posts: 20756 Location: LI,NY
|
You did not specify the CSD's or fix packs.
There is somewhere a cheat sheet that tells you which version/fixpack of WAS plays nice with which version/fixpack of MQ.
I remember moving from MQ CSD4 to CSD6 we had to add some fixpacks to WAS.
Enjoy  _________________ MQ & Broker admin |
|
Back to top |
|
 |
tarunmq1 |
Posted: Wed Oct 08, 2008 7:01 am Post subject: |
|
|
Newbie
Joined: 15 Oct 2007 Posts: 7
|
Thanks for all the replies guys. Sincerely appreciated.
fjb_saper, can you point to some documentation regarding rfhutil(c) ? Based on what i understood, this is an utility. Would this run in tandem with my JMS based app?
atheek, we are bouncing WAS and the version we are using is 5.3 CSD04
I will try to find that cheat sheet but i don't know if that will help because the the other MDB has never had an issue and both the Q Managers are on the same box.
Thanks again guys, |
|
Back to top |
|
 |
atheek |
Posted: Wed Oct 08, 2008 11:38 am Post subject: |
|
|
 Partisan
Joined: 01 Jun 2006 Posts: 327 Location: Sydney
|
|
Back to top |
|
 |
|