| Author |
Message
|
| pankajthakkar |
 Posted: Mon Mar 06, 2006 8:43 am Post subject: Websphere MQ with Weblogic 8.0 - BOTHRESH not working |
 |
|
Newbie
Joined: 26 Aug 2005
Posts: 5
|
We are using Weblogic 8.0 sp4 to connect to Websphere MQ v 5.3 using the "Foreign JMS Server" connectivity of Weblogic. Websphere MQ Qmgr is being accessed in "bindings" mode.
Using JMSAdmin tool, we have configured an XA Queue connection factories and queues in the file based jndi tree (.bindings) to connect to MQ. Weblogic is configured to use this bindings file to look up the file base JNDI tree for Weblogic MQ and also all binaries such as "com.ibm.mq.mqjms.jar" etc are available in its classpath.
An MDB is deployed on weblogic to listen to an MQ queue. The connectivity works fine and messages are delivered to the MDB (XA mode) and processed fine.
However the poison messages coming on the MQ queue are not being handled as expected.
The MQ queue (that the MDB is listening on) has both the "Backout Threshold" and "Backout Requeue name", "Hardened Get Backout" attributes set. Even then, when the MDB rolls back the message, the "backout count" on the MQ queue keeps on incrementing and even though it is past the "backout threshold" limit, the message is not being transferred to the "Backout Requeue".
I am not sure, if we are missing some configuration or something that tells the MQ queue to back out the message, but if someone can provide some insight on why is this happening OR has come across a similar issue, I did appreciate any tips/resolutions on that.
Thanks in advance, |
|
| Back to top |
|
 |
| wschutz |
| Posted: Mon Mar 06, 2006 9:43 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
MQ itself doesn't do anything with "Backout Threshold" and "Backout Requeue name", if you look at the doc for those fields its says:
| Quote: |
Apart from maintaining a value for this parameter, the queue manager takes no action based on its value. |
Its up to the "applcation" to use those fields (the Broker is an application in this sense)...
Maybe there's something in weblogic that you can set to pay attention to those fields..... 
_________________
-wayne |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Mar 06, 2006 9:51 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| wschutz wrote: |
| MQ itself doesn't do anything with "Backout Threshold" and "Backout Requeue name" |
Correct, if you restrict the definition of "MQ" to the queue manager and MQI libraries. If you're talking about the MQ JMS classes, some use is made of backout-requeueing. Search the Using Java manual for "poison message". |
|
| Back to top |
|
|
| wschutz |
| Posted: Mon Mar 06, 2006 10:08 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
fair enough.... so unless the "ConnectionConsumer" interface is being used, the requeue doesn't happen.... so does that make the question: is there a way to make weblogic use the ConnectionConsumer interface....
_________________
-wayne |
|
| Back to top |
|
|
| pankajthakkar |
| Posted: Mon Mar 06, 2006 10:10 am Post subject: |
|
|
Newbie
Joined: 26 Aug 2005
Posts: 5
|
When using Weblogic JMS, there are options to set the redelivery limit and redelivery delay on the JMS queue.
However, Weblogic Foreign JMS server configuration does not provide much configuration. The console only asks to provide the location of the bindings file, remote and local queue JNDI names. No other configuration is available.
The BOTHRESH did not work even for MQ JMS. We tried the below test case.
A malformed MQRFH2 message is received on a MQ queue. There is a weblogic internal JMSMessagePoller thread that polls the MQ queue for any new messages. In this test case, The malformed MQ message is then handed over to the MQ JMS api (com.ibm.mq.mqjms.jar) to transform it a JMS Message. The MQ JMS api fails because of a parsing error with the below exception.
####<Feb 21, 2006 4:29:55 PM EST> <Error> <EJB> <seigipas01> <Backend_managedServer_1> <ExecuteThread: '1' for queue: 'weblogic.ejb20.internal.JMSMessagePoller'> <<WLS Kernel>> <> <BEA-010079> <An error occurred while attempting to receive a message from JMS for processing by a message-driven bean: [EJB:010196]'javax.jms.MessageFormatException: MQJMS1050: The MQRFH2 header has an incorrect format' Linked exception = 'java.util.NoSuchElementException'
The exception is : javax.jms.MessageFormatException: MQJMS1050: The MQRFH2 header has an incorrect format
at com.ibm.jms.JMSMessage.newMessageFormatException(JMSMessage.java:4716)
at com.ibm.jms.JMSMessage._parseUsrFolder(JMSMessage.java:3203)
at com.ibm.mq.jms.MQJMSMessage.createJMSMessage(MQJMSMessage.java:487)
at com.ibm.mq.jms.MQMessageConsumer.getMessageQ(MQMessageConsumer.java:1660)
at com.ibm.mq.jms.MQMessageConsumer.getMessage(MQMessageConsumer.java:3860)
at com.ibm.mq.jms.MQMessageConsumer.receiveInternalQ(MQMessageConsumer.java:2303)
at com.ibm.mq.jms.MQMessageConsumer.receiveQ(MQMessageConsumer.java:2131)
at com.ibm.mq.jms.MQMessageConsumer.receive(MQMessageConsumer.java:3802)
at weblogic.deployment.jms.WrappedMessageConsumer.receive(WrappedMessageConsumer.java:191)
at weblogic.ejb20.internal.JMSMessagePoller.processOneMessage(JMSMessagePoller.java:281)
at weblogic.ejb20.internal.JMSMessagePoller.pollContinuously(JMSMessagePoller.java:334)
at weblogic.ejb20.internal.JMSMessagePoller.pollForParent(JMSMessagePoller.java:450)
at weblogic.ejb20.internal.JMSMessagePoller.execute(JMSMessagePoller.java:466)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)
In the above case, the message was being rolled back infinitely to the queue and backout count on the MQ queue just kept on increasing. Also, since the message failed at the MQ JMS driver level and never made it to the MDB thus leaving no chance for the application to handle this issue.
"Mq Using java" document mentions about poison message and asks to configure the BOTHRESH and BOQNAME on the queue. Howver it does not mention if MQ JMS driver is equipped to make use of the threshold values set NOR mentions (other than programmatic) any other configurations settings. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Mar 06, 2006 8:44 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Remember a "poison message" in this sense is not a malformed message.
Your message happens to have an invalid RFH2. Well JMS has enough tools to create the RFH for you and as such expects it to be present and valid or absent. This is clearly a case for fixing the sender.
Now if you have a valid RFH you can always check the JMSDeliveryCount.
This would be the number of times the message has b een delivered to the app. If the count is greater than the number of retries for the MDB just ask that the MDB put the message to the back-out queue of if not defined the DLQ.
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mvic |
| Posted: Tue Mar 07, 2006 2:01 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| wschutz wrote: |
| fair enough.... so unless the "ConnectionConsumer" interface is being used, the requeue doesn't happen.... |
The same logic is applied to MessageConsumers too, in both JMS and XMS.
| Quote: |
| so does that make the question: is there a way to make weblogic use the ConnectionConsumer interface.... |
I think no, since MessageConsumers use the same logic.
Actually, if he original problem description is true:
| Quote: |
| even though it is past the "backout threshold" limit, the message is not being transferred to the "Backout Requeue" |
then this warrants a call to Support, in my view.
pankajthakkar:
* Ensure the backoutRequeueQ is (a) created, (b) authorized for the userid to put to (c) not full (d) not put-inhibited.
* If you want a further level of protection, ensure a DLQ is created, and all the above checks have been done for the DLQ too.
* What CSD is installed? |
|
| Back to top |
|
|
| mvic |
| Posted: Tue Mar 07, 2006 2:09 am Post subject: Re: Websphere MQ with Weblogic 8.0 - BOTHRESH not working |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| pankajthakkar wrote: |
| We are using Weblogic 8.0 sp4 to connect to Websphere MQ v 5.3 using the "Foreign JMS Server" connectivity of Weblogic. |
What MQ patch level are you at? The mqver command should tell you. |
|
| Back to top |
|
|
| pankajthakkar |
| Posted: Tue Mar 07, 2006 6:52 am Post subject: |
|
|
Newbie
Joined: 26 Aug 2005
Posts: 5
|
| mvic wrote: |
Ensure the backoutRequeueQ is (a) created, (b) authorized for the userid to put to (c) not full (d) not put-inhibited.
|
We created the backoutRequeue Q. It is empty and has both put and get enabled. We tried with 3 different local queues as the backout Requeue Q.
| mvic wrote: |
| What CSD is installed? |
CSD10 has been applied.
| mvic wrote: |
| What MQ patch level are you at? The mqver command should tell you. |
Below are the details from mqver command:
Name: WebSphere MQ
Version: 530.10 CSD10
CMVC level: p530-10-L050419
BuildType: IKAP - (Production)
This test (of getting BOTHRESH anbd BOQNAME to work) was performed on both Solaris and Win XP.
Thanks, |
|
| Back to top |
|
|
| mvic |
| Posted: Tue Mar 07, 2006 7:03 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
OK I am out of ideas now. My best advice is to collect
* output from dis ql(MYAPPQUEUE) all ... to prove the backoutRequeueQ settings are right on the app queue
* output from dis ql(MYBACKOUTQUEUE) all ... to prove the settings are right on the backoutQ
* detailed trace from JMS and the queue manager, showing the problem occurring - ie. the backoutThreshold being reached, and no requeue actions being taken
and call Support.
(you might wish to get up to the latest CSD first if at all possible, it makes life easier when calling for support from IBM or any vendor for that matter...) |
|
| Back to top |
|
|
| pankajthakkar |
| Posted: Tue Mar 07, 2006 7:23 am Post subject: |
|
|
Newbie
Joined: 26 Aug 2005
Posts: 5
|
| fjb_saper wrote: |
Remember a "poison message" in this sense is not a malformed message.
Your message happens to have an invalid RFH2. Well JMS has enough tools to create the RFH for you and as such expects it to be present and valid or absent. This is clearly a case for fixing the sender.
|
Well, I think a poison message is anything that could cause an application to tie up its resources processing it.
There could be external senders and while it is expected that MQRFH2 if present is valid for messages from external senders, however it cannot be assumed by the receiving/consuming application. If a malicious message (intentionally or unintentionally) makes over to the queue, the consumer application should be able to handle the situation.
This was just an example, that MQRFH2 can be malformed (and hence causing an issue) and cause an error for a JMS client even before its gets the message. While we are on MQRFH2, in somes cases it can be so malformed that it is even not possible to issue an MQGET on the queue for that message. Other than manual intervention (to delete/move the message off the queue), if there is no automated way to control this issue, then I think its a limitation that should be researched upon.
| mvic wrote: |
| OK I am out of ideas now. My best advice is to collect .... and call Support |
We have a PMR into IBM for this issue. Hopefully, we don't get adviced on buying out products such as MQ integrator or broker that use the BOTHRESH and BOQNAME attributes. 
Thank you all for your advice and support so far. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Mar 07, 2006 7:47 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Please do post the results of that PMR...
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| pankajthakkar |
| Posted: Mon Mar 20, 2006 11:17 am Post subject: |
|
|
Newbie
Joined: 26 Aug 2005
Posts: 5
|
| PeterPotkay wrote: |
| Please do post the results of that PMR... |
For those interested in the resolution of this problem posted in this thread.
After providing all the necessary traces and logs to IBM, it was identified to be an issue/limitation (the way you look at it), with MQ-JMS driver (MQ classes for Java aka MQ client) shipped along with MQ 5.x or MQ clients downloaded to work with MQ 5.x (For instance MA88 etc).
"This problem in this thread is addressed under the APAR IY59124 which is fixed in v6,
and the changes too large too port down into the 5.3 release.
If the customer moves to a v6 client, he should find the problem resolved
providing requeuing based off the backout threshold."
It is generally not advisable to mix code from different releases, hence it is recommended to upgrade to MQ 6.0
We executed the same test cases using Websphere MQ 6.0 client as well as server and it seems to behave well so far.
  |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Mar 20, 2006 12:11 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| pankajthakkar wrote: |
| It is generally not advisable to mix code from different releases, hence it is recommended to upgrade to MQ 6.0 |
You can use a 6.0 client with a 5.3 server. It's a supported combination. |
|
| Back to top |
|
|
|
|
