| Author |
Message
|
| woodoo2k |
 Posted: Sun Mar 05, 2006 10:38 pm Post subject: Error 2082 received by channel when receiving message. |
 |
|
Apprentice
Joined: 07 Feb 2005
Posts: 28
Location: USA
|
Hi All,
I have a very well known and talked about problem at hand.
I have standard bridge configuration that connects queues from one cluster CL1 to another cluster CL2 involving 3 qmgrs. Q1 is full repo for CL1 and Q2 for CL2. Q3 is partial repo for CL2. Message needs to travel from Q3 to Q1 via Alias Queue definitions shared in clusters.
Local Queue LQ1 is hosted in Q1 and shared in CL1 via alias definition QA1. Bind is specified as NOTFIXED. The clustered definition of QA1 is visible in Q2 and another alias QAA1 is created in Q2 which is in turn shared in CL2. Here again, the Bind is set as NOTFIXED.
When I am trying to send message from Q3 to Q1, it ends up in SYSTEM.DEAD.LETTER.QUEUE on Q2 with 2082 error.
Apparantly there is nothing amiss as this definition was working till yesterday.
Today this error occurred for the first time and messages went to DLQ. The QA1 was not visible in Q2 over CL1. But that problem was fixed and all queues are properly visible now.
Still the same error.
Could somebody please give some hint as to what might be going wrong.
There is no error in mq error / qmgr error logs other than related to message being written to DLQ.
Thanks in advance for your help !
Cheers !
Victor |
|
| Back to top |
|
 |
| Nigelg |
| Posted: Mon Mar 06, 2006 1:37 am Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
| Quote: |
Apparantly there is nothing amiss as this definition was working till yesterday.
|
Something changed in your configuration since yesterday.
It is up to you to find out what.
_________________
MQSeries.net helps those who help themselves.. |
|
| Back to top |
|
|
| wschutz |
| Posted: Mon Mar 06, 2006 1:46 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
| Quote: |
| Today this error occurred for the first time and messages went to DLQ. The QA1 was not visible in Q2 over CL1. But that problem was fixed and all queues are properly visible now. |
Nigel's point is well taken, what exactly did you "fix", and why did you need to fix it?
_________________
-wayne |
|
| Back to top |
|
|
| Ivans |
| Posted: Mon Mar 06, 2006 4:02 am Post subject: |
|
|
Apprentice
Joined: 03 Jan 2006
Posts: 48
Location: Hursley
|
You should check that the application is not overriding the default bind on the queue definition, by specifiying MQOO_BIND_ON_OPEN.
Cheers,
Ian Vanstone
WebSphere MQ |
|
| Back to top |
|
|
| flaufer |
| Posted: Mon Mar 06, 2006 5:04 am Post subject: Re: Error 2082 received by channel when receiving message. |
|
|
Acolyte
Joined: 08 Dec 2004
Posts: 59
|
[quote="woodoo2k"]Hi All,
I have a very well known and talked about problem at hand.
I have standard bridge configuration that connects queues from one cluster CL1 to another cluster CL2 involving 3 qmgrs. Q1 is full repo for CL1 and Q2 for CL2. Q3 is partial repo for CL2. Message needs to travel from Q3 to Q1 via Alias Queue definitions shared in clusters.
Local Queue LQ1 is hosted in Q1 and shared in CL1 via alias definition QA1. Bind is specified as NOTFIXED. The clustered definition of QA1 is visible in Q2 and another alias QAA1 is created in Q2 which is in turn shared in CL2. Here again, the Bind is set as NOTFIXED.
When I am trying to send message from Q3 to Q1, it ends up in SYSTEM.DEAD.LETTER.QUEUE on Q2 with 2082 error.
Apparantly there is nothing amiss as this definition was working till yesterday.
Today this error occurred for the first time and messages went to DLQ. The QA1 was not visible in Q2 over CL1. But that problem was fixed and all queues are properly visible now.
Still the same error.
Could somebody please give some hint as to what might be going wrong.
There is no error in mq error / qmgr error logs other than related to message being written to DLQ.
Thanks in advance for your help !
Cheers !
Victor[/quote]
Some additional information might be helpful (I'm working with Viktor on this problem).
It's WMQ 5.2 CSD03 running AIX 4.3.3 with some late ML on it. Both queue managers (Q1 and Q2) are running on the same machine.
I also can confirm that the objects we are discussing here have not been altered lateley (last alter date was somewhat in 2003). Usually this environment is running very stable and no new implementations are done (actually it is in the process of become outdated).
We also tried to take the cluster queue QA1 out of CL1, refresh the cluster, then put the cluster attribute back. No success.
When looking at the queues and qclusters with runmqsc, everything seems to be fine, however, the messages end up in the DLQ (put application is the listener).
Cheers,
Felix |
|
| Back to top |
|
|
| flaufer |
| Posted: Mon Mar 06, 2006 5:07 am Post subject: |
|
|
Acolyte
Joined: 08 Dec 2004
Posts: 59
|
[quote="Ivans"]You should check that the application is not overriding the default bind on the queue definition, by specifiying MQOO_BIND_ON_OPEN.
Cheers,
Ian Vanstone
WebSphere MQ[/quote]
The application has not changed. (btw. it's running on Q3 - W2K with MQ5.2.0.3)
Felix |
|
| Back to top |
|
|
| flaufer |
| Posted: Mon Mar 06, 2006 5:10 am Post subject: |
|
|
Acolyte
Joined: 08 Dec 2004
Posts: 59
|
[quote="wschutz"][quote]Today this error occurred for the first time and messages went to DLQ. The QA1 was not visible in Q2 over CL1. But that problem was fixed and all queues are properly visible now. [/quote]Nigel's point is well taken, what exactly did you "fix", and why did you need to fix it?[/quote]
Ok...
Last night... I discovered the messages in the DLQ and went looking for the queues with RUNMQSC. No success. Since the qmgr was running for almost 140 days now without interruption, I choose to just restart it (clearing up all leftover IPC stuff as well). After this the cluster queue (QA1) was visible again as I expected it in the first place (again, I was checking with runmqsc). But still, messages for this queue, when coming in from Q3 end up in the DLQ.
Felix |
|
| Back to top |
|
|
| woodoo2k |
| Posted: Mon Mar 06, 2006 5:20 am Post subject: |
|
|
Apprentice
Joined: 07 Feb 2005
Posts: 28
Location: USA
|
Hi Ivans,
The thing is that it is the receiver channel agent which is putting message to DLQ that are meant for QAA1 only. Same channel is processing all other messages meant for other similar queues WITHOUT any problem. So we can rule out problems with either Binding option of Queue as they are ok OR binding of channel which is also ok and common to all the messages flowing over various other queues in this queue manager Q2.
I am guessing if we have to restart the Q2 as well along with Q1 as both are on same m/c ?
We cannot experiment too much as it is live system.  (
Any hints ! |
|
| Back to top |
|
|
| flaufer |
| Posted: Mon Mar 06, 2006 5:58 am Post subject: |
|
|
Acolyte
Joined: 08 Dec 2004
Posts: 59
|
| woodoo2k wrote: |
Hi Ivans,
The thing is that it is the receiver channel agent which is putting message to DLQ that are meant for QAA1 only. Same channel is processing all other messages meant for other similar queues WITHOUT any problem. So we can rule out problems with either Binding option of Queue as they are ok OR binding of channel which is also ok and common to all the messages flowing over various other queues in this queue manager Q2.
I am guessing if we have to restart the Q2 as well along with Q1 as both are on same m/c ?
We cannot experiment too much as it is live system. :-((
Any hints ! |
Again... now the queue is no longer visible with runmqsc and when I look at the repository of Q2 I can't see it also... (amrfdm also tells thehe queue is not there).
When I check with other queue managers in the same cluster who are also full repositories I can see the queue !
Felix |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Mar 06, 2006 4:12 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
PRs only acknowledge the existance of qclusters when someone has asked for them.
Try running amqsput to connect (but don't write any data) to the qcluster on the Q2. Then try dis qcluster(*).
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| flaufer |
| Posted: Tue Mar 07, 2006 2:30 am Post subject: |
|
|
Acolyte
Joined: 08 Dec 2004
Posts: 59
|
| jefflowrey wrote: |
PRs only acknowledge the existance of qclusters when someone has asked for them.
Try running amqsput to connect (but don't write any data) to the qcluster on the Q2. Then try dis qcluster(*). |
We tried. No success. We decided to have a workaround by just creating another pair of queues and redirect the queues to these. This is now working.
But still the root issue is not resolved.
Felix |
|
| Back to top |
|
|
| HubertKleinmanns |
| Posted: Wed Mar 08, 2006 12:09 am Post subject: |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
Felix,
it seems to me, that some cluster information is lost (your MQ version - especially your CSD - is quite old). Try to alter the cluster queue, your alias points to:
- Remove the cluster queue from the cluster, by altering the cluster attribute to a blank.
- Wait some seconds
- Re-add the queue to the cluster, by altering the cluster attribute to the cluster name.
Now the cluster should refresh its repositories.
_________________
Regards
Hubert |
|
| Back to top |
|
|
| flaufer |
| Posted: Wed Mar 08, 2006 1:48 am Post subject: |
|
|
Acolyte
Joined: 08 Dec 2004
Posts: 59
|
| HubertKleinmanns wrote: |
Felix,
it seems to me, that some cluster information is lost (your MQ version - especially your CSD - is quite old). Try to alter the cluster queue, your alias points to:
- Remove the cluster queue from the cluster, by altering the cluster attribute to a blank.
- Wait some seconds
- Re-add the queue to the cluster, by altering the cluster attribute to the cluster name.
Now the cluster should refresh its repositories. |
Hubert,
we did this. No help.
We also
- removed queue from cluster (unshare on FR, where the Q is hosted)
- removed queue from qmgr (FR)
- refresh cluster (from FR)
- wait
- recreate queue from scripts on FR.
- cluster queue visible on all other FRs, but not on the other PR. Tried to access the queue on the one PR that caused the trouble, not help.
- checked in the repository with repository dump on PR, no information about that queue.
channels between the FR and the PR are working, S.C.X.Q empty, S.C.C.Q also empty.
queue managers have been restarted many times now, no success.
To your statement about the "old version" - we know this... an unsupported Application (MQ5.2) on an unsupported OS (AIX 4.3.3) is not what one would expect to be running in a highly productive environment, but unfortunately, here it is. :( but we are heavily migrating towards 5.3 ;-)
Felix
P.S. there hasn't been any deployment during the last months.. so probably the repository update hasn't been working for a while and we just did not notice... probably we will check and see if the repository IS received ANY update at all, but then I would not understand why this suddenly stops without reason. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Mar 08, 2006 4:54 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Clustering became MUCH more stable when 5.3 came out. I do in fact remember bugs specific to Alias queues, although the details escape me.
Since your set up seems OK (at least as far as we can tell from your description), the normal course of action would then be to call IBM for support. But....You're years out of support.
If u connect to a PR to access a clustered q on the FR, and you can't see it, even though all the channels are running, and clustered, then its a bug.
What if you create a brand new local queue on the FR, cluster it, and then connect amqsputc to a PR and try to put to it? Does it work with a brand new queue?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| flaufer |
| Posted: Thu Mar 09, 2006 5:43 am Post subject: |
|
|
Acolyte
Joined: 08 Dec 2004
Posts: 59
|
| PeterPotkay wrote: |
If u connect to a PR to access a clustered q on the FR, and you can't see it, even though all the channels are running, and clustered, then its a bug.
What if you create a brand new local queue on the FR, cluster it, and then connect amqsputc to a PR and try to put to it? Does it work with a brand new queue? |
Peter,
this is the workaround that is currently in place. We've just created another cluster queue (on the FR) and redirected the alias (on the PR) to point to it. This is working. But if we try to remove and recreate the original cluster queue (on the FR), this does not help much, it just won't work.
Since this is really a bit outdated and the environment will be decommissioned soon, we will probably not put more energy into this, but I'm somewhat worried what happens if this comes up in a really productive and important environment so that's wehy I'm still trying to figure out what happened and how to prevent it from happening ;)
Felix |
|
| Back to top |
|
|
|
|
