MQSeries.net :: View topic

kaitlyn · Posted: Wed Nov 09, 2005 9:05 am Post subject:

We have encountered a really strange thing happening. We have some queues set up that have a failure queue associated with them (no, this isn't the strange thing). As an example, we have a queue called A and then we have a queue called AFailure and this allows us to know which failure queues are associated with which queue, just by the name. What we do is to set the AFailure as the backout queue and only have the A queue try to process 1 time, then on a failure it will go to the AFailure queue (via MQ). This process seems to work quite nicely for us as this PREVENTS the listeners from EVER shutting down, yet still allows processing to continue for those that can. With that being said, we also have some failure queues that we actually PUT messages into WITHOUT letting MQ put them there (this is done via Java). The A queue gets and xml formatted message (called a batch) that has 1 to many records to process. As this example goes, I will tell you that the message in A queue has three records. The first one processes with and error, so the Java code creates a mini-batch and puts ONLY that 1 record into the AFailure queue, then processes record 2. Two passes. Then it processes record 3 and it fails. It will then create a mini-batch and put that into the AFailure queue. So, with the first message (batch), there were three records to process, one processed successfully, and the other two failed, so they got put into AFailure queue by Java, not by MQ. The exception for those two failures is caught and this allows that first message off the A queue to go away and allow it to move on to the next batch.

Now that all of that is out of the way, I noticed something strange this morning. There were three mini-batches created and put on the queue at 9:44 pm last night (as evidenced in our logs on the Java side). When I looked this morning, they were NOT on the AFailure queue. I was asked to re-process (move from the AFailure queue to the A queue) three messages. At that time, that dropped the count on the AFailure queue from 593 to 590. Everything good so far. Since it sat at 590, I thought that the three I ran back through had processed successfully (this was all done at 6:55 am this morning). I went and checked the logs and it indicated that the three messages I tried to process errored out and were (via Java) put onto the Failure queue. I found this strange as the count still said 590. I refreshed my list and it still said 590. I double checked the log and saw I wasn't still asleep and then went back and refreshed my mq monitor and noticed that it finally jumped up to 593. Everything good, right? NO! The 3 messages it put on there were from 9:44 LAST NIGHT!

What the heck is going on? I now can't see the 3 messages that happened at 6:55 am this morning! And why would the 3 that failed 6 hours beforehand just now show up in the queue? I did confirm the message limit was 5000, so that can't be it. Any ideas? Sorry this message is so long, but I figured you needed to understand what was happening in order to help me figure out what the problem is.