| Author |
Message
|
| offshore |
 Posted: Wed Nov 24, 2004 8:15 am Post subject: Bad Message |
 |
|
Master
Joined: 20 Jun 2002
Posts: 222
|
All,
I wasn't sure what topic this fell under, so I threw it in the miscellanous discussion.
How does your company (Acme Corp) handle bad messges?
I've tried to pitch this before that something needs in place to handle/detect bad message data, well all the sudden there is a recommendation to do just that.
Background Info:
This is a batch mq get/read on a z-series machine.
Currently the job performs a destructive get on 'x' number of messages on a queue and writes out to a file. Well, if the job encounters a bad message it blows up.
I suggested doing a browse - write - destructive get. The question I have now, [and I think it should] should the program (if written correctly) be able to detect the bad message and move it to another queue that holds bad messages. |
|
| Back to top |
|
 |
| clindsey |
| Posted: Wed Nov 24, 2004 8:40 am Post subject: |
|
|
Knight
Joined: 12 Jul 2002
Posts: 586
Location: Dallas, Tx
|
In a recent implementation, I get messages in syncpoint. If any exception occurs, the message gets rolled back.
With each get, I check the backout count. If it exceeds the threshold (which I set to 3), I move the message to an exception queue. Our message design allows for a place to put exception information with the message. You could do this with rfh headers or your own proprietary msg structure.
You can set the backout threshold (BOTHRESH) and exception queue (BOQNAME) in the queue object so you can retrieve with MQINQ calls or make it available as application instance data.
This is one approach, for what is it worth.
Charlie
Last edited by clindsey on Thu Nov 25, 2004 4:51 pm; edited 1 time in total |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Nov 24, 2004 5:08 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
If possible I would suggest that you treat the bad message as binary data.
Add a dead letter queue header on top of it and stuff it either in a special queue or in the dead letter queue.
The advantage of the DLQ header is that you can then move the bad messages around, resubmit them, forward them etc... using the dead letter queue handler.
Enjoy  |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Nov 24, 2004 8:17 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
What reason code would you put in the DLH?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Nov 25, 2004 1:57 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| PeterPotkay wrote: |
| What reason code would you put in the DLH? |
The java WAS catch all we use with MDB's : Backout count exceeded (2063 ?).
But if you really want to make it fancy you could use your pick of the MQRC messages...
Enjoy  |
|
| Back to top |
|
|
| EddieA |
| Posted: Fri Nov 26, 2004 11:11 am Post subject: |
|
|
Jedi
Joined: 28 Jun 2001
Posts: 2453
Location: Los Angeles
|
| Quote: |
| What reason code would you put in the DLH? |
Well, you could start here:
| Quote: |
MQFB_APPL_FIRST
Lowest value for application-generated feedback.
MQFB_APPL_FIRST 65536 X'00010000'
|
Cheers,
_________________
Eddie Atherton
IBM Certified Solution Developer - WebSphere Message Broker V6.1
IBM Certified Solution Developer - WebSphere Message Broker V7.0 |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Nov 26, 2004 2:57 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
My question was meant to make one think, "Is the DLQ the best place for application's garbage messages?"
What is a Dead Letter Handler gonna do with a Backout Count Exceeded RC, or worse yet, some application genereated feed back code?
Applications should put unwanted messages into application specific Exception Queues, and leave the system DLQ for use by the system.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Nov 26, 2004 5:13 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Peter,
The DLQ is not necessarily the right queue. However the advantages of the DLQ header are there, even in an application error queue.
We have all the errors from our WAS going to the DLQ. They are few and far between. We have experienced even fewer system related error messages. We use some utility programs for removing the messages (after analysis) depending on their content. But we sometimes like to redirect onto an application error parking queue depending on the destination queue....
For us the hasle of one backout queue per application queue is not worth the trouble. Our DLQ and application error parking queue are monitored.
The DLQ handler will work fine on any queue (unix env) as long as the messages have a DLQ header, regardless of whether the queue they are on is the qmgr's DLQ or a regular queue.
Enjoy
|
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun Nov 28, 2004 12:54 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Unless you are extremely familiar with every single app in your system, how could you or a dead letter handler know what to do with all those bogus application messages in the DLQ?
I know what to do with queue missing errors, or queue full errors, or queue inhibited errors, but how can I (the human dead letter handler) or the dead letter handler I code the rules for possibly know what reason an app threw a message off? If all you or your handler is gonna do in these cases is move the message to some other queue, why not just have the app put the bad messages there in the first place?
DLQs should be empty, and if there is anything in there, I get paged. If an app can't process a message, because its DB is down, or the message has a period intead of an _, or the age field has ABC, how could I possibly debug that? There are dozens and dozens of apps, each with dozens and dozens of reasons that they could see a message as "bad".
Much better that an app puts a bad message to an app specific Exception queue, and have that queue monitored to page the *application area* when a message goes there. This is not an MQ error, so why should the MQ admins be bothered? When a new app comes on board, you gotta build their queues anyway. It only takes 10 seconds more to build them one extra Exception queue.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Nov 28, 2004 8:01 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I completely agree with you Peter.
What I did not specify is that in any case, regardless of the error, the MQ admin (i.e. me) gets to look at the messages and try to find a path to resolve them. Twice a day we distribute a list of queues to management where the current depth is not 0 (monitoring stuff).
For political reasons the SYSTEM.DEAD.LETTER.QUEUE gets much more attention from management than anything else.
As we do not use clustering the messages going to the DLQ are far and few between. The applications have up to 15 or 20 queues but any bad messages go to the DLQ first and get parked unto an application error queue if the resolution will take longer. I did not want to set up another 15 to 20 backout queues, that nobody would look at anyway. The application of origin is known by the generic queue name. XXX.YYY.ZZZ etc... XXX is the application of origin.
Using a DLQ header allows me to resubmit whatever was in the DLQ or APP error queue once the problem (application) has been resolved.
Talking about production you should not have to change message contents.
In a non production environment nobody cares about errors, whether apps or MQ related and if I want to make sure that the apps work before I get hit in prod, I'd rather see the errors before the coding that produced them makes it to prod....
So yes my answer is a little bit biased but not without reason.
Just trying to apply the KISS principle.
Thanks for putting up with my ranting.
F.J. |
|
| Back to top |
|
|
| Nigelg |
| Posted: Mon Nov 29, 2004 1:15 am Post subject: |
|
|
Grand Master
Joined: 02 Aug 2004
Posts: 1046
|
| Quote: |
| As we do not use clustering the messages going to the DLQ are far and few between. |
I am a little puzzled by this.
Is it your experience that in clusteing msgs are put to the DLQ more often than if clustering is not used?
What are the reasons for this? |
|
| Back to top |
|
|
| clindsey |
| Posted: Mon Nov 29, 2004 6:57 am Post subject: |
|
|
Knight
Joined: 12 Jul 2002
Posts: 586
Location: Dallas, Tx
|
| Quote: |
I did not want to set up another 15 to 20 backout queues
|
This is not really necessary. I also have around 20 application queues but the exceptions are all directed to a single exception (backout) queue. Since some require manual investigation, I didn't want them on the DLQ. The nice think about WebSphere MQ is that there is always more than one way to fulfill a requirement. I think that is why Offshore started this thread. Now he has a couple of approaches to consider.
Charlie |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Nov 29, 2004 12:17 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| clindsey wrote: |
| but the exceptions are all directed to a single exception (backout) queue |
Charley, this is not the way it works with Websphere WAS.
The JMS setup of the MDB allows you to reject after x (bakout threshold) attempts.
If you have a backout queue specified the message gets put onto the backout queue with not additional header.
If you do not specify a backout queue the message goes to the DLQ with DLQ header.
Hence the possibility to determine where the message was originally sent to without having to analyse its content and keep different sources easily separated all on 1 queue...
Thanks
F.J. |
|
| Back to top |
|
|
|
|
