| Best option for resubmitting messages |
| Rfhutil - using failure queue |
|
14% |
[ 1 ] |
| Small MQ application - using failure queue |
|
57% |
[ 4 ] |
| Script - using database |
|
28% |
[ 2 ] |
|
| Total Votes : 7 |
|
| Author |
Message
|
| nickkirk |
 Posted: Mon Sep 29, 2008 12:55 am Post subject: Resubmitting failed messages |
 |
|
Apprentice
Joined: 26 Sep 2008
Posts: 48
|
What solutions are there for resubmitting failed messages ?
1) Use a queue for failed messages and have the support team take the messages off the queue with rfhutil and resend them. Using rfhutil, the message id of the replayed message could stay the same.
2) Use a database. Have a script replay the messages from the database. The message id of the replayed message could stay the same.
3) Write a small MQ app to take the messages off the 'failed' queue and resend them with the same message ID.
The reason why using the same msgId is important is for the following reason :
An error handling subflow sends an email detailing the failed message to the support staff.
If the resubmitted message fails again using a different msgId, this could be very confusing. The support staff may believe that several different messages have failed when actually only the one did.
I am considering the database option.
Advantages:
-A persistent store will allow analysis on the failed messages.
-No user errors as the extraction/resubmission process is automated with scripts
-Will be more secure (i.e. users can't change the data prior to resubmitting a message).
Disadvantages:
-Resource (if many failed messages).
-Provision of scripts to do the data extraction and resubmission of failed messages. |
|
| Back to top |
|
 |
| Maximreality |
| Posted: Mon Sep 29, 2008 5:56 am Post subject: Re: Resubmitting failed messages |
|
|
Acolyte
Joined: 04 Jun 2004
Posts: 65
Location: Copenhagen
|
What if let's say 10000 messages fail, will the support staff then get 10000 e-mails ?
btw i voted the db option...., but i would build some sort of app. that can support the staff in resubmitting the messages instead of using handmade scripts.
When making error handling with db i suggest you also use a seperate messageflow or program to insert the failed messages into the DB, don't use a subflow. |
|
| Back to top |
|
|
| fschofer |
| Posted: Mon Sep 29, 2008 6:22 am Post subject: |
|
|
Knight
Joined: 02 Jul 2001
Posts: 524
Location: Mainz, Germany
|
|
| Back to top |
|
|
| nickkirk |
| Posted: Mon Sep 29, 2008 7:14 am Post subject: |
|
|
Apprentice
Joined: 26 Sep 2008
Posts: 48
|
I forgot to mention:
The order of messages is important (i.e. messages must not be processed within the broker out of order ). |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Sep 29, 2008 10:02 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| nickkirk wrote: |
I forgot to mention:
The order of messages is important (i.e. messages must not be processed within the broker out of order ). |
Why the  is everybody so keen on using a design with message affinity...
Imagine you have 10,000 messages that go to the resend queue... Does that mean you stop the message flow 10,000 times and manually resolve the problem??
Seriously review the design and get rid of the message affinity.   
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Maximreality |
| Posted: Tue Sep 30, 2008 2:27 am Post subject: |
|
|
Acolyte
Joined: 04 Jun 2004
Posts: 65
Location: Copenhagen
|
| nickkirk wrote: |
I forgot to mention:
The order of messages is important (i.e. messages must not be processed within the broker out of order ). |
It can be done, but who the hell came up with that requirement?? and is the business ready to take the possible impact when something goes wrong?
Many business analysts will come up with silly requirements like message affinity, when you ask them why they won't give you a good reason....
/Rasmus |
|
| Back to top |
|
|
| hopsala |
| Posted: Mon Oct 06, 2008 8:57 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| nickkirk wrote: |
I forgot to mention:
The order of messages is important (i.e. messages must not be processed within the broker out of order ). |
Will message affinity never die? Will we always be plagued with this %#@$%# design scheme? Next time someone starts a sentence with "Oh yea, I forgot to mention" you'll find me on the other side of the building, standing on my head, wearing earplugs, a blindfold, and shouting "Guatanamo Bay! Guatanamo Bay!" at the top of my voice for no apparent reason.
(and did I mention that FJB is the emoticon master?)
That being said - if you want message affinity, DB is really your only option for resending. You must have some way to say "resend complete message sequence 10" which includes, say, 30 messages, and this is best achieved using databases. |
|
| Back to top |
|
|
| nickkirk |
| Posted: Tue Oct 07, 2008 12:29 pm Post subject: |
|
|
Apprentice
Joined: 26 Sep 2008
Posts: 48
|
350 views so far and only one intelligent reply ? WOW !
Does no-one have anything sensible to say?
As for my requirements; Having messages processed in order is very important for this project.
Maybe this is not the case for your project. However, from the stupid comments people are making, I doubt these people have used the broker outside of their bedrooms.
Please, only contribute if you have had some real experience ! |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Oct 07, 2008 1:35 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Wow indeed. You come off as a supremely intelligent being. Surprised you bother asking us morons of our opinion.
If you are breaking one of the fundamental best practices of MQ, fine, that is sometimes a business requirement. State that up front, explain why, and avoid having people telling you what you already know, so we can get to the root of your problem. In case you haven't noticed a lot of people new to MQ like to do things that experience has shown is not such a hot idea - message affinity, inquire the q depth and then do that many MQGETs, treat message IDs as text data, etc. How were we troglodytes to know you invented MQ and were just testing us with your question?
Besides, this is a free, public forum. Certainly you must expect some posts that aren't properly formatted and suited for a white paper on the topic to be distributed to leading CIOs. Cheer up! 
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| dkeister |
| Posted: Wed Oct 08, 2008 5:42 am Post subject: |
|
|
Disciple
Joined: 25 Mar 2002
Posts: 184
Location: Purchase, New York
|
Without getting in to the affinity issue, and as I don't believe I saw much about the real requirement...
I wrote the following capabilities in MQExerciser. I would appreciate your feedback to see if any of these options are of interest. Assuming the messages are on one queue and you want to get them to another queue, you can:
1. Use the copy function to copy just the message body, the message body and MQMD, or the message body/MQMD but generate a new MsgID. This leaves the original messages on the queue.
2. Use move which works like copy with the same options, but the messages are removed from the queue.
3. Save the messages with or without the MQMD and then at some later point, reference to the folder with the messages and have them placed on a queue with the same options as copy.
None of these options get into the issue of why you might need to do this but it can be done.
_________________
Dean Keister |
|
| Back to top |
|
|
| hopsala |
| Posted: Wed Oct 08, 2008 11:26 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| Quote: |
350 views so far and only one intelligent reply ? WOW !
Does no-one have anything sensible to say?
... However, from the stupid comments people are making, I doubt these people have used the broker outside of their bedrooms. |
I'm assuming mine was not that "one intelligent reply"  No worries, I'm not going to get into a "my experience is bigger than yours" argument. Suffice to say you've just insulted some of the more experienced people in the business (myself excluded), and more specifically, the people you just asked to help you. Besides, why use insults at all?
Anyway, it is experience which has repeatedly taught us that message affinity is just plain BAD, and can usually be avoided. I have had about a dozen client projects which began with a demand for message affinity, only two were stubborn enough to keep that demand after my attempts to dissuade them. This was three years ago, and these two projects have three times as many production problems, and code change cycles are increased twofold due to the added complexity.
That said, there are indeed cases in which message affinity is necessary and adequate. Is this the case? Let's assume that it is. If you're interested to hear more about why affinity is bad, search the forums or open another thread, and we'll be happy to discuss it.
As for the topic at hand, could you tell us a little more about your project? what's the throughput? are we talking request-reply or datagram style? are these merely queries or change-commits? Without more info, it's rather difficult to answer your question. The database way is obviously the robust way, but in small or request-reply projects, it may be redundant or not worth the effort.
Peace! |
|
| Back to top |
|
|
|
|
