| Author |
Message
|
| PRKUMAR |
 Posted: Wed Sep 05, 2012 2:28 am Post subject: how to implement temporary storage of messages? |
 |
|
Apprentice
Joined: 04 Sep 2012
Posts: 36
|
Hi,
We have a requirement that, an application puts 100 messages/sec in a queue. We need to put each incoming message in one particular database table. In the meanwhile, if any problem occurs(database down or queue manager down or problem with incoming data) the current processing message must be store temporarily. The processing of the message will continue only after the issue is resolved and then it should get updated in the database. The time for resolving issues may take 1hr, 2hrs or max 24hrs. Until the current message would successfully got updated in the database, no other messages would get updated in database. It means based on the order of receiving messages in queue we need to update it in database. How best we can achieve this requirement? |
|
| Back to top |
|
 |
| McueMart |
| Posted: Wed Sep 05, 2012 3:03 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
When an error is detected in your application, put the 'bad' message to an error queue, and use the CMP API to get-disable your input queue (which will stop your application processing any other messages).
Have an offline process for looking at the error queue, fixing the message, and re-putting it back onto the input queue. Then you can get-enable your input queue.
Thats one idea. I bet there are 1000's more alternatives! |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Sep 05, 2012 3:49 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Here is a much better idea.
Rewrite the process so you ARE NOT dependent on the order of messages on a queue.
With your solution as it is, you can merely NOT set any BOTHRESH on the queue that has the messages, and ONLY deploy a single instance of the flow (additional instances 0).
Then a bad message will be INFINITELY retried and NO more messages AFTER it will be processed if there's an error.
It will cause a lot of messages in the system log until you manually halt the flow, or use the CMP API to halt the flow. |
|
| Back to top |
|
|
| PRKUMAR |
| Posted: Wed Sep 05, 2012 6:20 am Post subject: how to implement temporary storage of messages? |
|
|
Apprentice
Joined: 04 Sep 2012
Posts: 36
|
Hello,
I am not heard about CMP API. Can anyone provide some information regarding this?
We can halt the flow, but here we need to consider two points:
1. Till the issue on the current message gets resolved, they don't want to keep the messages in input queue(since message rate is 100 messages per second). If it will take 24hrs to resolve the issue on the current message or broker/database outage then thousands of messages will pile up in the queue. It could be a problem. So they want to store in some temporary database(or any other solution).
2. Once the issue(bad message or database down or broker unavailable) is solved then message flow should re-process the same message where normal processing gets halted. Because the order in which the messages arrive in the input queue should be the same order when they should get updated in the database.
How to handle these two points for better designing of the flow? Please help me. |
|
| Back to top |
|
|
| nathanw |
| Posted: Wed Sep 05, 2012 6:29 am Post subject: |
|
|
Knight
Joined: 14 Jul 2004
Posts: 550
|
Sounds to me like you need to look at your processes rather than the flow.
Is there a reason why you cannot process any more messages until the issue of a single(?) failing message is resolved?
I have worked in environments like yours and the simple answer is to capture the failing message and continue and then replay the failed message when solution is made.
What happens if one of the messages being held up is critical but it takes 24 hours to resolve your error.
I am assuming that the business using this is not a business critical function because if my company did not process anything for up to 24 hours because of a single error message I would be asking WHY?
_________________
Who is General Failure and why is he reading my hard drive?
Artificial Intelligence stands no chance against Natural Stupidity.
Only the User Trace Speaks The Truth  |
|
| Back to top |
|
|
| McueMart |
| Posted: Wed Sep 05, 2012 6:31 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
Storing the backlog of messages in a queue is a very good idea. Why would you thing its a bad one??
100 messages per second = 6000 per minute = 360000 per hour = 8640000 per day
I see no technical reason why MQ couldnt just store these messages on the input queue for you (Obviously you'll need to ensure you have enough disk space but thats would be the case if you stored them in a DB as well!).
The beauty of the queue is that the messages are already in order for you when you start to process them again!
You are paying lots of money for a high speed, reliable messaging platform - why would you suddenly want to store things in a database?? |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Sep 05, 2012 6:32 am Post subject: Re: how to implement temporary storage of messages? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PRKUMAR wrote: |
| I am not heard about CMP API. Can anyone provide some information regarding this? |
Look it up in the documentation. It's the programmatic method of controlling WMB resources.
| PRKUMAR wrote: |
| How to handle these two points for better designing of the flow? |
1. You can store the messages anywhere you want, but consider that if the reason you can't add the messages to the target table is because the database server is down, then adding them to a temporary database is also likely to fail. What's wrong with messages piling up in the queue, and why does the sending application have to keep running for the full 24 hours? Why can't it be halted when the outage is detected?
2. Having messages processed in order like this is called message affinity. If you want a better design, eliminate this as my most worthy associate indicated. It's a bad design and I would question what the requirement is. I suspect you're being told to process them in order because when this data used to be in a file it was processed in sequence (obviously) and the client is still in that mind set. See exactly how many messages really need to be in sequence (e.g. all the transaction for a given customer might need to be in order) and design to preserve only that.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Sep 05, 2012 6:43 am Post subject: Re: how to implement temporary storage of messages? |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| PRKUMAR wrote: |
Hello,
I am not heard about CMP API. |
Then you need to go learn. Good Luck, Happy Reading!
| PRKUMAR wrote: |
| Can anyone provide some information regarding this? |
Yes, IBM's website. In the documentation on the product you are using.
Good Luck! Happy reading!
| PRKUMAR wrote: |
We can halt the flow, but here we need to consider two points:
1. Till the issue on the current message gets resolved, they don't want to keep the messages in input queue |
That's a BAD idea. KEEP them on the input queue. KEEPING them on the queue is a GOOD idea.
As my colleague says - anything that removes the message from the queue is no more reliable than the correct destination for the message. So if you want to guarantee that you can process the message, DON'T remove it from the queue.
| PRKUMAR wrote: |
| If it will take 24hrs to resolve the issue on the current message or broker/database outage then thousands of messages will pile up in the queue. It could be a problem. |
It's only a PROBLEM if you don't PLAN for it.
| PRKUMAR wrote: |
| Because the order in which the messages arrive in the input queue should be the same order when they should get updated in the database. |
This is a BAD IDEA.
CHANGE THIS.
DO NOT rely on the order of messages on the queue.
Change the nature of the data being sent such that it does not represent a CHANGE to the current state (insert, update, or delte) but instead represents the WHOLE current state.
Or otherwise make sure that the messages themselves contain an index that identifies the position in the overall sequence, and use that to ensure that your messages are processed in sequence. |
|
| Back to top |
|
|
| PRKUMAR |
| Posted: Wed Sep 05, 2012 7:08 am Post subject: how to implement temporary storage of messages? |
|
|
Apprentice
Joined: 04 Sep 2012
Posts: 36
|
Thanks for all your good suggestions.
It is just only their view that storing all the messages in a temporary database. We can make them convince to keep as it is in queue also. Finally they don't want loose their messages.
Here is my another point:
1. Suppose if the issue is with the message, then we make available this message to production support people. So they can correct the problem(correct the message). But, how this updated message(correct message) again available to the message flow so it should be processed first.
Please suggest me how to do this. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Sep 05, 2012 7:27 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
that's one of the problems with relying on the order of messages on the queue.
You can't then process the last message as the first message, even if it needs to be. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Sep 05, 2012 7:46 am Post subject: Re: how to implement temporary storage of messages? |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PRKUMAR wrote: |
| Please suggest me how to do this. |
Change the design to eliminate message affinity.
The scenario you describe here is one of the many reasons we keep telling you to eliminate it. It causes all sorts of problems.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| McueMart |
| Posted: Wed Sep 05, 2012 8:53 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
| Quote: |
| But, how this updated message(correct message) again available to the message flow so it should be processed first. |
How about your flow has a second MQ Input node which uses a different input queue which is used only for the 'fixed' messages? You could also do it with message priority if you were so inclined. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Sep 05, 2012 9:19 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| McueMart wrote: |
| Quote: |
| But, how this updated message(correct message) again available to the message flow so it should be processed first. |
How about your flow has a second MQ Input node which uses a different input queue which is used only for the 'fixed' messages? |
You'd still be left with the problem that the "fixed" message has to be processed before all of the others in the "waiting" queue. You couldn't be sure that the flow wouldn't pick up from the original MQinput node first rather than this second one.
| McueMart wrote: |
| You could also do it with message priority if you were so inclined. |
You could, but you'd notice a performance impact with the kind of depths the OP is talking about as the queue manager looks for the highest priority message.
Removing the affinity removes all of these problems, and also allows this flow servicing 100 mps to scale.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| McueMart |
| Posted: Thu Sep 06, 2012 12:39 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
| Quote: |
| You'd still be left with the problem that the "fixed" message has to be processed before all of the others in the "waiting" queue. You couldn't be sure that the flow wouldn't pick up from the original MQinput node first rather than this second one. |
By coordinating the get-inhibitedness of the input queues you could. You would need to enforce a strict process when processing these 'fixed' messages though. |
|
| Back to top |
|
|
| PRKUMAR |
| Posted: Thu Sep 06, 2012 4:50 am Post subject: how to implement temporary storage of messages? |
|
|
Apprentice
Joined: 04 Sep 2012
Posts: 36
|
Hello,
The client want that, if any issue or outage happens the current processing should halt and all the messages in the input queue needs to be store in temporary database(they don't want to keep it in the queue). Once the issue got resolved(corrected the bad data that causes the issue or database is up) they want to resume the processing of the messages as the order in which they arrived in the queue. It means whatever the messages are stored in temporary database should process first and then we have to allow the messages in the input queue.
What is the best approach in order to achieve this? |
|
| Back to top |
|
|
|
|
