| Author |
Message
|
| zzha99 |
 Posted: Thu Jul 30, 2009 12:16 am Post subject: How to commit with Multithreaded client |
 |
|
Newbie
Joined: 29 Jul 2009
Posts: 2
|
In order to speed up data polling from server, we programmed two threads to access the same queue, and each thread will commit once if it has successfully downloaded 100 messages and saved them to a file. My problem here is if one thread fails to save messages, and another thread has committed, we may not able to re-download the messages failed by the first thread.
Is here any good way to deal with such problem?
P.S, we have to use multi-threaded client to keep up with the server.
Thanks in advance. |
|
| Back to top |
|
 |
| AkankshA |
| Posted: Thu Jul 30, 2009 12:27 am Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
what were the factors taken into consideration while deciding the number 100 for a commit ??? why not commit each message ?
_________________
Cheers |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Thu Jul 30, 2009 1:10 am Post subject: Re: How to commit with Multithreaded client |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| zzha99 wrote: |
we may not able to re-download the messages failed by the first thread.
|
Wont they be in a backout queue? You could then process them one at a time from this queue until the "bad" message is encountered when its finally moved to the error queue (assuming the error was due to a bad input message rather than a transient failure in which case all the message could be processed). |
|
| Back to top |
|
|
| zpat |
| Posted: Thu Jul 30, 2009 1:33 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Each thread will have a unique connection handle to the queue manager and commits will be separate.
Unless you implement handle sharing (which I suggest you don't).
If one thread backs out messages, they will become available to the other thread to get.
Move to a poison (backout re-queue) queue only if the message backout count exceeds a certain value (backout re-queue threshold) to prevent a loop.
These values can be held in the queue definition and inquired on. |
|
| Back to top |
|
|
| zzha99 |
| Posted: Thu Jul 30, 2009 6:00 pm Post subject: |
|
|
Newbie
Joined: 29 Jul 2009
Posts: 2
|
Thanks for your replies.
| Quote: |
what were the factors taken into consideration while deciding the number 100 for a commit ??? why not commit each message ?
|
Commit each message made us not able to catch up the server.
| Quote: |
Each thread will have a unique connection handle to the queue manager and commits will be separate.
If one thread backs out messages, they will become available to the other thread to get.
|
If this is true, I really don't know why we lost data. That was happened when a blackout hit our city and we lost a few hundred messages during power outage. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jul 31, 2009 6:38 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zzha99 wrote: |
| f this is true, I really don't know why we lost data. That was happened when a blackout hit our city and we lost a few hundred messages during power outage. |
The message were probably set as non-persistent. These message don't survive things like recovery from a power outage induced queue manager crash.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
