| Author |
Message
|
| atheek |
 Posted: Sat Sep 12, 2009 12:17 pm Post subject: |
 |
|
Partisan
Joined: 01 Jun 2006
Posts: 327
Location: Sydney
|
| Quote: |
upon receiving message from queue 1 move to queue 2 and record following in the DB :
original msgid, correlid, characteristic; processed status, current msgid (q2)
have a process that reads the DB (index it) by processed status and characteristic. It will retrieve the current msgid and build the selector to use this on Q2. It needs to lock the current record during processing and update the status when done.Depending on how you lock your db / characteristics you can even allow concurrency of processing for different characteristics.. |
.
Is this db write really required? If I can use a header/property that is indexed, I can use the selector directly on Q2. In MQ V6.XX, I can use correlation id field for this purpose in ID:xx format. In MQ 7.xx I can use any custom property as I understand that v7.0 supports indexing for custom properties as well. I can hash the orderID field to get the value for selector key. The hashes generated by the hashing algorithm should be
known in advance for setting the Message selectors on Q2. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Sat Sep 12, 2009 7:43 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| atheek wrote: |
| Quote: |
| Not necessarily. Your assumption (that you did not state) is that it takes the same amount of time to process each of the colors (G,B,R). |
OK. I didn't state it, but unit of order concept is used generally for similar type messages which require some ordered processing within a group. So the processing time for each type is roughly same. Steffi's message affinity requirement was not for different types of orders, If I understand correctly. It was for similar type orders, but the required ordered processing was for messages with same orderID. We had a similar requirement for flight schedule update messages. We used unit of order in Weblogic JMS for guaranteeing ordered processing of messages for a particular flight on a particular day. |
But that would require for the sender to group the messages by say producing all messages with the same order id in the same group and using default open fixed for the group if in a cluster.
You could then only retrieve messages from the same group... by using correctly your selectors...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| atheek |
| Posted: Sat Sep 12, 2009 8:26 pm Post subject: |
|
|
Partisan
Joined: 01 Jun 2006
Posts: 327
Location: Sydney
|
| fjb_saper wrote: |
| But that would require for the sender to group the messages by say producing all messages with the same order id in the same group and using default open fixed for the group if in a cluster. |
Yes. This is what unit of order does out of the box in a clustered weblogic JMS environment. It ensures that all messages belonging to the same Unit of order is send to the same JMS destination in the cluster. The downside with this approach there is no automatic failover possible; if the JMS Destination is down then all mesages intended for that gets blocked. BEA endorses this fact. |
|
| Back to top |
|
|
| steffi2 |
| Posted: Mon Sep 14, 2009 7:53 am Post subject: |
|
|
Apprentice
Joined: 04 Sep 2009
Posts: 31
|
I continue my expermentation but for now I'm trying to use Temporary queues. the idea being that I have one listener for the main queue and async receive each of those messages I dispatch them based on their order id to their appropriate temporary queue. All these temporary queues are being created off the same connection. If I want to create N temporary queues that async message using each N's listener. can I use the same connection to create those temporary queues?
i want to see N + 1 async delivery threads but I'm not currently seeing that.
Eventually I will replace the temporary queues with static queues because I need to support recovery (ie. acknowledgement of the N'th queue only when I know I've processed a message)
The idea for now is if I can create temporary queues that gives me more flexibility for testing just to see what the optimal bucket size would be.
main queue
1 connection CA
1 session SA
1 queue QA
1 receiver RA
bucket queues
1 session SB using connection CA
N temporary queue T's using SB as the session
N receivers R's
will this have two async delivery threads or only one?
do I have to create either N sessions using connection CA to have
independent async consumption for my N bucket queues?
Presumably when I move to a model that has N static queues I still only need one connection and not N+1 connections? |
|
| Back to top |
|
|
| mvic |
| Posted: Mon Sep 14, 2009 8:10 am Post subject: Re: When to acknowledge when using mulitple threads? |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| steffi2 wrote: |
| acknowledge the message immediately after the details have been written to a working table |
acknowledgements - these are per HConn, not per message with MQ. (Also they're not really known as acknowledgements under the covers, but units of work). Does this affect your per-message acknowledgement architecture at all? To which part of the wider system were you thinking of acknowledging receipt of each message? Hope this helps |
|
| Back to top |
|
|
| steffi2 |
| Posted: Mon Sep 14, 2009 10:35 am Post subject: Re: When to acknowledge when using mulitple threads? |
|
|
Apprentice
Joined: 04 Sep 2009
Posts: 31
|
well I only want to acknowledge a message from my listener when I know that that message was handled/processed successfully. If i'm dispatching the message to another queue for processing I want to make sure if there's an error (say resource limit error on the write to the queue) that any subsequent acknowledged message doesn't acknowledge the message that originally caused me the error. I think that's not possible and whenever I see a send to queue operation fail I must record ie. persist the record and replay it later. I don't think I have the ability to only acknowledge the message that failed and no others as a side affect. If a message fails because it wouldn't write to the queue I guess it should be moved to a dead letter queue rather than just left on the queue. In this case my persistent store of the message achieves that purpose.
| mvic wrote: |
| steffi2 wrote: |
| acknowledge the message immediately after the details have been written to a working table |
acknowledgements - these are per HConn, not per message with MQ. (Also they're not really known as acknowledgements under the covers, but units of work). Does this affect your per-message acknowledgement architecture at all? To which part of the wider system were you thinking of acknowledging receipt of each message? Hope this helps |
|
|
| Back to top |
|
|
|
|
