| Author |
Message
|
| neo_revolution |
 Posted: Tue Jun 01, 2004 4:38 am Post subject: optimizing msgflow processing |
 |
|
Voyager
Joined: 21 Oct 2003
Posts: 80
Location: NJ
|
Hello all,
We are getting msgs in xml. We are expecting higher volume of around 25000 msgs per day...
The problem we have is that,
-> for each incoming msg we need to check a flag in the database,
* if the flag is set msg flow routes the message to a queue,
* if it is not set msg flow routes to a hold queue which are processed as soon as db flag is set...
We have to check the db flag for each incoming msg as WMQI doesn't have memory to store the state...
With the above setup the processing time is quite a bit more.
Is there a better design to handle above situation?
Thanks. |
|
| Back to top |
|
 |
| Missam |
| Posted: Tue Jun 01, 2004 5:50 am Post subject: |
|
|
Chevalier
Joined: 16 Oct 2003
Posts: 424
|
| Quote: |
The problem we have is that,
-> for each incoming msg we need to check a flag in the database,
|
Can you skip the database operation here and store the values in localEnvironment. |
|
| Back to top |
|
|
| neo_revolution |
| Posted: Tue Jun 01, 2004 6:45 am Post subject: |
|
|
Voyager
Joined: 21 Oct 2003
Posts: 80
Location: NJ
|
It it would be great if we skip the db access.
But how can we store information in the LocalEnvironment?
As soon as I process next message I will be loosing the LocalEnvironment tree, right? |
|
| Back to top |
|
|
| Missam |
| Posted: Tue Jun 01, 2004 7:07 am Post subject: |
|
|
Chevalier
Joined: 16 Oct 2003
Posts: 424
|
| Please look into ROW and LIST constructors in ESQL reference.you will get an idea how to do this. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Jun 01, 2004 8:21 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I don't see how ROW and LIST constructors will help him add state to his message flow.
WMQI is not a stateful processing environment. You can use a database to hold state, or a queue, or a custom node that interacts with something else to hold state.
If the performance of your message flow, including your database select, is too slow, then you need to look at either speeding up the database lookup, siwtching to some other persistance mechanism, or increasing the efficiency of your message flow.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| neo_revolution |
| Posted: Tue Jun 01, 2004 8:34 am Post subject: |
|
|
Voyager
Joined: 21 Oct 2003
Posts: 80
Location: NJ
|
Thats right Jeff. DB query is not an issue as the table is small, its the number of queries that matters.
With the present infrastructure we can't implement Pub/Sub immediately.
Actually what we are doing is, when the db flag is set we want to stop processing remaining msgs, either by stopping msg flow. To achieve this we are putting msgs onto a Hold queue and a trigger message is passed when db flag is set so that a job can copy msgs from hold queue to the input queue.
I am trying to get a better design so we can do this in a efficient manner.
Thanks |
|
| Back to top |
|
|
| fazz |
| Posted: Tue Jun 01, 2004 8:48 am Post subject: |
|
|
Centurion
Joined: 20 Feb 2004
Posts: 144
Location: England
|
Theres a Cache node avaliable as a support pac (IA91)
Dosen't work on all platforms though.......  |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Jun 01, 2004 8:48 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Create a separate flow, whose job it is to create PCF messages that Get-Disable the main queue.
Send a disable message to the new flow when you want to stop processing, and an enable message to the new flow when you want to start processing.
Then your main input queue becomes your "Hold" queue.
_________________
I am *not* the model of the modern major general.
Last edited by jefflowrey on Tue Jun 01, 2004 9:23 am; edited 1 time in total |
|
| Back to top |
|
|
| neo_revolution |
| Posted: Tue Jun 01, 2004 9:07 am Post subject: |
|
|
Voyager
Joined: 21 Oct 2003
Posts: 80
Location: NJ
|
| But I read in the forum that this is not supported by IBM !!! |
|
| Back to top |
|
|
| fazz |
| Posted: Tue Jun 01, 2004 9:15 am Post subject: |
|
|
Centurion
Joined: 20 Feb 2004
Posts: 144
Location: England
|
Well......No. 
Think Jeffs on to more of a winner. |
|
| Back to top |
|
|
|
|
