| Author |
Message
|
| Ravee Bhat |
 Posted: Wed Aug 06, 2008 3:38 am Post subject: need help regarding broker performance |
 |
|
Apprentice
Joined: 28 Apr 2004
Posts: 28
Location: Bangalore, INDIA
|
As part of the building connectivity between an external and few of our Development environments we are routing the incoming messages through the WMB (6.0), We are planning store the destination queue names in a database table, this is on order to provide configurability regarding which systems should receive the messages and which should not.
Everytime a message flows in, the Compute node in the msg flow will read the destination queues from database table and send the messages to these queues by setting them to the destination data.
Our concern is that, will this cause any substantial performance impact? (During a time of the day the queue might get a 100+megs file (as segmented messages)).
Is there anyway we can reduce the database table hits during this? Is there any mechanism for caching of such data in the message flows?
Also what would be the broker behavior in case of non availability of data base?
thanks in advance
Ravee
ps: hope i am not overshooting my newbie limits for something simple already available  |
|
| Back to top |
|
 |
| AkankshA |
| Posted: Wed Aug 06, 2008 3:53 am Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
|
| Back to top |
|
|
| Ravee Bhat |
| Posted: Wed Aug 06, 2008 4:19 am Post subject: |
|
|
Apprentice
Joined: 28 Apr 2004
Posts: 28
Location: Bangalore, INDIA
|
Thanks for the pointer.
cheers
Ravee
ps: i did overshoot, there is a sample message flow in the toolkit help for this  |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Wed Aug 06, 2008 4:22 am Post subject: Re: need help regarding broker performance |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
| Ravee Bhat wrote: |
Also what would be the broker behavior in case of non availability of data base?
|
Then Gone, all your business will be stuck.
use broker database for keeping this details is also an option
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Aug 06, 2008 4:50 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Sounds like you should use a broker pub/sub design for this.
Using a database (ideally on the broker platform) will not cause much overhead (especially compared to 100 MB messages!).
DB/2 will cache the data in memory anyway, if it is set up properly. |
|
| Back to top |
|
|
| Ravee Bhat |
| Posted: Wed Aug 06, 2008 5:01 am Post subject: |
|
|
Apprentice
Joined: 28 Apr 2004
Posts: 28
Location: Bangalore, INDIA
|
| zpat wrote: |
| Using a database (ideally on the broker platform) will not cause much overhead (especially compared to 100 MB messages!). |
Do you mean for a set of segmented messages reading of table is done only once!? i.e. on the first mesg!
| zpat wrote: |
DB/2 will cache the data in memory anyway, if it is set up properly. |
Hard luck! there is a oracle thing already in place! |
|
| Back to top |
|
|
| sridhsri |
| Posted: Wed Aug 06, 2008 5:04 am Post subject: |
|
|
Master
Joined: 19 Jun 2008
Posts: 297
|
| Have you considered using 'in-memory' tables ( You could persist the same in a database too ). Typically your flow could load required data from a database table first time (reload will require a flow restart) and store it in cache. Every other time, your flow refers to the table in memory. This way even if the DB is down the flow could continue. |
|
| Back to top |
|
|
| zpat |
| Posted: Wed Aug 06, 2008 7:03 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
All database products will cache frequently accessed tables into memory (unless you try very hard to set them up badly). The key to database performance is memory usage (just look at any TPC benchmarks!).
Don't worry about the performance. Make sure the database is as available as the broker (eg use the broker database). |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Aug 06, 2008 7:15 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
You can also store this info as MQ messages in an MQ queue, and use the MQGET node to browse the hold queue.
I don't know how this will compare in performance to using a DB.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Ravee Bhat |
| Posted: Thu Aug 07, 2008 3:32 am Post subject: |
|
|
Apprentice
Joined: 28 Apr 2004
Posts: 28
Location: Bangalore, INDIA
|
Thanks so much for all the pointers
Yes, databases do cache the frequently used objects. here my concern is more towards the number of times i am going to ask the database to give me something.
As of now decided to settle down for a simple solution like the one given in the samples gallery. Perhaps when I get more time (expecting a phase 2 of development here) will be able to try the options.
cheers,
Ravee |
|
| Back to top |
|
|
| LazyBoy |
| Posted: Thu Aug 07, 2008 12:34 pm Post subject: |
|
|
Voyager
Joined: 04 May 2006
Posts: 78
|
|
| Back to top |
|
|
| zpat |
| Posted: Thu Aug 07, 2008 11:27 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Have a look at the IBM WMB samples in the toolkit.
| Quote: |
The Message Routing sample shows how a database table can be used to store routing information that a message flow can use to route messages to WebSphere MQ queues. This is an alternative approach to creating hard-coded routing information in a message flow compute node.
Using a database table is the recommended way in WebSphere Message Broker to store any form of routing or transformation data for the following reasons:
It removes business data from the ESQL code in the message flow
It allows for updates in the routing data without message flows having to be changed and redeployed
It makes ESQL easier to read by removing the need to have large IF ... THEN ... ELSE code structures
It allows routing information to be stored in one place but used by many different message flows (and possibly other applications)
The one problem with using databases to store this information is the performance overhead that is incurred to access the database each time a message passes through a message flow.
Before version 6.0 of WebSphere Message Broker, a design decision had to be made between hard coding routing or mapping information in ESQL, or using a database table to store the information. The decision would often be based on whether the performance cost of using the database was acceptable. For high message rates, it was often decided to create hard-coded information that would be easier to maintain in a database table.
In version 6.0 the concept of shared variables was introduced. With shared variables you can store state in message flows that is available to all messages using the flow. The exact details of how shared variables are used can be found in DECLARE variables. Using shared variables, database tables can be stored in-memory in a message flow. This removes the need for continued access to the database table, and gives almost all the advantages of using a database table without the performance overhead. Only one advantage is missing: the cache will not pick up changes to the database table as they happen. To pick the changes up, either you need to restarted the message flow or you need to add a refresh mechanism to the message flow.
The Message Routing sample shows how to use a database table to route messages in a message flow, and how a message flow can use shared variables to store the database table. It also shows how the cached database table can be refreshed by restarting the message flow or by sending a refresh message to the message flow.
|
|
|
| Back to top |
|
|
| vaibhav_vy |
| Posted: Thu Aug 07, 2008 11:39 pm Post subject: |
|
|
Apprentice
Joined: 04 Aug 2008
Posts: 28
|
| You can use CacheConfig node to load all destination queue details in memory at execution group level from the database tables. All your message flows can then either ESQL code or CacheGet node to read this in-memory cache. This requires only one time loading of data from database into memory. Obviously, this cache wont survive execution group restart or broker restart. This functionality wont be available by default. You have to download & configure IA91 fixpack to do this according to your broker version. |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Aug 08, 2008 12:04 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Don't bother unless the message volume is fairly high. If it's less than one per second it's probably not necessary. |
|
| Back to top |
|
|
|
|
