| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Database Question=>Process huge no of records |
« View previous topic :: View next topic » |
| Author |
Message
|
| broker_new |
 Posted: Tue Jul 14, 2009 5:06 pm Post subject: Database Question=>Process huge no of records |
 |
|
Yatiri
Joined: 30 Nov 2006
Posts: 614
Location: Washington DC
|
Hi Guys,
I have a requirement, have to query the database(oracle) and it returns 100,000 messages in one select statement. We knew that it consumes a lot of memory in processing them.. do you guys suggest me in doing this in a better way to save the resources.
_________________
IBM ->Let's build a smarter planet |
|
| Back to top |
|
 |
| jbanoop |
| Posted: Tue Jul 14, 2009 6:02 pm Post subject: |
|
|
Chevalier
Joined: 17 Sep 2005
Posts: 401
Location: SC
|
I would re-visit the requirement.
One option would be to use propagate and try to split the query into sections with the transaction mode on the MQOutput node set to 'no', but that would bring with it other issues I guess ... |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Tue Jul 14, 2009 8:57 pm Post subject: Re: Database Question=>Process huge no of records |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
| broker_new wrote: |
Hi Guys,
I have a requirement, have to query the database(oracle) and it returns 100,000 messages in one select statement. We knew that it consumes a lot of memory in processing them.. do you guys suggest me in doing this in a better way to save the resources. |
are you strictly looking to implement the same in ESB (i mean MB)
if its a huge size or batch and having some huge transactions, could think of ETL over here. this is just a suggestion or thought.
If you are using MB as the solution:
you have to keep this as a separate flow and in separate EG. Just to make it independent and keep the huge batch processing apart
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| broker_new |
| Posted: Wed Jul 15, 2009 7:10 am Post subject: |
|
|
Yatiri
Joined: 30 Nov 2006
Posts: 614
Location: Washington DC
|
Definitely it is a ETL process, i accept it.
Here we have some transformations and routing the data to several other systems.
Nothing is striking in my mind at this time  |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
