| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Performance issue due to bottleneck JDBC Adapter |
« View previous topic :: View next topic » |
| Author |
Message
|
| varshasvin |
 Posted: Wed Feb 27, 2008 11:40 pm Post subject: Performance issue due to bottleneck JDBC Adapter |
 |
|
Newbie
Joined: 27 Feb 2008
Posts: 9
|
Hi ,
I think you were facing problem of bottleneck in JDBC adapter due to huge data coming from database.
I am also facing same problem. In my case the data is really huge, its about 120,000+ documents and we r finding a way to process this data in 1 hr.
In our case JDBC will fetch the data from database and will create the xml file for corresponding key.
Although we have brought down the processing time to around 6 hrs, still we need to find a way to process 120,000+ documents in 1 hr.
JDBC adapter is a bottleneck in the processing time.
We are trying to see if ETL can directly create XML and load into staging table in CLOB format. This should bring down the processing time even more.
Also, since they would be creating XMLs, we can load chunks of data at a time in the staging table. I believe this way the processing would be faster since adapter does not have to search into 120,000 header records.
Is this correct way for this problem or do you have any other solution?
please reply me as soon as possible |
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Thu Feb 28, 2008 12:14 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
Hi
As you said is correct,
How many Adapter instances are running in this case.
if you haven't established diff instances of adapter, you can think of this option
Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| varshasvin |
| Posted: Thu Feb 28, 2008 1:01 am Post subject: |
|
|
Newbie
Joined: 27 Feb 2008
Posts: 9
|
Thnanks,
In this case only one instance of adapter is running |
|
| Back to top |
|
|
| varshasvin |
| Posted: Thu Feb 28, 2008 1:18 am Post subject: |
|
|
Newbie
Joined: 27 Feb 2008
Posts: 9
|
Can I solve this issue using multiple instances of adapter?
If yes then , how can i do it? |
|
| Back to top |
|
|
| Gaya3 |
| Posted: Thu Feb 28, 2008 1:19 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
Hi
Try out of having multiple of instances which helps to poll the database.
Note: it will be required to bounce the adapter once in every day (usually in the non peak hours)
Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
