| Author |
Message
|
| abhyyy |
 Posted: Tue Jul 16, 2013 11:10 pm Post subject: Get 100 MB data from MSSQL and transfer using Web-service |
 |
|
Voyager
Joined: 29 Sep 2011
Posts: 83
|
Hi Friends,
One of my clients has a requirement where he wants WMB to fetch 100MB data from MSSQL server 2012 twice a day at designated off peak time and then wants that data to be transferred via web-service.
I have the flexibility of breaking the data over 10 chunks of 10 MB each and then send across as 10 responses.
Can anybody give me the best fit or best recommended way to do this.
Can be creating WS and returning attachment over MTOM or any other.
The only way client dooesn't want to use it is by using a file system.
Would really appreciate you advise on this. Thanks in advance. 
_________________
----------------------
NeVeR StOp LeaRnInG. |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Jul 16, 2013 11:50 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
What does it mean "fetch 100mb of data from SQL Server"?
Is it "fetch one row that contains a 100mb BLOB field"?
Is it "fetch all rows in a single select that will usually amount to 100MB of data"?
Does it mean "read a file created from a batch offload process from SQL Server, that is expected to be 100mb in size, and represents lots of rows"?
In general, dealing with this amount of data is better handled in a streaming approach, rather than as large in-memory chunks. How that can be handled in Broker depends entirely on how the data is read and how the data is written afterwards. |
|
| Back to top |
|
|
| abhyyy |
| Posted: Wed Jul 17, 2013 1:23 am Post subject: |
|
|
Voyager
Joined: 29 Sep 2011
Posts: 83
|
Thanks for your reply and Sorry for not being clear.
Actually, upstream will request data using a webservice and then WMB will select all the rows of one or another table and retrieve from MS SQL server using 1 sql select statement, since the data is very big so it will come around 80MB to 100 MB.
After retrieving all the rows (100 MB data) it has to sent across to upstream using web-service response.
What approach do you suggest ?. Thanks
_________________
----------------------
NeVeR StOp LeaRnInG. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jul 17, 2013 1:24 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
You're going to have a hard time sending 100MB over a single HTTP call.
The approach I suggest is to review the business requirements, and consider using an alternate transport. |
|
| Back to top |
|
|
| abhyyy |
| Posted: Wed Jul 17, 2013 1:29 am Post subject: |
|
|
Voyager
Joined: 29 Sep 2011
Posts: 83
|
Is there a way in which this 100 MB data can be split in 100 chunks of 1 MB data and then sent using a web-service , may be using MTOM??
Please advise if there is any other way I can do it using web service as the client wants it to done using web-service only. Many Thanks
_________________
----------------------
NeVeR StOp LeaRnInG. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Wed Jul 17, 2013 3:28 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| someyears |
| Posted: Thu Jul 18, 2013 3:56 pm Post subject: |
|
|
Newbie
Joined: 17 Jul 2013
Posts: 5
|
just come in and  what a crazy idea to do 100M webservice call.
haha
you know, architect! |
|
| Back to top |
|
|
|
|
