| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Network question |
« View previous topic :: View next topic » |
| Author |
Message
|
| PeterPotkay |
 Posted: Mon Aug 15, 2011 2:54 pm Post subject: Network question |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
I'm assuming that a gigantic 500GB file will be moved from Server A to Server B 10 times faster over a 10Gbit network versus a 1Gbit network.
Why? Is it because the 'pipe' is 10 times wider and 10 times as much data can go at the same time? Or is it 10 times faster?
Is it
A. a 10 lane highway with a 65 MPH speed limit versus a 1 lane highway with a 65 MPH
or
B. both are one lane highways, but one has a 65 MPH speed limit and the other has a 650 MPH speed limit?
(Humor me here and don't tell me to just try it (I wish I could) or just ask my network team. This is going somewhere.)
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Mon Aug 15, 2011 5:21 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Bandwidth is to network traffic as pipe-diameter is to water. That is, each defines maximum carrying capacity. More water through a 4" diameter pipe than a 1" inch diameter pipe.
Network speed (per unit of time) is to network traffic as water-pump horsepower is to water flow. More horsepower pushes more water down the pipe. Network speed is the maximum rate at which the end equipment NICs can successfully send/receive.
Choice A. presumes multiplexing (more than one water pipe, and a method of splitting the water across the multiple pipes). Or in network speak, 10 NIC cards, 10 IP stacks, which packet goes where in the 500G file? On the other hand, it is common for fiber channels (lanes) to carry many separate (distinct) network flows from different sources. Again, this is different from one app, one 500G file.
Sure, a 10-lane highway can carry more car traffic than a 1-lane highway. More lanes, more cars. Increase the speed-limit, and even more car traffic in a given time period. But, an single application to move one 500G file cannot easily split packets across 10 NICs, and reassemble then at the destination Cars are not related to each other, as TCP packets would be.
I like choice B for its simplicity. But throughput involves bandwidth, speed, latency, and other incidentals.
With luck, someone will be along shortly with better analogies.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Aug 15, 2011 8:09 pm Post subject: Re: Network question |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| PeterPotkay wrote: |
I'm assuming that a gigantic 500GB file will be moved from Server A to Server B 10 times faster over a 10Gbit network versus a 1Gbit network.
Why? Is it because the 'pipe' is 10 times wider and 10 times as much data can go at the same time? Or is it 10 times faster?
Is it
A. a 10 lane highway with a 65 MPH speed limit versus a 1 lane highway with a 65 MPH
or
B. both are one lane highways, but one has a 65 MPH speed limit and the other has a 650 MPH speed limit?
(Humor me here and don't tell me to just try it (I wish I could) or just ask my network team. This is going somewhere.) |
I don't like particularly either analogy...
Let's look at the 1GB network vs the 10GB network
Is it because you have 10 times the bandwidth or is it because you have 10 times the speed? It is probably a combination of both.
Now think about other limiting factors like
- potential network collisions
- number of control blocks needed with 10 GB vs 1 GB
- network noise
- network devices (10 GB cards only on a physical media that is good for 100 GB allows for excellent quality and possibly some multiplexing down the path if you have the correct nic cards (100 GB) for some part of the path...
- network quality (number of resends needed because the packet could not be correctly read at the other end...
All this means that sending something over 10 GB network through a 1 GB pipe might still be significantly faster than sending it through a 1GB network: less collision, interference and noise because of the increased bandwidth can mean a significant number of transport success on the 10 GB pipe that would have to be resent on the 1 GB pipe.... and yet it does not mean that you used up more bandwidth (nic cards)... just that it was available to you...
This goes to quality of the network...
The more you approach bandwidth saturation, the more you experience network quality degradation.
An easy example is: try sending anything through an MQ channel when an FTP sucks up all the available bandwidth on a stretch of the route...
Your network quality goes from 100 to 0 in no time flat... because as long as the ftp is running you will have a hard time getting a single packet in edgewise...
This is why some network devices can limit the available bandwidth per protocol... or port.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Aug 16, 2011 4:46 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
fjb_saper is correct. "korek" in Filipino. There is a law of diminishing returns. The network line speed is only one factor in the equation. And if you are going any distance at all, the latency for ack packets are increasing exponentially. Granted for short hops, the relative ack latency is small. But over one meter, ack latency is X, two meters, 2X, 10 meters 10X. If your traffic path happens to go over your WAN, that would be the extreme limiting factor since WANs typically don't support an even upstream line speed compared to downstream.
For example, if you get your home broadband from a cable provider, your downstream line speed is typically ten times faster than your upstream line speed. When Cox cable advertises 10 MBps line speed, they are referring to the cumulative line speed between upstream and downstream. Most Cox cable installations will see 768 kilobits per second upstream and 9.23 megabits downstream.
Note that line speed is most often expressed in bits per second. Not bytes per second. It takes 7, 8, 9 or 10 bits to make a byte, depending on your MAC-level ethernet switch configuration with regard to handshake protocol and parity bit setting. So 10 Gigabits per second may only by 1 Gigabyte per second line speed, less TCP/IP protocol overhead, making the throughput closer to 800 megabytes per second.
http://www.2wire.com has a bandwidth meter to measure your bandwidth.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Aug 16, 2011 5:06 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Also a Gigabyte is 1073741824 (1024^3 or 2^30) bytes. Therefore, what you perceive as a billion bytes throughput is actually less than that because there are more bytes per billion than 1,000,000,000 bytes.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Aug 16, 2011 5:11 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| lancelotlinc wrote: |
| Also a Gigabyte is 1073741824 (1024^3 or 2^30) bytes. |
If only it were that simple. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Aug 16, 2011 5:36 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| mqjeff wrote: |
| lancelotlinc wrote: |
| Also a Gigabyte is 1073741824 (1024^3 or 2^30) bytes. |
If only it were that simple. |
That wiki page lists a "youtube" byte: yobibyte (YiB) 2^80
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Aug 16, 2011 5:40 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| lancelotlinc wrote: |
| fjb_saper is correct. "korek" in Filipino. There is a law of diminishing returns. The network line speed is only one factor in the equation. And if you are going any distance at all, the latency for ack packets are increasing exponentially. |
Yep, distance is an issue; but in defense of latency...
As you might imagine, mechanical latency is very high with soup-can and string networks. Bandwidth in such networks is extremely limited. Electrical latency with copper wire networks is much lower, and bandwidth is higher.
With fiber optic (light-transmission) networks, latency is extremely low; and bandwidth (carrying capacity) is extremely high. A pretty good wiki on fiber optic networks: http://en.wikipedia.org/wiki/Fiber-optic_communication#Bandwidth-distance_product
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Aug 18, 2011 5:17 pm Post subject: Re: Network question |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| fjb_saper wrote: |
| PeterPotkay wrote: |
I'm assuming that a gigantic 500GB file will be moved from Server A to Server B 10 times faster over a 10Gbit network versus a 1Gbit network.
Why? Is it because the 'pipe' is 10 times wider and 10 times as much data can go at the same time? Or is it 10 times faster?
Is it
A. a 10 lane highway with a 65 MPH speed limit versus a 1 lane highway with a 65 MPH
or
B. both are one lane highways, but one has a 65 MPH speed limit and the other has a 650 MPH speed limit?
(Humor me here and don't tell me to just try it (I wish I could) or just ask my network team. This is going somewhere.) |
I don't like particularly either analogy...
Let's look at the 1GB network vs the 10GB network
Is it because you have 10 times the bandwidth or is it because you have 10 times the speed? It is probably a combination of both.
|
If you move that 500GB file over a 10Gbit network roughly 10x faster than a 1Gbit network, and that's the end of it, yes, who cares if its 10x faster or 10x "fatter", the total data gets there faster.
Now lets add into the discussion the app that sends little 1K chunks, frequently, sporadically. Yes, I know the network portion of this is probably minuscule compared to the rest of what the app does. But lets say its really, really frequent, and for the purpose of this discussion the resource that this app is so chatty with has to be on a separate server.
Right or wrong people are saying the "network latency" of having the 2 pieces separate is impacting performance. Will moving to a 10 GB network "solve" it? One camp says yes (the ones that say 10Gbit is faster). The other camp says no (the camp that says 10Bit has 10x more bandwidth, and the fatter pipe makes no diff for this little chunks of data because the overhead of all the switches and cards and routers is the same). That first group says the line speed is 10x faster, the tcp/ip packets are 1500 bytes no matter what and they always travel serially, not 10 of them side by side, so 10Bit really is 10x faster.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Aug 18, 2011 8:24 pm Post subject: Re: Network question |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| PeterPotkay wrote: |
If you move that 500GB file over a 10Gbit network roughly 10x faster than a 1Gbit network, and that's the end of it, yes, who cares if its 10x faster or 10x "fatter", the total data gets there faster.
Now lets add into the discussion the app that sends little 1K chunks, frequently, sporadically. Yes, I know the network portion of this is probably minuscule compared to the rest of what the app does. But lets say its really, really frequent, and for the purpose of this discussion the resource that this app is so chatty with has to be on a separate server.
Right or wrong people are saying the "network latency" of having the 2 pieces separate is impacting performance. Will moving to a 10 GB network "solve" it? One camp says yes (the ones that say 10Gbit is faster). The other camp says no (the camp that says 10Bit has 10x more bandwidth, and the fatter pipe makes no diff for this little chunks of data because the overhead of all the switches and cards and routers is the same). That first group says the line speed is 10x faster, the tcp/ip packets are 1500 bytes no matter what and they always travel serially, not 10 of them side by side, so 10Bit really is 10x faster. |
Ok ... so let's look @ the network.
If the 2 apps / servers reside on the same subnet and talk IP to IP the network latency is at its minimum.
If the apps talk hostname to hostname, the network latency can be "augmented" by the time it takes to resolve the hostname to the IP.
This may include a round trip to your preferred DNS server which may well reside on a different subnet... and have a network path with more latency and less bandwidth... However remember that some of that would be cached in the tcp layer.... or in /etc/hosts ... or...
Now if the network needs to hop over multiple subnets etc... its top speed will be that of its slowest link...
You can try and evaluate network latency by using ping and if there is a significant slowdown use trace route to research where the problem might be. Most of the time significant delays occur because of poor routing information... and you need to work this out with your network team.
This said, having the 2 pieces on 2 separate servers will always be slower than having them on the same server...
Does that mean that the impact will be perceptible? Not necessarily.
It depends on how chatty the apps are.
Seriously what kind of bandwidth are they using? What kind of latency is expected?
If a super low latency is expected, you may not have the network hardware to run them from 2 different servers..., or may need to upgrade your Nic Cards and network to fiber optics...
The question then shifts from a bandwidth problem to a latency problem and may have absolutely nothing to do with the bandwidth.
I am sure your network engineers can provide you with average latency, and standard deviation, for the different types of networks that you are looking at.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Aug 19, 2011 4:19 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Aug 19, 2011 5:25 am Post subject: Re: Network question |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| PeterPotkay wrote: |
| ...who cares if its 10x faster or 10x "fatter"... |
I thought the op was a good question.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
