Commenting on the paper referred to above, I think that Mr Odlyzko overall got his head screwed on right with regard to some of his main points: It's perfectly possible to stream data over TCP and play from the buffer in the majority of cases (Whether or not it has to be quicker than realtime is a whole other ballgame, but it isn't really that relevant - the question is whether it works and is an acceptable solution for watching a movie or clip over the net - it is) - also, I'd be quite inclined to agree that voice carries a much higher revenue per byte transferred than video.
There's a few things I disagree with as well. Content is king, but it's the content that's important to the user that is king, no matter if it's a YouTube clip, a torrent download, World of Warcraft or your local news page. You do have different expectations about delivery times and availability for different content, though - so it's not always about impatience. I also don't agree that the ISP's motivations are truly nefarious (even though I'm guessing I won't be seen as a credible source there if you think that they are.. :-)
Finally, streaming (be it pig streaming over TCP + buffer or real-time streaming), is a bit of a challenge, but not necessarily because of realtime requirements.
Why video is hard
This one is best answered by two words: Sheer volume. No matter how you transfer it, it's a whole lot of data. As faster access gets cheaper, users expect more and more of it too. I regularly click the HD links when watching trailers - that's a cool 132 MB for two minutes eight seconds - or 8.25 Mbps of access speed eaten while I download it if the network's quick enough to support realtime. Your average DVD torrents would clock in at the vicinity of 750 MB each - or several GB for the HD rips out there. Given the choice between DVD and HD, HD is a very popular option indeed. For torrents, you can expect the upload utilization to be about as high as the download one, at least for users who are serious about their movies.
So in short, it's not really about fancy-schmancy streaming methods - it's about hauling a metric shitload of data across the network - and as the networks improve, so will user expectations and demands, thus further increading the load. This is true both for streaming (whatever flavour of it) and torrents.
Access networks & distribution networks
There's three pipes that gets interesting when we're talking Internet and bandwidth - the bandwidth between you and your Internet Service Provider (access network), the bandwidth between the providers' equipment that you connect to and their core network (distribution network) and finally, whatever capacity they have from the core to other parts of the internet (peering).
First things first - bandwidth is expensive, make no mistake. That said, it's relatively easy to add peering capacity since you do it in a very limited number of points, and any new peering tend to take the pressure off the existing points as well. It's usually not a bottleneck, especially for the big players. For a smaller ISP, it might well be (see 'expensive' above)
As for the distribution network, you might have issues, depending on your technology already in place. Say you have a GigE fiber pair - that's 1000 Mbps up, 1000 Mbps down. Anything dangling off that distribution point shares the available bandwidth there. GigE is relatively cheap, anything higher might require an investment in new equipment (that groks 10GigE) - and population density will affect things here. Japan and Korea - both places oozing with cheap and good bandwidth - aren't exactly sparsely populated. Less density = more equipment required per user. And things tend to be priced in a manner so that three boxes covering 1000 users each well could be as much as one box covering 10000 users.
Finally, for the access network, the available bandwidth varies greatly depending on the technology is in use. In some cases - Cable modems, Wireless Ethernet and 3G being notable examples - the bandwidth is also shared among a number of users, sometimes several hundred.
This is where you start having issues. DOCSIS/1.1 network? 38Mbps downstream, 9Mbps upstream, shared among a good number of users. My HD trailer above would chew 25% of the available downstream for the building or area while I was loading it. Even in a DOCSIS/3.0 network (which you won't see in that many places thus far), you'd at least notice the load as a slight peak off the baseline. And that's from one user.
In reality, you'd have to look at aggregates to get a good view of what's going over a network. I'm not watching trailers 24/7, but I'm not the sole user either. 10 HD-1080 viewers could well chew up 50% of the available downstream even on a DOCSIS 3 network. It's a fine line to walk between cheap (= many users per loop, sharing the costs) and responsive even during peak (= enough bandwidth to cover even the peak requirements without a sweat). Or, in the case of Wireless and 3G, between useful/restricted in many ways and useless/totally unrestricted.
To give you an idea about aggregates, I present the following screenshot. It's a shot of a Swedish PacketLogic just after peak on a weekday and represents just over 1000 active hosts - i.e hosts with at least one connection of some sort going, be it a DNS query, an IM convo or a torrent download. Mixed fiber, DSL and Cable users. I'm not trying to make a point, just provide some sort of numbers.
Do note though that there's really no such thing as an average network and this can't be seen as a very representative for 1000 machines anywhere on the net.
Introducing fairness
So if we fly by the assumption that there will be some sort of congestion somehwere in the network at least at some times or at some point in the future, the question becomes how do we deal with it? There isn't any really easy answer to that, but let's have a look at some of the effects and options..
For UDP (many action games, some video streaming, DNS, VoIP), congestion will manifest itself as lag, jitter or choppiness. Much, but not all, UDP tends to be used by fairly interactive apps where this will be noticeable by the user. If a DNS query is dropped, it might be a short while before the system tried to resend it, meaning a few more seconds worth of wait before that web page starts loading. Or a missed shot in Quake.
For TCP (Web browsing, downloads, most P2P, many MMORPG's), congestion will make TCP cut the tranfer speed noticeably, and it takes a while for it to ramp back up. If you're doing a 25 man raid in WoW, it's perfectly possible that this will manifest itself as sudden choppiness that'll last for a while before returning to normal. If the congestion remains, your lag goes up for the duration and healing just became a whole lot more difficult.
If you're downloading a torrent, however, you'll be connected to a good number of hosts rather than just one. Even if a number of them takes a hit, a number of them won't, and frankly, you're not sitting in front of your monitor just waiting for the torrent to finish. The BitTorrent network is very good at utilising bandwidth, sometimes at the expensive of other apps.
So how do you deal with the congestion? There are several ways.
One way to prioritize is to do it by the service in use and from a pure customer happiness perspective, it makes some sense. 49 WoW users at 10Kbps (average) who are statistically very likely to be glued to the screen chew less bandwidth than one moderate BitTorrent user. So if some pipe is becoming full and you need to drop something, do you tell the 49 highly interactive users to back off (dead warriors and cries of agony), or do you tell your BitTorrent users to back off, inducing a very minor bump in the wire that would be pretty damn unnoticeable? This might sound like a very contorted example, but it's actually not too far from the truth - a very minor 'priority lane' for a few given protocols can make the experience for those users a lot better without degrading the transfer speeds for bulk data more than a few percent. (I realize that the main gripe with DPI and P2P is where the P2P is shaped down to abysmal transfer rates or blocked altogether. While there are cases where this is done for pretty boneheaded reasons, it's also perfectly possible that a segment is oversubscribed and that letting P2P run rampant would mean that the P2P wouldn't be much faster and a lot of other things would be fairly slow. I'm not going to defend this other than by saying that generally speaking, your ISP do want to give you the best possible service, even if they're not exactly stellar at planning. You don't bite the hand that feeds you.)
If you don't want to go down that particular alley, you can control the available bandwidth for a user based on total bandwidth transferred. Transfer a lot during a 24 hour window and your transfer speeds get knocked down to a lower tier. Decent equipment can implement this as a rolling window - so if your peak your bandwidth usage is average-high but your average is average-low, you'd likely not get affected - and if you do, it's a pretty temporary state. This also means that heavy P2P'ers would be pretty much living in the lower tier.
You could also go protocol agnostic but just plain relegate the heavy users into a lower priority tier than the less heavy users. Less bandwidth available for the heavy users at peak hours since the more casual users would be doing their stuff (depending on the gear used to shape, it'd show up as plain slower downloads or pretty annoying packet loss)
There's also the option of just plain selling bandwidth you can use 24/7 unhampered. This might mean that you get 512/128 - or way less, depending on what technology you're using to access the net (DSL, Cable, Wireless, etc) and you're likely to pay more or much more than you do today. Not very encouraging and forget any service that depends on high available bandwidth.
Or, you can just charge per megabyte transferred. I'm not too keen on this one myself, but it'd be a possible service model. Sadly it'll likely mean that users would be reluctant to use bandwidth intense applications - even casual YouTube'ing chews quite a bit of bandwidth - and people can get quite nasty bills out of the blue if they're not aware of how much their apps chew or if they get some talkactive malware. Forget heavy P2P. Cringe at software updates.
There's other ways as well. But the bottom line is that as an ISP, you probably will run into congestion somewhere in the network at some point and you'll have to deal with it. In some economies, you have to live with the congestion and just deal with it best as you can (there are places in the world where a 4Mbps line costs an arm, a leg and the better part of the torso) - this is also true for some access technologies. There's only so much bandwidth available in a wireless cell.
DPI uses
Quoting Mr Odlyzko,
"On the other hand, you do need DPI in either of two situations:
– You want to prevent faster-than-real-time progressive downloads that provide low-cost
alternative to your expensive service.
– You want to control low-bandwidth lucrative services that do not need the special video
streaming features. "
Feel free to call me biased, but neither of these are the standard use cases I see for DPI (and by 'DPI' I refer to DPI and shaping as implemented in traffic shaping devices), and they do strike me as at least a bit tinfoil hat'ish.
For ISP's, I'd say that some of the common uses are:
- Fairness between users (ensuring that Joe doesn't overly affect his neighbour Bob if they share the same medium)
- Prioritising VoIP. Yes, really. Sure, if the ISP is a telco and is selling POTS as well, they'd be keeping a pretty damn close eye on how much business they're losing to VoIP. Still, I never ran across a single case where SIP got hampered in any way by a normal commercial ISP. There are countries where VoIP is outright illegal (and this is weird), but that's really a whole other ballgame.
- Prioritising some other interactive services. DNS, WoW and IM are a few of the ones I see.
- Shaping down P2P in the access network so it doesn't chew up all of the available bandwidth.
- Shaping down P2P at peering points and peak hours to keep the bandwidth costs low. Mostly smaller to midsize ISP's that pay the larger fish for bandwidth.
- Prioritising HTTP based media streams, RTMP and other streaming (Flash video over HTTP - i.e YouTube, BBC iPlayer, various windows media based streaming, Apple iTunes, etc). It makes sense to do so since faster buffering = happier user, if you've got the bandwidth to spare or just prioritise it over BitTorrent transfers.
- Shaping down HTTP based media streams (Again, Flash video over HTTP) since they do buffer and aren't as sensitive to congestion as other streaming can be) - useful if there's very limited bandwidth available and you don't want to cannibalise 'more' interactive services.
- Above all, statistics and insight. Network operators are as curious as the next guy and it makes a lot of business sense to know what your users are using.
Bottom line though, I can't see any good scenario where you'd want to hamper video over HTTP because it's video. There might be cases where you might want to punish HTTP downloads in general and some equipment won't see any difference between a ZIP and a FLV - both are just a large HTTP transfer - but these really wouldn't be restricted to video and we're talking very limited bandwith networks.
0 comments:
Post a Comment