Level 3’s Lower Cost Comes From Owning The Network, Not Free Peering
One of the things I've seen some bloggers, Wall Street analysts and readers commenting on regarding the Level 3 and Comcast dispute, is the idea that Level 3 only won the Netflix business with their low price, because Level 3 thought it could get free peering from Comcast. Many also want to imply that Level 3 only won the Netflix deal because of that low price and suggest that Netflix is somehow sacrificing quality by using Level 3, in exchange for saving some money. None of this could be further from the truth.
There is this false notion amongst many that Level 3 can't make money on the Netflix deal if they have to pay Comcast for peering and that this is the reason Level 3 is so upset. Some have even suggested that Level 3 is being unfair because they don't want to pay for something the other CDNs pay for. I've also seen many suggest that Level 3 didn't know what it was getting itself into when it signed Netflix as a customer or that Level 3's approach to the CDN business is the same one they had four years ago when they first entered. Some want to use the example of how Level 3 might charge Cogent for peering and suggest Comcast is simply doing the same thing except that Cogent is not a last mile provider like Comcast. All of this has me wondering why almost no one has noticed that Level 3 owns the network and has a lower cost which means they can offer a cheaper price in the market to begin with, regardless of whether or not they have to pay Comcast.
I've seen some posts say that without free peering from Comcast it would, "erode Level 3's profitability on the Netflix business." Really? Based on what data? Because if you ask Level 3, and I have, they say they can make money on the business regardless of whether or not they have to pay Comcast simply due to the fact that Level 3 has a lower cost to distribute the content since they own the infrastructure. Not to mention, the fee that Comcast charges Level 3, which is based on each port Level 3 turns up, is not a lot of money. Level 3 is not balking about the price they have to pay, it's not a lot. They are arguing about the principle of what Comcast is doing. The rate Comcast is charging Level 3 today has no financial impact on Level 3's business and you don't see Level 3 or Comcast debating this.
The issue at hand is not about Level 3 getting something for free. It is about Level 3 using their network, which they made a huge investment in, to carry traffic further than anyone else. Any network connection is two way and a CDN connection is one way. So for some to compare Level 3's infrastructure costs to those CDNs who don't own the network is simply not an apples to apples comparison. For some to suggest that this is an "unfair" advantage that Level 3 has, they are wrong. It is an advantage, but it is not unfair, Level 3 spent a lot of money to build out the network. The idea that Level 3 should simply pay Comcast because that's what all of the other CDNs do is not a valid argument. Level 3 is not like the other CDNs.
Capex, space and power are cheaper for Level 3 and they run on a very predictable capex cost decline. Commodity servers, like the ones Level 3 users, as well as some other CDNs, decline at 20-30% per annum. The throughput is then what drives Level 3's cost of that capex on a unit basis. If Level 3 doubles the throughput of their servers they just halved that element of the cost. Since Level 3 has been very focused on a particular segment, broadcast and media and entertainment, the right sort of library and traffic actually improves Level 3's cost base the more they get, as in the case of Netflix. In parallel they do a lot of work on the software to improve the throughput with every code release they do. I'm not sure other CDNs think about this in as maniacal a way as Level 3 does. Maybe they do, but since Level 3 owns and operates a network, they have a different engineering culture.
Level 3 uses their own colocation facilities to house their own servers. They know exactly what the difference is between their cost and the price of colocation because Level 3 also sells it. It's a big gap and one that is increasing as colo rates have been going up steadily over the last few years. Yes some of the other CDNs have servers that are in "free" facilities within ISPs but not the ones that drive most of their traffic. I think we all know by now the fallacy of "thousands of locations" used to deliver large objects. Akamai, Limelight and Level 3's distribution architecture is very similar for that sort of traffic. And other CDNs are in third party colocation facilities, including Level 3's.
Last mile delivery is the same. Level 3 users their own network and they sell the same thing. While prices are not going up for this offering, Level 3 can still make a healthy margin and their cost is a lot less than what their competitors pay for IP bit delivery. And they all buy IP Transit, again some from Level 3. Here it gets a little trickier to access though because Level 3's competitors do peer and they do also get free delivery where they have persuaded an ISP to embed their servers in their network for free. But peering isn't free. You need a routing edge to deliver traffic to your peers. That's exactly the same as Level 3's, but at far less scale. No one even approaches Level 3's cost to deliver bits at the edge. For the free bits delivered inside an ISP network (without a router), you have to consider that it might be free, but Level 3 gets paid to deliver a portion of their bits. So even here free isn't low enough.
These are some of the advantages Level 3 has of network ownership. But it is also a very particular network ownership. One that owns colocation; one that operates at the greatest scale; one that sells IP transit to ISPs. There aren't too many of those. Even amongst the biggest IP networks Level 3 has the lowest cost.
As for the idea that Netflix is sacrificing quality for a cheap price, that's laughable. For those that don't know, Netflix is very particular about the quality of their suppliers and hands-down one of the most sophisticated content owners in the industry in how they determine that quality. Netflix has employees on staff called "Business Intelligence Architects" and employees in their product development group who build the analytic systems Netflix uses to measure performance and usage of their streaming service. Any Netflix member knows that Netflix routinely sends out emails asking customers to rate the quality of the video they just watched and Netflix has an incredible amount of data on what the real-world experience is for their customers at any given time.
Not to mention, Netflix now sees the delivery of video via streaming media as their core business and the DVD business second. That means in order for Netflix to grow their business, they have to focus on the quality of the video they are delivering. It's laughable to think that Netflix is going to purposely deliver a poor quality video experience and put their business at jeopardy just so they can save money. Yes, price is always an issue, but Netflix is balancing performance AND price together and they do not and cannot separate the two.
Some are also writing about the five to one ratio of the volume of traffic taking place between Comcast and Level 3 and using that as an excuse to suggest that it's unfair to Comcast. Comcast used the word "dump" on their blog saying that Level 3 was dumping traffic onto their network yet Level 3 is simply sending content to Comcast's subscribers who have requested it. Level 3 is not "dumping" anything. For those that think the five to one ratio of traffic is out of whack and not fair to Comcast, go read Rob Powell's post on TelecomRamblings.com which very clearly explains how when connecting to a last mile network on the internet, that ratio is the norm, not unique.
Some have suggested that Level 3 could just refuse to pay Comcast and buy transit from those who peer with Comcast but the way Comcast's network is setup, it keeps Level 3 from even having that as an option. Level 3 could go deeper into Comcast's network, but so far, Comcast is not looking at this as a solution, even though it would remove a lot of burden from Comcast, which is what they are complaining about. If the networks interconnect in 5 cities but there were ultimately more cities where the ISP has customers then obviously connecting in more cities is getting deeper. Comcast operates in 40 metro areas in the US for example and typically, interconnection in the US between large networks is done in less than ten cities.
Hot potato routing is used predominantly on the Internet (see here for details). What that means is that packets sent from one network to another will exit the first network as soon as they can. If network x is interconnected to network y in both LA and NYC and a packet needs to get from LA to NYC then it will be handed over in LA and carried right across the country by network y. In order for that not to happen (for network x to carry it further) you first need to connect in more cities.
Then the net receiver of traffic (the last mile network) needs to use elements of BGP like MEDs, communities and/or selective route announcements (see here for details). This means they announce details that the net sender can then use to carry the traffic further before handing it over. A CDN can do part of this but there is always a network connection from a CDN device and the consumer. So a combination of the two gives ultimate flexibility of who carries what packets how far. And of course CDNs are simply not used for all traffic exchanged by interconnecting networks.
If you cut through all of this, my opinion is that Comcast sells a competitive product to Netflix and is simply scared of over-the-top video. I think that's the real issue here with regards to Comcast and that they will do anything they can to try and combat it, including finding creative ways to try and get paid as many times as possible. But none of that has to do with how Level 3 operates their network and the cost advantage they have as a result of owning and operating it.