Article Info

Who Pays for the Online Video Boom?

Inflation

Everyone talks about the explosion in Video traffic. Everyone talks about the explosion in the bandwidth required to carry it. No one talks about who is going to pay for it. There is one likely source: transit bandwidth inflation.

Robert X. Cringely is hit or miss as far as I’m concerned. He’s on a rant lately about how Google (GOOG) is secretly planning to control the universe by purchasing dark fiber. I don’t buy it. However, the benefit of his frequent delusions are the interesting sources he uses to make his arguments. His most recent post cites a paper that investigates the costs of video distribution on the Internet.

Lazy Blog Reader Summary of Paper: P2P distribution of media is radically cheaper than distributing it yourself using your own datacenter or outsourcing it to a CDN like Akamai (AKAM). By a factor of 1000.

Not a huge surprise to folks who understand how the Internet works. Everyone from the datacenter that hosts this blog to Akamai to your ISP pay for the right to transmit and receive bits on the Internet. The big cost contributor to the non-P2P applications, and what makes them more expensive, is transit bandwidth – the cost of transporting bits from the NBC or CBS datacenter to the eyeballs in the household.

This blog sits on a shared server at Mediatemple. Mediatemple pays for connectivity to the Internet cloud that allows these bits to reach your screen, typically a fixed price per Mb/s. The rate companies like Mediatemple pay for transit is highly elastic- the more they buy the better rate they get.

Akamai’s business is technically complex, but fundamentally it is about buying transit bandwidth low and selling high. Big bandwidth users like Akamai pay the least for transit so there is an economy of scale in their business. They couple this with a distributed computing infrastructure that allows content owners to stay out of the datacenter business. But a big source of their advantage in the marketplace is the arbitrage gains of reselling transit bandwidth.

If this is a source of competitive advantage then this is the critical question – who has the lowest transit bandwidth costs?

It’s the Telcos and Cablecos themselves. If Verizon wants to distribute content to subscribers their transit costs are zero. They don’t need to peer with anyone, in fact the Akamai’s of the world pay to send data into Verizon’s network.

Theoretically, this means that the Telcos and Cablecos could all enter the CDN business with a structural cost advantage. I have little faith in the creative business strategy of these companies but I have full faith in their ability to extract a pound of flesh from others. And I think this day is coming.

Therefore it would seem that folks like Verizon (VZ), AT&T (T ), Comcast (CMCSA), Time Warner Cable (TWC), Cablevision (CVC), etc are in a very good position to charge higher transit costs into their network, as the majority of traffic growth (Video) is going into their network.

The amount of bandwidth going into Verizon’s network destined for FiOS customers watching YouTube may explode, but the costs of carrying it will explode too. Verizon has a monopoly on the connectivity to these eyeballs. Theoretically, they could charge whatever they want for transit into their network.

Akamai and others will grimace at this bill but can effectively pass this cost on to their customers. It’s the content providers that will ultimately absorb the increase.

As a result, Network neutrality will never need to happen. Carriers will simply charge more for everything. Cablecos and Telcos can simply keep their transit rates flat while the amount of bandwidth explodes. Since they own all the end customers, maybe they even jack up rates. And surprise surprise, the cost of transit bandwidth no longer is headed to the floor (note log y-axis).

Carrier transit bandwidth cost inflation will pay for the video bandwidth explosion. Prediction: Total fees paid for transit into Cableco and Telco networks are going way, way up. Couple this with the possibility of the Mb/s transit bandwidth rates increasing and you can see the carriers are sitting on a very attractive source of cash flow. This cash will finance whatever new infrastructure they care to purchase, and maybe a little (or a lot) left over to pay shareholders.

That is, of course, if they can keep legitimate content providers from using P2P to eliminate the need to pay transit altogether. That scenario completely destroys the above assumption. I’ve said before P2P isn’t a business, it’s a catalyst to force media companies to distribute digital content. It’s clear to me it’s also the only weapon they have to avoid transit bandwidth inflation.

Full Disclosure: Author is long AT&T and Verizon

Discussion

Comments are disallowed for this post.

  1. * interesting analysis.
    * I’d add that this issue of “who pays for it” also has a long-distance vs. local flavour as well.
    * Akamai has global-ish reach vs most telco’s who are regional.
    * It seems unlikely that telco’s can do it themselves because of their regionalization ( though they are all getting more global ).
    * A good part of the Internet players success has been their ability to transcend regional to get the arbitrage/scale you note.
    * Again good stuff. (apologies for my scrambled logic)

    Posted by Iain | February 5, 2007, 5:05 PM
  2. Let me know when the CBOT starts selling bandwidth futures. I may want to pick some up.

    Posted by Sak | February 5, 2007, 5:06 PM
  3. My point in the above article is that bandwidth connectivity into a Verizon or Comcast may be priced differently than hypothetical CBOE bandwidth.

    Posted by Andrew Schmitt | February 5, 2007, 5:34 PM
  4. With access and switching gear getting smaller, there should be some room in the CO for Google/Microsoft/Yahoo/Akamai to put some servers in if needed. Of course I expect those servers to store my email, photos, documents, applications etc. For most of the mass media stuff, P2P will probably be better. I imagine some clever person has found a way to distribute protected content via P2P by now? Heck, it doesn’t even need to be an open P2P platform that runs on your PC, it could run on the set top box, they already have some sort of multi room DVR functionality on new DVRs, with enough local bandwidth the other room it streams to could be the guy down the street. That would be one way to save on server costs. Both STBs could be closed systems, encrypt the stream in between, it should be possible.

    Probably both of these will play part of easing transport demands, otherwise we need to invent some really high bandwidth transport systems. Access speeds are ramping very quickly to 100 Mbps for everyone but AT&T users, possibly transport can not keep up?

    Posted by Paul Warner | February 5, 2007, 11:55 PM
  5. I still feel caching at the edge will solve the video transport problem. Put the top 100 youtube VIdeos and Netflix titles in every CO (About a 1 Tb Sata RAID) and a big part of your bandwidth is cached. The question is what caching guys like Akamai will have to pay for transit into the access network. I see no reason why Verizon and Comcast won’t rape and pillage.

    There has been little attention paid to the enterprise… that is data that cannot be cached and could be substantially larger. Too much attention on the consumer today, not enough on the Enterprise.

    Posted by Andrew Schmitt | February 6, 2007, 7:46 AM
  6. Here’s the problem. Cost is not the biggest deciding factor anymore when it comes to distributing video so the single selling point that p2p has is not valid. p2p can’t do live, it can’t do DRM, etc… so by people simply saying p2p is better because it is cheaper is not valid. better than what? you have to look at the functionality each solution provides and compare it apples to apples. is it cheaper? yes. can it do many of the features streaming does? no.

    p2p solutions have been around in the U.S. for years but to date, theyt have NOT been adopted. Chaincast, Centerspace and all the others have gone under because all that matters is what people adopt, NOT which technology is the best. Why was the VHS player adopted over the BETA? We’ve seen this lesson many times over the years. People are not buying on cost alone.

    the ONLY thing the p2p guys ever try to sell you on with p2p is cost. have you ever gotten a sales pitch from someone offering p2p who told you it delivered content faster, at a better quality, with a better QOS, with better reporting and demographics….no. all they can pitch is cost.

    shipping bits is a commodity today and the cost to do it keeps getting cheaper. for major content providers, their biggest cost is still the content creation, not delivery.

    dan rayburn
    evp
    http://www.streamingmedia.com

    Posted by Dan Rayburn | February 6, 2007, 11:22 AM
  7. Dan – very good points. Shipping bits will get cheaper.

    But access to the consumers of these bits is not a commodity. It’s a duopoly. And P2P is the only way to bypass this duopoly.

    I fully expect Akamai to get into the P2P business. It’s inevitable.

    Posted by Andrew Schmitt | February 6, 2007, 11:39 AM
  8. I still only understand about 10% of what you write about, but the 10% I pick up is pure genius. Very fascinating, if Verizon and AT&T can ever get their fiber actually deployed, they will be a real juggernaut in the industry. I think cable probably has the competitive advantage for another 2 – 3 years, but the telephone companies aren’t sitting on their hands and once their fiber becomes more widely available, Comcast had better watch out.

    Posted by Davis Freeberg | February 6, 2007, 3:54 PM
  9. Dan –

    I agree 100% that it is not all about cost. It is also about performance, QOS, and reporting. I would also add that it is important that P2P to allow the ability to set business rules around both the content objects and user experience that are aligned with the business models of the content distributors.

    These are the things that Solid State Networks is talking about every day, and the message seems to be resonating. We are not offering the 90% cost reduction solution most other P2P companies tout. Our DDN offering enables the combination of reliability/QOS you expect from a CDN with the economic and performance advantages of P2P (specifically, our own extended implementation of the BitTorrent protocol) to deliver faster performance, at a guaranteed QOS, while lowering per-unit delivery costs.

    I also agree with Andrew. It is inevitable that Akamai will want or need to incorporate P2P delivery into their business, as will most every other serious CDN. That is why we see a need for enterprise class software tools to enable this next evolution of content delivery not only for the service providers, but for the growing number of content publishers that choose to manage at least a portion of their own infrastructure.

    Rick Buonincontri

    CEO
    Solid State Networks

    Posted by Rick Buonincontri | February 6, 2007, 5:49 PM
  10. Andrew,

    Great post. I agree (not surprising given that you used our market data to make your point — thanks :) Not to nitpick, but rather than “inflation”, I’d bet on “decelerating deflation”: price changes were about -35%/year in 2000-2005, now it may be -10% to -20% per year. Of course traffic has been increasing fast enought all along that the dollar value was never shrinkning, and therefore I fully agree that the dollar amount being spent on bandwidth is going go way way up and that the broadband access providers will be in a great position to charge more…

    However, content delivery networks are not necessarily going to end up losers. If they play their cards right, CDNs can position themselves as the quality and price mediators, which could be an enormously valuable position. To do that, instead of creating their margin by driving the price of their supply down faster, they have to add value by implementing ways for the content to get and pay for quality delivery. Whether that’s more for some of the bits than others (i.e. differentiation) or simply more for all bits (inflation) remains to be seen .. but the opportunity is there for CDNs jump in bed with broadband providers and benefit at the expense of the transit provider.

    -nemo-

    P.S. Sak, yes you can play the bandwidth market… See http://www.invisiblehand.net

    Posted by Nemo Semret | February 6, 2007, 8:35 PM
  11. I expect that you are right Andrew, the RBOCS will see a nice up tick in revenue from leasing connections and maybe rack space in COs, which is good, it should offset the revenue loss from voice.

    As for distributing popular media, things will push to the edge, and the last step is P2P. P2P should be able to do DRM and streaming if one wants, I don’ think anything about P2P precludes them. If you want DRM it does seem tricky, but in theory at least it can be done. Check out http://en.wikipedia.org/wiki/Superdistribution If one want real time streaming that should be possible too. BitTorrent and other swarming techniques don’t support it, but think multi room DVR where one room is your house and the other is your neighbor’s.

    It is a ways off to be sure, but it looks like BitTorrent and Solid State Networks are trying to go legit. I thin the evolution to wide acceptance will go like this, and I admit there are a lot of steps… Set top boxes are the likely host. Storage and processing power is increasing. They are closed systems often owned by the service provider, should make DRM easier. Upstream bandwidth, especially with GPON is huge, but mostly idle. New access boxes in the CO are being built with more bandwidth between users than towards the uplink. QoS should be possible using RSVP or something better. It should all be possible.

    It is pretty clear that the trend in all of computing, is and will be for many years, to parallelize and distribute. No one buys a mainframe any more, they move their application to a cluster. Intel can’t make a single faster chip any more, they put two slower ones in the same chip. Centralized data centers will move to regional, to local, to everyone’s home. Until then streaming from the data center will rule, especially with the aid of IGMP in carrier live IPTV offerings.

    Posted by Paul Warner | February 7, 2007, 12:27 AM
  12. I agree with nemo. In the end the RBOCs make their money through the subscribers, not CDNs.

    CDNs infact cut the RBOC operation cost by caching data close to RBOC. This is becoz if not for caching RBOC would have to pay upstream provider for repeated access to data.

    Ofcourse there is always a possibility that RBOCs might allow competitive CDN POPs in their location to reduce what they have to pay to any CDN.

    But the final deciding factor for RBOC is the subscriber experience. Their feedbacks will demand that RBOCs give priority to quality. And I believe the CDNs on their part will be differentiated by their customer list and delivery quality and reliability.

    Posted by akshay | February 7, 2007, 12:44 AM
  13. I’m not an Internet expert, but I wonder if search could be P2P as well? I’m thinking about the RAM in all the routers of the Inernet. Hashes of search terms can be made persistent in RAM, so that’s a sort of page rank. Here’s a dumb estimate: say you’ve got 1 GB average per router, then 100,000 such routers can store 100 TB of data at any instant. Of course, 100,000 servers could store 100 times this much data in disk, but hashing reduces the storage requirement. Besides there may be a Peter principle that 10% of the data is accessed by 90% of the people.

    Posted by M.G.Rajan | February 7, 2007, 7:36 AM
  14. Great post and thorough analysis as usual Andrew. However, I’ll disagree with your premise and prediction about transit prices going up. I do think we’ll see a slow in the decline of transit costs as there certainly is a margin floor, but I don’t think we’ll see transit costs rise (especially dramatically) for the following reasons:

    – Peering arrangements
    – Foreign ISP competition
    – The muscle of CDN’s & big content players (youTube etc.)

    Lets start with peering arrangements which will lead us into the second point about the Euro & Asian guys. As we’ve discussed before all the Tier 1 guys have free peering arrangements with each other, and for the most part the Tier 2 guys only pay a couple of folks here and there. Why is this important? Because screw Verizon and screw ATT – we (CDN’s in this case) don’t need them to get to an eyeball! You make a good point that Verizon may have the FIOS customers at their behest, but so what? Picture it this way: A Verizon fios customer is sitting at home and wants to download a video from siteX.com. SiteX is a colocated at a facility in Dallas and uses Savvis (not Verizon) for their transit uplink. Once the packets hit the Savvis link, SiteX will be incur the charge and thats it – done and paid for. Savvis doesn’t pay Verizon – neither do GLBX, Qwest, LVLT, and even some Tier 2’s like Tiscali (this is an example – i don’t know the exact relationship here). If Verizon all of a sudden tried to start make some of the really big guys start paying for peering, the big guys would laugh. If by chance they succeeded in squeezing out some Tier 2’s, the Tier 2’s have multiple ways of routing to the destination so they’d de-peer with Verizon and go through another ISP.

    Now, speaking of Tiscali, lets talk about foreign ISP competition. Its these guys who have quietly thrown a catalyst into the US transit market by building a quality backbone, running leaner and meaner organizations and networks, and using their European (or Asian if you’re PCCW for example) leverage to negotiate peering relationships. So for instance, if a Tiscali comes to Level3 and says I need 50Gbps of peering with you in San Jose, LA, Miami, Ashburn, and NYC and Level3 says “no” or “ok, but you’re paying,” then Tiscali would say, “ok, there goes your free transit in Milan, Rome, and Frankfurt.” They’ll eventually work out ratios and understandings of the relationship, but at the end of the day Tiscali can now offer cheaper transit than Level3 directly and I don’t give a damn what Verizon is charging anyone (on the backbone/transit side – not the customer side) because at the end of the day I don’t need Verizon to reach a Verizon customer.

    A major reason that we’ve seen this massive growth of BW usage is because transit did become more affordable. The telco’s have enabled the content providers to push quality bits at high speeds by offering them – or the CDN’s – cheaper transit. After the bubble recovery, there was all this capacity with unused dark fiber. GigE and 10GigE ports replaced OC12 and OC48’s and this made interconnecting more affordable. Also, CDN’s (and youtube/myspace/etc) are major customers of the ISP’s and we carry heavy weight with them due to volume. If one of my carriers was to come to me and say we’re raising transit rates when your 2 year term ends. I’ll then say, “ok, but you just lost any burst traffic you were getting from us (commit overages) and we will not longer be a customer of yours at the end of term.” If they lose 20Gigs or so from an CDN, thats significant to that Telco sales rep and billable Mbps. In addition, ISP’s need CDN’s b/c they help improve their customers’ performance, reduce the need for backbone build out, and effectively increase billable Mbps.

    Oh, one more thing. At least for now, p2p means nothing in the enterprise content sector. Its a great technology and has its applications. But we’re a long way from seeing any content that can potentially be monetized be dependent on random hard drives. Just my opinion.

    Posted by Jason Evans | February 8, 2007, 10:44 AM
  15. Akamai is a software company and should not be compared to the multi pop hosting companies that are essentially bandwidth resellers. To explain the statement think of an Internet start up like a three-stage rocket. In phase one they need flexibility and don’t want to burn capital on hardware. The use Akamai to prototype and gauge interest in their product. In the second stage of the rocket as the company scales they need to bring down their marginal unit cost. They use Akamai for scale and buying power. This is as far as a bandwidth arbitrage play can take a client. In stage three the client has their own economies of scale and can buy transit bandwidth at the same prices as a datacenter. Even at this stage Akamai still has tremendous value to the company to help run and manage their network. This client is no longer buying Akamai bandwidth they are buying the software as a service operational efficiencies.

    Andrew stop by Cambridge for a NOC tour you will be a believer.

    Peer-to-Peer being a 1000% less is a sensational but untrue statement. Peer-to-Peer operators replace their network costs with development and deployment costs. Ask any IT director the cost to maintain desktop software. That is usually for a homogenous deployment but imagine trying to maintain thousands of desktop applications across multiply configurations. If a Peer-to-Peer company wants to charge a price per GB delivered type model to be viable they need to be in a range that is not much lower than current CDN transit rates today. Most cannot get companies to pay this due to feature gaps like lack of Quality of Service SLA, privacy and security concerns. This will force Peer-to-Peer business models to vertically integrate. You see Joost in the ITV space, BitTorrent having a storefront and a home run hit with Skpe. The issue here is that consumers will only lend their 150Kbps of upstream bandwidth to one or two of these services. This will make it necessary for hybrid models to emerge like Kontiki. In this model they use both transit bandwidth in combination with peer to peer. This now creates a company with an org chart that is massive on the engineering side with weak sales and low margins. This will lower their share prices and without being properly capitalized they will fail.

    Posted by Tim Napoleon | February 10, 2007, 4:10 PM
  16. Trackbacks / Pingbacks

  17. Innowave - the next wave of internet innovations | March 28, 2007, 6:43 AM