A primer on net neutrality

[NB: this post was written in May '09. While the conceptual issues and the structure of the debate remain largely unchanged, the interim has seen important developments that render the discussion of the status quo obsolete -- e.g. the FCC's explicit adoption of net neutrality policy. I plan to address this in a new version of this post.]

Along with piracy and investment in superfast networks, net neutrality is at the centre of the debate about broadband public policy. More and more policymakers, media executives and concerned citizens are having to come to terms with what is by any standards a very complex area full of technical and economic jargon. Unfortunately, there are precious few accessible balanced and professional introductions out there.

This post is an attempt to provide just that. I've tried to make it as accessible as possible without sacrificing the important subtleties. The first section gives the big picture quickly; the nuances come later. Stop reading when the level of detail is more than what you need. Links to further reading are at the end.

Net neutrality in a nutshell

Net neutrality means different things to different people, but this definition will do here:

Net neutrality means that your Internet Service Provider (ISP) should have no say in deciding which legal websites you can and cannot visit

This means that your ISP should not play favourites between websites, and should leave you to decide freely where to go. If your ISP has a commercial interest in you visiting certain websites over others (perhaps because it made a deal with them, or is owned by the same people), it should not use its control of the 'pipes' to privilege its preferred sites. For reasons I will explain, many people believe that net neutrality is 'at risk' and should be protected by law.

Clearly, net neutrality is a principle about consumer rights. But it is also about competitive strategy: it limits ISPs' actions by telling them not to use their control over the 'pipes' and their subscribers

1. as a way of favouring certain websites, or
2. as a way of forcing content providers (website owners) to the negotiating table, threatening to block them unless a 'carriage fee' is paid

While you could argue that (1) is just about preventing anticompetitive practices, (2) is more unusual in that it essentially forces ISPs to 'carry' all content providers 'for free', relying only on subscribers' fees for revenue.

However, complaints by ISPs that content providers get a 'free ride' should be put in perspective. In the cable TV industry, where most carriage decisions are matters of negotiation between content providers and cable companies, it is usually the pipes people who pay the content people for the right to carry their content. 'Fairness' here is a very blurry concept.

Today, ISPs are generally not required to observe net neutrality, and yet things are much as if they were: ISPs let consumers access all content providers, and content providers don't pay ISPs. This has led many people to think that net neutrality is a natural outcome of market forces and that therefore there is no need for legislation to protect it. But the claim that the status quo is just the result of market mechanisms is questionable, at least on two grounds:

• Before it was (largely) privatised, the internet was set up by the US government with net neutrality built-in
• The US Federal Communications Commission (FCC) has taken a pro-net-neutrality stance in certain key disputes in the past

There is a possibility that these state interventions may now lose their momentum. The FCC's right to take a pro-neutrality position is currently being has been successfully challenged in the courts. And changes in technology and consumer behaviour suggest that while net neutrality may have made commercial sense for ISPs in the past, in the future it may not (more on this below). Lack of clarity about all this has led some people to call for net neutrality to be explicitly enshrined in legislation.

Two caveats. First, nobody is seriously considering blocking access to ordinary websites. This is about clarifying the rules ahead of a future in which much of living-room TV viewing may happen over the internet. Second, unlike fixed (landline) ISPs, mobile/wireless operators have historically not observed net neutrality. Their right to continue doing so is increasingly a central part of the debate.

Other versions

As I'm using the concept here, net neutrality is not opposed to an ISP prioritising emergency Skype calls or slowing down certain kinds of traffic (e.g. peer-to-peer downloads or mass emails) if that is a sensible way of managing their network (e.g. controlling costs, avoiding network congestion or preventing spam), provided this is done fairly and transparently, and is not done for anti-competitive reasons.

However, this notion is at odds with certain other definitions of net neutrality you may have found out there. According to them, ISPs should treat all traffic (or 'information packets') equally. More specifically, while

• the definition I'm using here only says that ISPs should not discriminate between different providers of content or applications (i.e. on the basis of the content's source),
• other definitions add that ISPs should not discriminate on the basis of technology (or protocol) being used, such as peer-to-peer downloads, streamed video or internet telephony. A variant of this notion that is often used by engineers is the end-to-end principle, which holds that the internet's networks should treat all information flowing through them equally – at a technical, not just a commercial level (see the world of ends website for more on this).

Although the second definition can sometimes amount to the first, the two are different animals: the first is about competitive behaviour, while the second is about specifying exactly what service ISPs should provide.

Because it is more reasonable and involves less regulatory micromanagement, the first definition has a higher chance of being adopted as policy than the second. It is the version being advocated by corporate pro-neutrality lobbyists such as Google, and is in the spirit of the FCC's current push. In what follows I'll use this definition only – not because I'm advocating pro-net neutrality legislation, but because otherwise the discussion would be less relevant to public policy.

Why does net neutrality matter?

Net neutrality matters for at least three reasons:

• First, there is the political dimension: many people worry that if ISPs have discretion over which websites to carry, the future will see a very different internet to the one we have today. Websites from big media corporations would be able to make deals with ISPs to secure carriage, while smaller, unknown and challenging voices might get "squeezed out" (Obama). The internet's promise of a more plural democracy would be thwarted. (Of course, this view is contested).
• Second, net neutrality is a battlefield in the fight over consumers' wallets. For ISPs, it means they can't a get bigger a slice of the money that companies like Google, Amazon, Myspace et al are getting. To a large extent, they are condemned to acting as boring, low-margin utilities (more on this below). As the NY Times once said in a related context, "this is in good part a business negotiation being conducted in a policy arena."
• Third, net neutrality matters for national economic policy. If you want to create an environment where lots of new Googles can spring up from garages, you don't want the Pages and Brins of tomorrow to have to negotiate with every ISP for the right to reach their users. But if your vision is to have lots of film studios (the content owners) making lots of content, lots of ISPs delivering this content via superfast networks, and consumers accessing (and paying for) this content, then you can leave that job to ISPs and studios, and you don't need to be nice to the website industry (today's content providers, not owners). Of course the answer will probably lie somewhere in the middle – and you can guess who is pushing for which view.

Three related debates

Net neutrality is related to, but different from, these three other controversies:

• Piracy, and calls for ISPs to crack down on it via measures known as "graduated response", "three strikes" and the "Hadopi" law (in France). Until now, ISPs in both the US and Europe have been largely exempt from legal liability for their subscribers' activities (this is called "safe harbour" in the US and "mere conduit" in Europe). If ISPs are made liable for ensuring that their users don't engage in piracy (not the only option being discussed), they might just choose to block all content downloads whose legal status they are unsure of. This could force content providers to enter into arrangements with ISPs so that legality can be ascertained. Whether this would lead to net neutrality being undermined is unclear, but in any case the resulting ecosystem would be very different from today's internet.
• Metered broadband – i.e. ISPs charging their subscribers according to how much they use the internet. In April 2009 Time Warner Cable backed off from attempts to introduce metered broadband after politicians got involved. Metered broadband is commonplace in Europe.

  Metered broadband is related to net neutrality in that a cable company could potentially make internet downloads so expensive that consumers find it cheaper to use the operator's own video-on-demand services than view a film from, say, Hulu or iTunes. But barring anticompetitive cases like these, metered broadband and net neutrality are unrelated. If ISPs simply make broadband expensive but not for anticompetitive reasons this may irritate consumers and/or video website owners, it may delay the advent of internet-delivered TV, and may even be an abuse of monopoly power – but it is not in itself against net neutrality.

• Open networks, or unbundling. This is the regulatory process of forcing those who own the 'last mile' networks (those connecting consumers' homes) to share their pipes with other ISPs. This is mandated in Europe but not in the US, although it is a requirement for the new US broadband stimulus funds.

  Unbundling regulations are designed to strengthen competition between ISPs. In some people's view, strong competition between ISPs is enough to guarantee net neutrality without further regulation—but among economists the jury is still out on this (see, e.g., this paper). Essentially, the question boils down to what is the greater loss: allowing internet video to eat away pay-TV's revenues (while also incurring bandwidth costs), or risking users' frustration by not allowing them to surf the internet freely. While this remains unanswered, so does the question of whether or not any new rules are required if net neutrality is to be preserved.

Why now?

Leaving the possibility of sanctions aside, there are good reasons to believe that whatever commercial rationale ISPs may have had in the past not to block websites may not apply in the future:

• As the internet becomes a viable medium for the distribution of films and TV content, two things begin to happen:
  • The internet begins to compete with pay-TV (cable TV, satellite TV and ISP-run IPTV). In Europe, ISPs are increasingly also (and mainly) in the pay-TV business; and in the US cable operators are the main broadband providers. The result is that on both sides of the Atlantic, broadband is beginning to compete with ISP's TV businesses.
  • The cost of carrying this content begins to increase, which makes ISPs keen to find new sources of revenue. Now, any economist reading this might object at this point: if ISPs have always been retail-only businesses, always passing all of their costs to consumers, what prevents them from passing on the costs of video distribution as well? Part of the answer is that some ISPs pay more for bandwidth than others. If you have a lot of competition between them, those with higher bandwidth costs suffer, since they can't reflect these costs in higher retail prices without being undercut by their competitors.

  As the internet begins to move to the TV set and the living room, these conflicts are set to intensify. This move is already happening. The Apple TV lets you buy, rent and instantly watch premium movies in high definition without your ISP even knowing what you are doing. In the US you can also get movies from Amazon delivered straight to your Tivo, or to a variety of high-end TV sets; and you can download movies from Netflix to LG televisions and the Xbox games console.

• Until recently, monitoring and selectively blocking people's internet activities was beyond ISPs' technical capabilities. But in the last few years a new set of technologies (called Deep Packet Inspection, or DPI) has changed that.

Is this about laws and regulations?

To this day, net neutrality has not been explicitly protected by legislation in either the US or Europe. And yet – with the exception of some mobile operators – ISPs do not block access to legal websites on commercial grounds, and website owners don't pay ISPs. This has led some people to claim that net neutrality just 'happens' naturally as a result of market forces, and that hence there is no need for governments to intervene.

This is a strong argument – 'if it ain't broke don't fix it'. But it has a pitfall: although they have not passed laws, governments have played an important role in net neutrality in the past:

• The internet was not created by private companies but by governments over two decades – with net neutrality built-in. When it was eventually privatised in the mid-90's this was under a gradual and carefully managed process which ensured that the original modus operandi was maintained.
• Over the last decade the US Federal Communications Commission has intervened several times to ensure that some degree of net neutrality is respected – see this and this. In 2005 it introduced its now-famous four "internet freedoms" principles:
  • Consumers are entitled to access the lawful Internet content of their choice.
  • Consumers are entitled to run applications and use services of their choice, subject to the needs of law enforcement.
  • Consumers are entitled to connect their choice of legal devices that do not harm the network.
  • Consumers are entitled to competition among network providers, application and service providers, and content providers

  The FCC's authority to enforce these principles is currently being was successfully legally challenged by US cableco Comcast.

• Legal experts are unsure as to whether net neutrality is already implied by existing competition law and telecoms regulations. While this is so, and lacking legal precedents, any ISP contemplating blocking a website would need to consider the possibility of lengthy lawsuits.

None of this is not to say that ISPs would discriminate if they are told they can, but it is clear that until now the uncertainty around (and risk of) possible regulatory or legal challenges may well have played an important role in ISPs' decision not to block websites.

Net neutrality is tangled up with a number of other policy issues:

• As part of its concessions for its merger with AT&T, SBC agreed in 2006 that the new AT&T would respect a high-level version of net neutrality (the FCC principles above) for two years. That period ended in December 2008.
• Governments around the world are keen to encourage investment in superfast broadband infrastructure. But imposing net neutrality regulations may weaken ISP's business case for investment. See this. I will write about investments in a separate post.
• Piracy and the content industries' (mainly film studios and music labels) lobbying efforts to stop it: one of the options they have proposed is that governments should require ISPs to prevent people from downloading pirated content. See above. I will discuss piracy in a separate post.
• Obama: net neutrality featured prominently in President Obama's campaign pledges, and of one the key people behind his technology policy, Julius Genachowski, has been appointed to chair the Federal Communications Commission.
• The EU's 'telecoms package' which, among other things, may set the ground rules on net neutrality and internet anti-piracy enforcement across the EU. After years of debate and lobbying, the package got derailed at the last minute on May 6^th 2009 because of controversy around some proposed anti-piracy measures (not because of net neutrality). The package will be discussed again late in 2009, but it is not clear whether net neutrality will be revisited then; as it stands, the proposed package does not explicitly protect net neutrality.
• The UK Government's Digital Britain report – a preamble to the next UK Communications Act – emphasised the need to fight piracy and to invest in broadband infrastructure, while stating that "the Government has yet to see a case for legislation in favour of net neutrality". This made some people angry.

Who is involved?

At the corporate level, net neutrality's main supporters are those who stand most to benefit from it: large internet companies like Amazon, Google and Ebay (owners of Skype); the Open Internet Coalition lists many of these. The main opponents are those with most to lose: telcos and ISPs, some of which have joined forces behind the Hands Off The Internet group. This NY Times article discusses the corporate lobbying effort in Europe. In the UK, BT has recently called publicly for video websites to contribute towards distribution costs. This quote from AT&T CEO Ed Whitacre has become a bit of a cliché but is still mandatory:

How do you think [Google, MSN, Vonage and others] going to get to customers? Through a broadband pipe. Cable companies have them. We have them. Now what they would like to do is use my pipes free, but I ain't going to let them do that because we have spent this capital and we have to have a return on it. So there's going to have to be some mechanism for these people who use these pipes to pay for the portion they're using. Why should they be allowed to use my pipes?

Politically, in the US the divide is generally along partisan lines, with Republicans generally siding with telcos and opposing pro-neutrality legislation, and Democrats favouring it (this Wikipedia article lists various failed attempts to legislate net neutrality in the US). In Europe the debate has not been very political yet, but if the recent (and related) debate about internet piracy is anything to go by, Europe is likely to see a similar left/right divide.

As to advocacy groups, the Free Press, Public Knowledge and Electronic Freedom Foundation groups have all been vocal supporters of net neutrality in the US, and the Save The Internet Coalition lists civil liberties groups and liberal intellectuals as its members. In Europe, La Quadrature du Net, Blackout Europe and Monica Horten's IPTEGRITY blog offer pro-net neutrality commentary. Opposing groups in the US include the Progress and Freedom Foundation, NetCompetition.org (members include AT&T, Comcast, Qwest, Sprint and Verizon) and the CATO institute.

As to experts and academics the list could be endless, but here are a few names worth mentioning. Net neutrality's main supporters include law professors Larry Lessig (see this testimony), Susan Crawford and Tim Wu (who coined the term), economist Nicholas Economides, mathematician Andrew Odlyzko, inventor of the web Tim Berners Lee, and "father of the internet" and Google employee Vint Cerf. Key opponents of neutrality regulations include Adam Thierer and Bret Swanson at the PFF, Timothy B. Lee, Scott Cleland from netcompetition.org, and various renowned economists who have been at least sceptical about the idea. Other experts worth following include Stefano Quintarelli (Italy), Chris Marsden (UK/EU) and Rob Frieden (US).

This page lists lots of statements grouped by side, plus papers and much more.

The flip side

Most people – even among those who oppose net neutrality regulations – agree that net neutrality as we have it today is a good thing. The problem is that any rules mandating net neutrality would risk 'unintended consequences'. Here are some:

• The net-neutrality concept may not be future-proof. It may have served us well until now, but will it work in ten or twenty years' time, when we use the internet for all our live TV? Perhaps the free market would be better equipped to come up with the right model than any policymaker. You could argue either way, but the question is not clear-cut (for techies: how do you roll out IP multicast in a non-discriminatory way?)
• It is hard to implement. Even if you agree that ISPs should not discriminate on commercial grounds, how do you decide what is discrimination and what is just good housekeeping? You could quickly end up mandating a 'technological' version of net neutrality such as the end-to-end principle – which is unlikely to fly, on grounds of regulatory micro-management.
• It is difficult to mandate net neutrality, even in a general commercial sense, without to some extent implicitly mandating a business model for ISPs – regulatory micromanagement again. More on this below.
• Net neutrality regulations may discourage firms from investing in new infrastructure – if it means that investors are stuck with a low-margin business model.

While some of these concerns may be addressable, there is no silver bullet: introducing net neutrality regulations would unavoidably involve a trade-off between ends. If you believe that preserving the internet as we know it is essential for the economy's future, and you believe that this cannot happen without new rules, then you may have no option but to favour legislation. But however much you try, you may be unable to avoid limiting some industries' prospects– much like when the Glass Steagall act limited what activities banks could be involved in. In that case there was a compelling rationale. Is there one here?

ISPs' perspective

If ISPs are not allowed to discriminate between content providers (i.e. website owners), they have little chance of ever receiving money from them– because content providers can reach their users without paying ISPs. And if ISPs can only get their money from consumers they are stuck with a retail business model. They are also stuck with an undifferentiated product to sell: access to all of the internet, exactly the same internet that the next ISP offers. This is a commodity business, and the main way of competing (if there are competitors) is by reducing costs and lowering prices. And because there are economies of scale, the result is consolidation and eventually an industry with a few big players – as is happening in the UK today. With effective competition, margins are likely to remain tight; and if competition disappears, regulators may step in and force margins to remain low.

What is an ISP to do? There seem to be four main ways out of this:

• Treating broadband as an add-on to other, more interesting retail services (mainly television and telephony).

  As I argued above, as the internet begins to compete more directly with television, concerns about the (anti) competitive aspects of net neutrality are bound to arise. If net neutrality regulations are introduced this is likely to hurt ISPs' revenues.

• Offering content providers better-than-normal service so that they can reach their end-users with higher quality – i.e. a system 'tiered' around 'quality of service' or 'QoS'.

  This approach is not devoid of controversy, as it may create anticompetitive incentives. The concern is that once enough content providers pay for premium carriage, ISPs may be tempted to pressure those who 'opt out' to 'opt in' by giving them 'worse than normal' service (this post by Rob Frieden explains the argument better). And if – for this or any other reason – one day you can't launch a competitive web site without paying ISPs, net neutrality will have disappeared in spirit if not in letter.

  Not all proponents of net neutrality are against paid fast lanes. Larry Lessig has recently written in favour of them, provided they are offered to all content providers on the same terms.

• Offering content providers ways of reaching their end-users at lower total cost – for example, by offering to bypass the Content Delivery Networks that often sit between content providers and ISPs.

  This is a nascent development. I have not come across strong arguments in favour or against this from a net neutrality perspective (but see this post by Rob Frieden on a related issue).

• Charging content providers for access to ISPs' relationship with users – e.g. by handling micro-payments for content, or providing (anonymous) information about end-users' demographics and/or browsing habits, so that advertisements can be better targeted.

  This strategy is the subject of much excitement among ISPs – largely thanks to the labours of the clever people at STL and their Telco 2.0 initiative. It is also controversial, as it sometimes involves issues around privacy.

If network neutrality becomes law, ISPs' future could lie in the last two strategies.

Conclusion: is the debate irrelevant?

At the heart of the debate about net neutrality there are three questions:

• Is net neutrality desirable?
• If it is, is there a risk that without new regulations neutrality will disappear?
• If neutrality is desirable and at risk without new regulation, how far are governments willing to go in its pursuit? What are the downsides of introducing regulation?

The first question is mainly a political one. The second is one for the economists. And the third is one for policymakers once the second has been answered.

In the US, political views on the first question seem to be relatively clear – but far from consensual. But the answer to the second question is far from clear, at least in the published literature. And while this is so, whether the first and third questions are of more than academic interest is unclear.

For interested parties this means that everything depends on the answer to the second question. Which means nothing but good news for economists: years of lucrative consulting opportunities await them.

Where can I learn more?

The Wikipedia pages on "network neutrality" and "network neutrality in the US" have good overviews with lots of links to more in-depth documents. This page lists lots of resources, grouped by side. This briefing at law.com gives a well-rounded overview of the issues and history from a (US) legal perspective, and this report by Oxera Economics gives a good and brief overview from an economic perspective. This Daily Finance article gives a detailed account of the US behind-the-scenes fight. The rather academic blogs by Rob Frieden (in the US) and Chris Marsden (in Europe) offer ongoing sophisticated commentary.

Updates:

08/06/09: Added links to netcompetition.org, Scott Clelan, Google Public Policy statement

11/06/08: Added various clarifications, links to BT statements, mention of QoS