The codec dilemma

Every day millions of videos are watched on video hosting sites like YouTube. Most viewers don’t think of how the videos are stored or how they get displayed on one’s computer. But these are critical questions in determining who is able to watch online videos and how much money they need to pay to do it, even on free hosting sites. Let me start by explaining what a “codec” is.

A video codec is a method of encoding and decoding video so it can be read from and stored on a computer. A good codec is one that produces files with high video quality that take up very little space on disk. Many different codecs exist with varying levels of quality and space usage. Wikipedia has a list of video codecs. Audio codecs, such as MP3, are similar to video codecs but for representing audio. Wikipedia also has a list of audio codecs.

Most audio and video codecs contain some encoding and decoding methods that are patented. A patent is a set of exclusive rights, including the right to sue anyone using a particular invention, granted by governments in exchange for disclosing that particular invention. Usually an organization will patent an invention so they can charge money to people who want to use their invention. This is the case with most audio and video codecs: organizations charge money to people that use their codecs because the organizations have acquired patents on particular parts of their codecs.

You may be thinking that you’ve never paid for a codec, but in fact, you probably have. If you’ve bought a DVD player, a Blu-ray player, an iPhone, an iPod, an MP3 player, a computer running Windows or Mac OS, or virtually anything else that plays audio or video content, you’ve paid for a codec. The price of codecs is built into the price of these products. As a very rough estimate, the codec price could be anywhere between $1 and $10, though it is difficult to know for sure because it’s often unclear which codecs you’ve paid for. Atmel provides a list of various codecs and their prices in “How to license audio and video codecs“.

Virtually all video hosting sites, such as YouTube, provide video with the Sorenson Spark codec and audio with the MP3 codec through the Adobe Flash plugin (I discuss the issues with this in Why I haven’t installed a Flash player). You may have noticed that you do not have to pay Adobe any money when you download the Flash player, even though it contains codecs. This is because Adobe pays for the codecs out of its own pocket; these costs are recouped by people buying Adobe’s developer tools. Because the Flash player contains patented codecs (and probably for other reasons), you are not allowed to give the Flash player software to someone else; they must download it from the Adobe web site so Adobe knows how much it has to pay patent holders.

Music playing software, such as Windows Media Player or iTunes, and MP3 players such as the iPod, all require a codec fee to be paid by the distributor. In the case of Windows Media Player and iTunes, the price of the codec is included in the price of Windows or Mac OS, which is in turn included in the purchase price of your computer. In the case of an MP3 player, the price of the codec is included in the price of the player.

There are many problems with patented codecs:

• Consumers and distributors have to pay for patented codecs in order to use them.
• It is difficult to know whether a particular audio or video file you have will work with someone else’s computer or device. They may not have the same codecs as you or they may not have any patented codecs at all.
• If a software developer chooses to write his or her own software that can read a codec such as MP3, that person could not give that software away for free even if they wanted to because they have to pay the patent holder.
• If a person chooses not to purchase an operating system such as Windows or Mac OS that includes patented codecs, but instead downloads a free operating system such as Ubuntu, they cannot view any content using patented codecs unless they obtain the codecs illegally or purchase them through software such as Fluendo Complete Playback Pack, which can be prohibitively expensive.
• Related to the above point, if a company is designing a new product but wants to keep costs as low as possible and also provide entirely redistributable software to its users, it cannot include any patented codecs. This is an issue for projects such as One Laptop per Child. For more information see their page on restricted formats and standards for software freedom

Seeing the problems with patented codecs, many software developers have chosen to create and freely distribute codecs that do not use any patents and thus do not require any royalty to be paid (ie. they are “royalty-free”). Organizations that do this include the Xiph.Org Foundation, which created the Vorbis audio codec and Theora video codec, and the BBC, which created the Dirac video codec. Anyone is free to download software that creates and plays audio and video using Vorbis and Theora or Dirac.

At first glance, there doesn’t appear to be a good reason why everyone hasn’t switched to royalty-free codecs. Unfortunately, a combination of a failure in the patent system and the business motives of Microsoft and Apple causes these companies to shy away from royalty-free codecs.

For every codec, there is some possibility that the codec uses a method for translating audio or video that is patented. This can occur even with codecs that were built without using any techniques the authors knew were patented because the authors didn’t read all the video codec patent documents carefully (which I wouldn’t blame them for; there are literally thousands of such documents) or because their interpretation of a patent document disagrees with what a court would decide. The authors also may have chosen not to look at any patent documents because if they did look at patents, they could be liable for three times the damages (treble damages) if they were found to infringe on a patent than if they had not looked at the documents.

To summarize, one can never guarantee that a particular codec does not infringe on patents. One can be somewhat certain that a codec does not infringe patents if there is a large company that is using the codec a lot and is not getting sued (since if there was a patent holder, they would likely have sued the company to get money from them). Even if a person examines every one of the millions of patents currently in effect, it is still possible that a court will disagree with his or her interpretation of a patent.

As a result, companies that do not wish to use royalty-free codecs can claim that the codecs are too risky because they have not yet been widely distributed by a large company (which is true in the case of Theora and Dirac). You can see an example of this in the controversy surrounding the Theora codec’s proposed inclusion in HTML5. Sometimes such a claim is legitimate, but often it can hide the real reason as I’ll discuss below.

Both Microsoft and Apple own patents on audio and video codecs so they are interested in collecting royalties from other companies for the use of these codecs. Because of this, they have an incentive keep royalty-free codecs out of their operating systems because these would compete with the codecs they hold patents. That is, people might start to use royalty-free codecs instead of the patented codecs so Microsoft and Apple would have a harder time trying to convince companies to use and pay for their patented codecs (since there would be less content out there using those codecs).

As we can see, neither of the major operating system vendors (Microsoft and Apple) are interested in supporting royalty-free codecs because their business interests preclude an unbiased stance on the issue.

You (users, content providers, and anyone else) should use royalty-free codecs because they are open standards, open for anyone to implement and anyone to use, without payment or fear of litigation. These are the reasons one should use any open standard (see Stephen Weber’s Beasts of the Standards World for a definition of “open standard”). By their nature, open standards foster innovation, leading to less expensive and more valuable products for consumers. I hope to expand on these points in a future post to explain the connection between open standards, innovation, and end-user benefits. You’ll just have to trust me on that one for now.

As I alluded to in the “Why everyone isn’t using royalty-free codecs” section, it is actually harder to use royalty-free codecs on Microsoft Windows and Apple’s Mac OS than it is to use patented codecs. For users on these operating systems, I recommend installing the VLC media player, which supports the Vorbis audio codec and the Theora and Dirac video codecs. Note that VLC is distributed for free, but contains some patented codecs so you should refrain from using those codecs to be on the safe side legally.

The unfortunate side-effect of the situation I described above is that few people will use royalty-free codecs on Windows and Mac OS because it involves an extra step that they aren’t interested in performing or don’t know about. Such is the situation until operating system vendors are convinced that focusing on royalty-free codecs and the wishes of their users will ultimately result in happier customers than focusing on selling patented codecs.

As a long-term solution to the codec problem and many other problems facing Windows and Mac OS users, I recommend using Ubuntu. Doing this will keep you free from the business interests of a few monopolists and bring you closer to a community that values appropriate technical and social solutions over squeezing every dollar out of customers.

With all of the issues facing royalty-free codecs, one might conclude that the situation is dire: codecs that are free for all to use will be squashed by big business and a patent system that doesn’t care about freedom. But there is hope. Here are a few of the trends that will eventually bring royalty-free codecs to everyone:

• The Vorbis codec is supported by an increasingly large number of portable music players. If you go into your local electronics store, chances are you’ll find several portable music players advertising “Ogg” or “Vorbis” support on their packaging.
• The Vorbis and Theora codecs will soon be distributed to all Firefox users (around 20% of all Internet users) when Firefox 3.1 rolls out with built-in support for them, as described on Why Ogg Matters and in the Firefox 3.1 features list. Firefox users will easily be able to view all the audio and video content on Wikipedia, which has chosen to use royalty-free codecs.
• The Dirac video codec, created by the BBC, is “pretty much ready now“, suggesting that the BBC will start rolling it out across their web sites soon. This may provide the requisite large company rollout test case for those concerned about patent risks.

To help spread royalty-free codecs, there are a number of things one can do depending on what role they play in the Internet ecosystem.

When you find a web site that only provides content through patented codecs, use their contact form to notify them that they should use royalty-free codecs and, if it affects you, tell them that you are unable to view their content because you don’t have patented codecs.

Make sure that any content you provide to others (through a web site, for example) is available through royalty-free codecs. If it is also available through patented codecs, that’s fine, as long as it is available through royalty-free codecs as well.

Content providers have a big responsibility in ensuring their content is available through royalty-free codecs because content providers have a monopoly on their content. Unless the content is freely-licensed, it is illegal for a user to translate the content into a royalty-free format themselves so it is crucial that the content providers do the translation into a royalty-free format.

If anyone with the power to make these decisions is listening, here is what would help royalty-free codecs the most:

• Inclusion of royalty-free codecs (specifically Dirac, Theora, FLAC, and Vorbis) in all supported versions of Microsoft Windows and Apple’s Mac OS X
• The option to view content using royalty-free codecs from major video hosting sites, most importantly YouTube, and news sites

Most people unknowingly use patented codecs on a regular basis. Patented codecs have a number of problems that stifle innovation. Royalty-free codecs already exist, but are impeded by the business motives of Microsoft and Apple, the two largest operating system vendors, and problems with the patent system. There is hope for royalty-free codecs thanks to various initiatives, including those of Firefox and the BBC. You can support royalty-free codecs by notifying sites when their content is not available through royalty-free codecs and by publishing content using royalty-free codecs.

The codec dilemma: Royalty-free codecs exist and solve many problems but they are not easily available to most computer users because Microsoft and Apple include only patented codecs in their operating systems.