The codec dilemma

Every day millions of videos are watched on video hosting sites like YouTube. Most viewers don’t think of how the videos are stored or how they get displayed on one’s computer. But these are critical questions in determining who is able to watch online videos and how much money they need to pay to do it, even on free hosting sites. Let me start by explaining what a “codec” is.

What is a codec?

A video codec is a method of encoding and decoding video so it can be read from and stored on a computer. A good codec is one that produces files with high video quality that take up very little space on disk. Many different codecs exist with varying levels of quality and space usage. Wikipedia has a list of video codecs. Audio codecs, such as MP3, are similar to video codecs but for representing audio. Wikipedia also has a list of audio codecs.

Why you pay for codecs and how much you pay

Most audio and video codecs contain some encoding and decoding methods that are patented. A patent is a set of exclusive rights, including the right to sue anyone using a particular invention, granted by governments in exchange for disclosing that particular invention. Usually an organization will patent an invention so they can charge money to people who want to use their invention. This is the case with most audio and video codecs: organizations charge money to people that use their codecs because the organizations have acquired patents on particular parts of their codecs.

You may be thinking that you’ve never paid for a codec, but in fact, you probably have. If you’ve bought a DVD player, a Blu-ray player, an iPhone, an iPod, an MP3 player, a computer running Windows or Mac OS, or virtually anything else that plays audio or video content, you’ve paid for a codec. The price of codecs is built into the price of these products. As a very rough estimate, the codec price could be anywhere between $1 and $10, though it is difficult to know for sure because it’s often unclear which codecs you’ve paid for. Atmel provides a list of various codecs and their prices in “How to license audio and video codecs“.

Where patented codecs are used and how they are paid for

Virtually all video hosting sites, such as YouTube, provide video with the Sorenson Spark codec and audio with the MP3 codec through the Adobe Flash plugin (I discuss the issues with this in Why I haven’t installed a Flash player). You may have noticed that you do not have to pay Adobe any money when you download the Flash player, even though it contains codecs. This is because Adobe pays for the codecs out of its own pocket; these costs are recouped by people buying Adobe’s developer tools. Because the Flash player contains patented codecs (and probably for other reasons), you are not allowed to give the Flash player software to someone else; they must download it from the Adobe web site so Adobe knows how much it has to pay patent holders.

Music playing software, such as Windows Media Player or iTunes, and MP3 players such as the iPod, all require a codec fee to be paid by the distributor. In the case of Windows Media Player and iTunes, the price of the codec is included in the price of Windows or Mac OS, which is in turn included in the purchase price of your computer. In the case of an MP3 player, the price of the codec is included in the price of the player.

Problems with patented codecs

There are many problems with patented codecs:

  • Consumers and distributors have to pay for patented codecs in order to use them.
  • It is difficult to know whether a particular audio or video file you have will work with someone else’s computer or device. They may not have the same codecs as you or they may not have any patented codecs at all.
  • If a software developer chooses to write his or her own software that can read a codec such as MP3, that person could not give that software away for free even if they wanted to because they have to pay the patent holder.
  • If a person chooses not to purchase an operating system such as Windows or Mac OS that includes patented codecs, but instead downloads a free operating system such as Ubuntu, they cannot view any content using patented codecs unless they obtain the codecs illegally or purchase them through software such as Fluendo Complete Playback Pack, which can be prohibitively expensive.
  • Related to the above point, if a company is designing a new product but wants to keep costs as low as possible and also provide entirely redistributable software to its users, it cannot include any patented codecs. This is an issue for projects such as One Laptop per Child. For more information see their page on restricted formats and standards for software freedom
Royalty-free codecs

Seeing the problems with patented codecs, many software developers have chosen to create and freely distribute codecs that do not use any patents and thus do not require any royalty to be paid (ie. they are “royalty-free”). Organizations that do this include the Xiph.Org Foundation, which created the Vorbis audio codec and Theora video codec, and the BBC, which created the Dirac video codec. Anyone is free to download software that creates and plays audio and video using Vorbis and Theora or Dirac.

Why everyone isn’t using royalty-free codecs

At first glance, there doesn’t appear to be a good reason why everyone hasn’t switched to royalty-free codecs. Unfortunately, a combination of a failure in the patent system and the business motives of Microsoft and Apple causes these companies to shy away from royalty-free codecs.

Patent system failures

For every codec, there is some possibility that the codec uses a method for translating audio or video that is patented. This can occur even with codecs that were built without using any techniques the authors knew were patented because the authors didn’t read all the video codec patent documents carefully (which I wouldn’t blame them for; there are literally thousands of such documents) or because their interpretation of a patent document disagrees with what a court would decide. The authors also may have chosen not to look at any patent documents because if they did look at patents, they could be liable for three times the damages (treble damages) if they were found to infringe on a patent than if they had not looked at the documents.

To summarize, one can never guarantee that a particular codec does not infringe on patents. One can be somewhat certain that a codec does not infringe patents if there is a large company that is using the codec a lot and is not getting sued (since if there was a patent holder, they would likely have sued the company to get money from them). Even if a person examines every one of the millions of patents currently in effect, it is still possible that a court will disagree with his or her interpretation of a patent.

As a result, companies that do not wish to use royalty-free codecs can claim that the codecs are too risky because they have not yet been widely distributed by a large company (which is true in the case of Theora and Dirac). You can see an example of this in the controversy surrounding the Theora codec’s proposed inclusion in HTML5. Sometimes such a claim is legitimate, but often it can hide the real reason as I’ll discuss below.

Business motives

Both Microsoft and Apple own patents on audio and video codecs so they are interested in collecting royalties from other companies for the use of these codecs. Because of this, they have an incentive keep royalty-free codecs out of their operating systems because these would compete with the codecs they hold patents. That is, people might start to use royalty-free codecs instead of the patented codecs so Microsoft and Apple would have a harder time trying to convince companies to use and pay for their patented codecs (since there would be less content out there using those codecs).

As we can see, neither of the major operating system vendors (Microsoft and Apple) are interested in supporting royalty-free codecs because their business interests preclude an unbiased stance on the issue.

Why you should use royalty-free codecs

You (users, content providers, and anyone else) should use royalty-free codecs because they are open standards, open for anyone to implement and anyone to use, without payment or fear of litigation. These are the reasons one should use any open standard (see Stephen Weber’s Beasts of the Standards World for a definition of “open standard”). By their nature, open standards foster innovation, leading to less expensive and more valuable products for consumers. I hope to expand on these points in a future post to explain the connection between open standards, innovation, and end-user benefits. You’ll just have to trust me on that one for now.

Practical issues with using royalty-free codecs

As I alluded to in the “Why everyone isn’t using royalty-free codecs” section, it is actually harder to use royalty-free codecs on Microsoft Windows and Apple’s Mac OS than it is to use patented codecs. For users on these operating systems, I recommend installing the VLC media player, which supports the Vorbis audio codec and the Theora and Dirac video codecs. Note that VLC is distributed for free, but contains some patented codecs so you should refrain from using those codecs to be on the safe side legally.

The unfortunate side-effect of the situation I described above is that few people will use royalty-free codecs on Windows and Mac OS because it involves an extra step that they aren’t interested in performing or don’t know about. Such is the situation until operating system vendors are convinced that focusing on royalty-free codecs and the wishes of their users will ultimately result in happier customers than focusing on selling patented codecs.

As a long-term solution to the codec problem and many other problems facing Windows and Mac OS users, I recommend using Ubuntu. Doing this will keep you free from the business interests of a few monopolists and bring you closer to a community that values appropriate technical and social solutions over squeezing every dollar out of customers.

Hope for royalty-free codecs

With all of the issues facing royalty-free codecs, one might conclude that the situation is dire: codecs that are free for all to use will be squashed by big business and a patent system that doesn’t care about freedom. But there is hope. Here are a few of the trends that will eventually bring royalty-free codecs to everyone:

  • The Vorbis codec is supported by an increasingly large number of portable music players. If you go into your local electronics store, chances are you’ll find several portable music players advertising “Ogg” or “Vorbis” support on their packaging.
  • The Vorbis and Theora codecs will soon be distributed to all Firefox users (around 20% of all Internet users) when Firefox 3.1 rolls out with built-in support for them, as described on Why Ogg Matters and in the Firefox 3.1 features list. Firefox users will easily be able to view all the audio and video content on Wikipedia, which has chosen to use royalty-free codecs.
  • The Dirac video codec, created by the BBC, is “pretty much ready now“, suggesting that the BBC will start rolling it out across their web sites soon. This may provide the requisite large company rollout test case for those concerned about patent risks.
What you can do…

To help spread royalty-free codecs, there are a number of things one can do depending on what role they play in the Internet ecosystem.

…as a user

When you find a web site that only provides content through patented codecs, use their contact form to notify them that they should use royalty-free codecs and, if it affects you, tell them that you are unable to view their content because you don’t have patented codecs.

…as a content provider or creator

Make sure that any content you provide to others (through a web site, for example) is available through royalty-free codecs. If it is also available through patented codecs, that’s fine, as long as it is available through royalty-free codecs as well.

Content providers have a big responsibility in ensuring their content is available through royalty-free codecs because content providers have a monopoly on their content. Unless the content is freely-licensed, it is illegal for a user to translate the content into a royalty-free format themselves so it is crucial that the content providers do the translation into a royalty-free format.

What would help the most

If anyone with the power to make these decisions is listening, here is what would help royalty-free codecs the most:

  • Inclusion of royalty-free codecs (specifically Dirac, Theora, FLAC, and Vorbis) in all supported versions of Microsoft Windows and Apple’s Mac OS X
  • The option to view content using royalty-free codecs from major video hosting sites, most importantly YouTube, and news sites
Summary

Most people unknowingly use patented codecs on a regular basis. Patented codecs have a number of problems that stifle innovation. Royalty-free codecs already exist, but are impeded by the business motives of Microsoft and Apple, the two largest operating system vendors, and problems with the patent system. There is hope for royalty-free codecs thanks to various initiatives, including those of Firefox and the BBC. You can support royalty-free codecs by notifying sites when their content is not available through royalty-free codecs and by publishing content using royalty-free codecs.

The codec dilemma: Royalty-free codecs exist and solve many problems but they are not easily available to most computer users because Microsoft and Apple include only patented codecs in their operating systems.

6 Responses to “The codec dilemma”


  • If a person chooses not to purchase an operating system such as Windows or Mac OS that includes patented codecs, but instead downloads a free operating system such as Ubuntu, they cannot view any content using patented codecs unless they obtain the codecs illegally or purchase them

    I’m not sure this is true in Canada, though it’s probably true in the U.S. I heard that in Canada it’s legal to download copyrighted material, but not to provide it. The providers of such software may be breaking the law, but I don’t consider it immoral for me to use that software, because I don’t believe in patenting software.

    But I agree that we should fight for royalty-free codecs by avoiding the use of patented codecs, and I wasn’t aware of much of that stuff. Thanks for bringing it to my attention.

  • I’m not sure this is true in Canada, though it’s probably true in the U.S. I heard that in Canada it’s legal to download copyrighted material, but not to provide it.

    This is, for most interpretations of a recent court case, true. For more information, see:

    http://blogs.itworldcanada.com/insights/2008/08/05/unauthorized-music-filesharing/#comments

    However, I still don’t believe it’s right; future court cases may overrule this pseudo-precedent.

    In any case, that ruling has nothing to do with patents (only copyright) so it doesn’t really influence the issue of distributing implementations of patented codecs.

    The providers of such software may be breaking the law, but I don’t consider it immoral for me to use that software, because I don’t believe in patenting software.

    Usually, the software that Ubuntu users use to view patented codecs is “free software” in the sense that it is licensed under a free software license (such software includes FFmpeg and mplayer). However, it is not “free” in the sense that you are free to redistribute it without fear of litigation from patent holders. I recommend reading the FFmpeg FAQ on the issue:

    http://ffmpeg.org/legal.html

    I’m not entirely sure if the onus is always on the distributor to pay royalties or whether it is sometimes on the user. The VLC FAQ claims the onus is on the user when the distributor doesn’t pay:

    http://www.videolan.org/doc/faq/en/index.html#id365856

    In any case, I would rather steer clear of patented codecs to avoid forcing anyone to break the law.

  • So, in short if I were to use VLC to watch something using the MP3 and AVI codec, I would be doing something illegal?

  • Assuming you had not received explicit permission from the patent holders, then most likely yes. This does depend a bit on whether the particular patents are registered in your country, though.

    To be clear, you will probably not be sued for doing this. It is usually not worth the patent holders’ time to go after individual infringers. I’m not aware of any cases where a patent holder sued an individual for infringing. But the point is that they could, so it is always best to use royalty-free codecs.

    As a side note, AVI is not a codec, it is a container format. It can contain audio and video data using many different codecs, including MP3 and H.264. I assume in this response that you mean an AVI file containing audio encoded with MP3 and video encoded with H.264 or some other royalty-requiring video codec.

  • You _do_ realize that, as a user, noone is willing to be unable to view some content just because they dont have $10-worth of codec in their system, dont you. And if I am mad enough, and have all the videos encoded in theora, how would I share any of this with friends who only have patented codecs? There is _no_ reason for user to use obscure but free format, neither for audio/video, nor for anything else.

Leave a Reply to Andrew