Feature Story

Network Trailblazers: A Conversation with Luis Von Ahn

by Anne Field

Computer science innovator, Luis Von Ahn, chatted with Anne Field about his work harnessing human brainpower and computers to address problems neither can solve alone.

You know those annoying stretched out words you need to type in order to get access to Ticketmaster, Yahoo, or dozens of other places? They're called CAPTCHAs and they were developed about 12 years ago by a then-22-year-old Carnegie Mellon University computer science graduate student named Luis Von Ahn and his PhD advisor to stop spam bots. Their solution: make anyone seeking access to a site type in distorted and random letters or numbers, something spam bots couldn't read accurately, but humans could (albeit, not without emitting a few epithets in the process). Soon after, a flood of other companies started adopting CAPTCHAs.

Since then, Von Ahn, now an associate professor of computer science at Carnegie Mellon and the winner of a $500,000 MacArthur genius grant, has gone on to pioneer an entirely new field. Called human computation, it aims to use large numbers of networked people to address problems that computers can't solve on their own.

To that end, he also developed reCAPTCHAs, which are those equally annoying  pairs of words bisected by a line that you also might encounter while trying to get access to web sites. They're hard-to-decipher words taken from the Internet Archive's project to digitize thousands of books. By typing them in, you help the project complete its work. At the moment, computer users are helping to digitize about 100 million words a day or 200 million books a year, according to Von Ahn.

But he didn't stop there. Von Ahn also created a game, called the ESP Game, to help label thousands of images on the Internet, in which two strangers connected online have to agree on a description for a picture. Users have described it as an addictive experience akin to taking a drug. Von Ahn, who formed a company around the game, sold the business to Google in 2009. His current project, called Duolingo, which he's been working on for the past two years, is aimed at enticing millions of people to translate Web content into different languages for free.

I recently spoke with Von Ahn at his office on Carnegie Mellon's Pittsburgh campus. We talked over the phone while a videographer--my daughter, Eleanor Lewis, a junior at the university--filmed him. We discussed his various projects, from CAPTCHA to Duolingo, as well as his approach to human computation.


"We originally developed this (CAPTCHA) because Yahoo at the time had a huge problem. There were people who would write computer programs to obtain millions of e-mail accounts for free in order to send spam. The solution that we came up with was a test that could distinguish whether the entity filling out the form was actually a human and not a computer program. It worked because humans have no trouble reading these distorted characters, but the computer programs can't do that as well."

"Since we started working on it, basically every major website has started using a CAPTCHA. So you see them on Google, every Microsoft property, Yahoo, Facebook. Approximately 200 million  CAPTCHAs are typed every day by people around the world."


"The first time I heard about the number of people using CAPTCHAs, I was quite proud of myself. I thought, look at the impact my research has had. But then I started feeling badly, because each time you do a CAPTCHA, you waste about 10 seconds of your time. If you multiply that by 200 million, you get the whole of humanity wasting about 500,000 hours every day typing these annoying CAPTCHAs. Then I started thinking about if there was a way in which we could use this effort for something good for humanity. The thing is, during those 10 seconds, your brain is doing something amazing. Your brain is doing something that computers can't do very well."

"Another way of putting it is, there are humongous problems we cannot yet get computers to do. But if we split them into tiny 10 second-chunks, then whenever somebody would do a CAPTCHA, they would (help solve) a little bit of this problem. This is what we're doing now with reCAPTCHA, where not only are you authenticating yourself as a human when you type a CAPTCHA, but you also are helping us to digitize books."

"The way that works is as follows. There are many projects out there trying to digitize books, taking old books that were written and scanning them. Now, scanning a book is like taking a digital photograph of every page of the book , which gives you an image of each page. The next step is that the computer needs to decipher all of the words in that image using a technology called Optical Character Recognition, which basically takes the image of the text and tries to extract the text that's within there. The problem is that the computer often cannot recognize a lot of the words in older books. We take all the words the computer can't recognize in the digitization process and get people to read them for us while they type a CAPTCHA on the Internet."

"So now you know that the next time you see a CAPTCHA, the distorted words are actually words coming directly from books a computer couldn't recognize."

On human computation:

"This is a new kind of computer science I've been trying to develop for the last eight or nine years, along with other computer scientists. Basically, we're trying to find problems that computers cannot yet solve. It turns out there are a lot of things computers cannot yet do, despite movies in Hollywood telling you otherwise. So we get problems that computers can't do, and we try to get people to solve them for us while they do other things on the Internet."

On the ESP Game:

"The basic idea of the ESP  Game, later called the Google Image Labeler, is that computers can't recognize the contents of images. You know, if you take an arbitrary image, the computer can't tell you if it's, say, a dog or a cat. Being able to label the contents of images on the Internet is very important. For example, it allows for image search. So, if you go to Google and type in the word "dog", you get back a lot of images related to the word "dog". Since there is no computer program that can tell you whether that image from the web contains a dog, the way Google images used to work is by searching through filenames. The problem is, it didn't work very well, since the computer couldn't recognize what's inside the images."

"So, what we did was turn the process into something that people wanted to do. We created a game that people really, really wanted to play. And as people played, they actually told Google what were the contents of images on the web. And by using these tags Google improves the image search engine. We turned the process into a really fun game which was able to get  literally millions of people to play it. And as they were playing, they were actually labeling or tagging all the images on the web."

"People really love it. Many have played it over 20 hours a week. Some people have played it 40 hours a week."

On Duolingo:

"We started by thinking: How can we translate the whole web into every major language? Right now, a very large fraction of the web is in English. And if you don't know English, you can't access it. Also there are many other languages where, if you don't know that language, you can't access the content."

"First, we tried to see if we could use computer translations. But if you really, really want to translate everything in a way that is correct and readable, it turns out that computers can't yet do that very well. Then we started thinking: How can we get people to translate the whole web into every major language? But not just 10 people…what we wanted was literally millions of people to help us translate the web into every major language."

"We started thinking how to motivate people to do this. One possibility was to hire professional language translators. The problem is that would be extremely expensive. Just translating Wikipedia, a tiny fraction of the web, would cost over $50 million in just one language. We started thinking, how can we motivate people to do this for free. And we realized there were two pretty major obstacles. One is the lack of bilinguals. The other, how to motivate people to do something for free that normally they would be paid to do."

"We figured there was a way to solve both of these problems and to transform language translation into something that millions of people would want to do. And that was through language education. Today, there are over 1 billion people in the world learning a foreign language. With our project, we developed a language learning website. But as you learn, at the same time, you help to translate the web. At the very beginning, you get very simple sentences to translate. As you get better at it, we give you more complex sentences. You're basically getting better at the language, at the same time that you are translating stuff."