Amazon Mechanical Turk: The Digital Sweatshop

1 / 3
Microtasking works by outsourcing small, virtual tasks to an army of online workers, who then perform them for pennies.
Microtasking works by outsourcing small, virtual tasks to an army of online workers, who then perform them for pennies.
2 / 3
Jeff Bezos described microtasking elegantly, as “artificial artificial intelligence,” humans behaving like machines behaving like humans.
Jeff Bezos described microtasking elegantly, as “artificial artificial intelligence,” humans behaving like machines behaving like humans.
3 / 3
If computers couldn’t do the work, why not hire humans to do it for them?
If computers couldn’t do the work, why not hire humans to do it for them?

The funny thing about the biggest shift in production in years is that almost nobody knows it happened. Which makes sense, if you think about it: It occurred invisibly, online, anonymously, all over the world, but at the same time, nowhere in particular. And it’s poised–if most people who know about it are to be believed–to completely change the way we think about work, the way we consume technology, and the way the global economy functions.

It’s called microtasking, and it works by outsourcing small, virtual tasks to an army of online workers, who then perform them for pennies. These tasks vary widely in scope and substance, but what links them all is that they’re essentially too difficult or too dependent on human analysis for a computer to do, but too simple for skilled labor. And they’re the bedrock of the internet.

Crowdsourced microtasking–conducted largely via’s Mechanical Turk site–is now a multimillion-dollar industry, and one that doesn’t appear to be slowing down anytime soon. Even as the global economy continues to falter, Turk is thriving, due in no small part to what it can do for companies under pressure to do more with less.

“There’s this sort of competitive insanity of the business environment,” said Six Silberman, a longtime observer of the field who helped create a forum, Turkopticon, for people doing this kind of work. “And everyone’s trying to cut costs as strenuously and as rapidly as possible.” In a globalized economy, that’s easy to do: Mechanical Turkers–even those who live in the U.S.–make somewhere around $1.50 an hour on average, enjoy no worker protections, and have no benefits.

Perhaps unsurprisingly, since Mechanical Turk’s inception, critics have emerged from all corners of the labor, law, and tech communities. Labor activists have decried it as an unconscionable abuse of workers’ rights, lawyers have questioned its legal validity, and academics and other observers have probed its implications for the future of work and of technology.

But at the same time, crowdsourcing has been hailed as a solution to one of the greatest problems of the 21st century: the massive volume of information provided to us by the internet, and the equally large difficulty associated with categorizing it. Technologists have praised Amazon Mechanical Turk for its efficiency, activists for its ability to employ people in the developing world, economists for its promise of creating new ways for people to supplement their incomes. On a 2008 NPR broadcast, Wendy Kaufman went so far as to call it “the biggest paradigm shift in innovation since the Industrial Revolution.”

One of the first and foremost observers of the crowdsourcing phenomenon was Jeff Howe, a tech reporter who coined the term in a 2006 Wired article and has since written a book on the subject, aptly titled Crowdsourcing. “[Turk is] both rather depressing and rather brilliant,” Howe wrote on his blog in November 2006. But if, back then, crowdsourced microtasking was still something of an open question, it’s now a force to be reckoned with. In seven years, Turk and its imitators have gone from experiment to emerging field to major industry.

The name “Mechanical Turk” connotes two legends–one old, one recent. The first one goes like this: In the midst of the Industrial Revolution, the Hungarian author and inventor Wolfgang von Kempelen invented what he touted as a fantastic technological innovation–a machine that could not only play chess against humans, but could beat some of the best players in the world. The Mechanical Turk eventually toured Europe, beating out Benjamin Franklin and Napoleon Bonaparte. It took 50 years for anyone to discover the hoax: Inside it was a human chess expert, using magnets to move the pieces.

More than two centuries later, in the midst of the technological revolution, Amazon’s Mechanical Turk was born. Its inventors, too, touted it as a fantastic technological innovation, and its story is no less compelling, though it’s compelling in a distinctly 21st-century way. In 2005, having built millions of web pages for its various products, the online giant was faced with the problem of finding which ones were duplicates–a task that, for various reasons, confounded computer algorithms, but that a human could easily do in seconds. Amazon founder Jeff Bezos had a clever solution: If computers can’t do the work, why not hire humans to do it for them–to act, essentially, as another part of the software, performing infinitesimal, discrete tasks, often in rapid succession? He described it, elegantly, as “artificial artificial intelligence”–humans behaving like machines behaving like humans.

Amazon Mechanical Turk eventually expanded to include work for other contractors; now, it’s spawned a host of imitators and become a massive market, with well over 200,000 workers. The industry as a whole made revenues of about $375 million in 2011–up 75 percent from the previous year. Within that field, the fastest growing segment is microtasking, which more than doubled its revenues last year. The nature of microtasking is such that the scope of its effect on the global job market can never fully be known, but it’s safe to say it’s huge.

Part of Mechanical Turk’s appeal is its simplicity: Employers (or “requesters,” in Turk’s jargon) post jobs (“Human Intelligence Tasks,” or “HITs”), specifying each HIT’s pay and duration. At the top of the site’s dashboard, there’s a live ticker of the number of HITs available; on any given day, or even at the middle of the night, there are upward of 100,000 different ones available. And that’s it: There’s no application process, no exchange of information beyond the most basic facts. Turkers don’t fill out tax forms, because they don’t make enough money, and the job contract comes in the form of what’s known as a “clickwrap” agreement–that is, a quick page of fine print at the end of which a user clicks “agree.”

After accepting a task, a provider has a set time in which to complete it, entirely online, and once he or she does, it goes to the requester for review. If it’s approved, money gets deposited in either the provider’s bank or account. Most tasks pay anywhere from a cent to a couple dollars, based on time and difficulty; Amazon takes a 10-percent cut for each transaction greater than one cent, and 50 percent for those that pay a penny.

The classic example of a Turk task is one of the ones Amazon originally built the system for: sorting merchandise into categories based on color or style for the site’s massive online warehouse. But at this point, the only real theme linking all of Turk’s tasks is the fact that they can be done virtually, by a labor force that’s willing to be paid pennies and that can’t expect to be paying too much attention. Companies use Turk to look up foreign zip codes, transcribe podcasts, match web sites to relevant search terms, categorize images based on their subject, and flag objectionable content on web sites; when I tried Turk myself, my task was to copy and paste information from a scanned business card into fields for name, email address, phone number, and the like–presumably for some kind of directory, though I, of course, had no idea where my work was going.

According to a study by Panos Ipeirotis, an NYU business professor who’s written at length about Mechanical Turk, some 40 percent of Turk’s tasks are actually related to the creation and dissemination of spam–if you’ve ever been sent a bizarre link on Twitter or Facebook from a stranger, chances are pretty good that the stranger is a Turker. But at the same time, Turkers are essential to the work of organizing the vast and growing wealth of information the internet age has given us; all told, including Google and Facebook, it’s all but impossible to interact with the world wide web for any meaningful amount of time without encountering the fruits of microtasked labor–whether you know it or not.

Because Mechanical Turk’s labor force is decentralized, anonymous, and invisible, reliable statistics about its demographics are hard to come by, but this is what we know: A 2010 survey by Ipeirotis suggested that 46.8 percent of Turk’s labor force lives in the U.S., 34 percent lives in India, and 19.2 percent lives elsewhere. In an earlier study, Ipeirotis found that the median annual income for American Turkers was somewhere between $25,000 and $40,000, and for Indian Turkers below $10,000. He also found that 49 percent of Turkers surveyed were doing so for “income purposes.” His study participants included a senior looking to supplement his or her fixed income; a laid-off accountant making $150 to $200 a week to stay afloat while looking for a new job; a schoolteacher trying to make ends meet–as well as many people who were on the site for more abstract, less financially immediate reasons.

What Ipeirotis’ work reveals, more than anything, is that the Amazon Mechanical Turk workforce is far from a monolith. But it also shows that a not-insignificant number of people, in this country and abroad, aren’t just using Turk as a means to make a couple bucks here and there–they’re using it to replace a job.

Robin, who asked to be identified only by her first name, started Turking a couple years ago as a means of supplementing her salary. She liked the idea of working from home, and of being able to set her own schedule and pick her own tasks. She quickly found that it wasn’t all that simple, though. “They make it sound like you can just do a few tasks in your free time in between other things,” she wrote in an email. “But if you worked like that, I believe you would make about a dollar a day.”

Rob (whose name has been changed) started Turking at around the same time Robin did, after reading about the site on a blog. He already had a full-time job in tech and didn’t need the money, but he figured this was a means of making some pocket change during the time he’d be watching TV anyway. Since then, he has completed some 2,000 tasks over countless hours, with a near-perfect 98 percent approval rate. In that entire time, he’s brought in a grand total of $157.

“It’s not worth it at all,” he said, his tone halfway between sheepish and matter-of-fact. “Return an aluminum can and you’ll make more money.” Of the half-dozen current and former Turkers I spoke to for this story, none said they made more than a couple dollars an hour, and the vast majority earned far less. “The money earned is so minimal, it’s laughable,” said Robin, who stopped Turking as soon as she no longer needed to financially. “You really have to be working all day long at top speed to earn minimum wage.”

Even that might be an overstatement: Ipeirotis has estimated the average hourly wage to be roughly $2, while Joel Ross of UC Irvine’s Department of Informatics places it closer to $1.25–and whatever it is, it’s certainly lower than the federal minimum wage of $7.25.

But that’s not all: Embedded in Amazon’s 5,200-plus-word Terms of Use agreement is a clause that essentially allows employers to reject an employee’s work whenever they want, no questions asked. Talk to enough Turkers and most of them will relay some version of the same story: They completed a task, all or part of which was rejected, and never found out why.

When Robin’s work was rejected, she knew exactly what had happened. She emailed an Amazon representative to try to get the issue sorted out–less because of the money and more because of what a mistake can mean: “An error like that lowers your score and makes you less desirable as a worker,” she explained. She sent customer service two messages and never received a response.

This is all part of a phenomenon economists call “information asymmetry,” and it’s a fundamental part of Turk’s system–as well as what draws detractors to it. Workers see only what requesters want them to see. Their work can be rejected for reasons unknown, but they’ll never know if the requester used it anyway and was just trying to stiff them. They can be effectively blacklisted, via low approval ratings. They could, theoretically, be working for companies whose policies or politics they don’t agree with. And they have no meaningful recourse for any of this. That imbalance of power is a big part of why Silberman along with Lilly Irani, a UC Irvine informatics PhD candidate, founded Turkopticon: as a means of allowing Turkers to do the work of weeding out bad requesters that Amazon refused to. It’s a big problem for Verlene Jones, the Western Regional Director of the United Association for Labor Education, as well, insofar as the more opaque a system is, the fewer protections employees have: “The concern is accountability,” she said. “This sounds like an opportunity for major abuses to happen to workers.”

Part of what makes Amazon Mechanical Turk so fascinating–and so problematic–is that, legally speaking, it’s basically operating in uncharted territory.

Alek Felstiner was among the first legal scholars to tackle this issue. “The laws that exist governing the employment relationship don’t apply here,” he said in a recent interview. Employment law, as it was written, had no need to make rules for workers who were anonymous–because until recently, working remotely wasn’t possible. There was no need to create payment regulations for micro-contracting–because micro-contracting didn’t exist. The law also didn’t make room for labor relationships that could be created and dissolved instantaneously. “Our existing framework is not completely useful,” Felstiner said.

That said, it doesn’t mean that Turkers may not be considered employees of the requesters in the eyes of the law. But in order to figure that out–and, consequently, Turkers’ rights to minimum wages, employee protections, and the like–someone needs to start asking questions. It’s just that they haven’t yet. Essentially, any number of entities or events could trigger regulation of the crowdsourcing industry: An agency like the IRS or the labor department could decide to investigate it, a class-action suit could prompt a court decision, Congress could adopt legislation, or the industry itself could even decide to set its own best practices. But thus far, no one’s taken it on. And until someone does, the industry exists in a sort of legal no-man’s land. “We don’t know if it’s legal, because there haven’t been any cases,” Felstiner said.

A few years ago, when Andrew Norman Wilson was working as a contractor for Google at the company’s sprawling Mountain View campus, he noticed something odd: A trickle of workers leaving a building adjacent to his, just as he was getting to work. Later, Wilson discovered that these employees–who worked in the literal and figurative dark, hidden from the rest of the campus and given none of the high-profile perks that people like Wilson had–were responsible for scanning each page in Google’s massive online books library.

“It takes a human hand to do this,” said Wilson, over the phone from his home in Chicago. “Each page has to be scanned individually.” Wilson sees this kind of labor as much more closely in line with that of the industrial proletariat than with the popular image of shiny, happy Googlers eating free, sustainably-farmed sushi and zipping around on rainbow-colored bikes. “It was just strange to me, for a company that’s prided on being such a great employer, that they would create this kind of secret and marginalized class–when this class is doing some of the more grueling work on campus, the more tedious work on campus.”

Google Books and microtasking aren’t quite the same thing, of course, but they illustrate similar points: about the factory floor lurking just below our internet, about consumers’ ignorance–willful, or, more likely, unintentional–of the human labor that goes into the internet, and about the lengths companies will go to keep things that way. Many of the labor activists and scholars I spoke to for this story had never heard of Amazon Mechanical Turk, nor had several of the tech employees I reached out to–even ones who work at companies that employ microtaskers by the hundreds of thousands. Like Google Books employees, microtaskers are, for the most part, invisible. And that makes them easy to ignore.

“We don’t see where the things that we use come from on an ongoing basis,” said Silberman, of Turkopticon. “I don’t know who made my phone; I don’t know who made the yogurt I had for lunch.” And tech is even more abstract and opaque: It’s not just that we don’t know who’s making it; oftentimes, we don’t even know how it’s made.

In other words: Nobody wants to see how his internet sausage gets made–and nobody making it wants to tell us. “Many companies have an incentive to look away from the issue,” said Anand Kulkarni, CEO and cofounder of MobileWorks, a company that’s working on creating a more sustainable model of crowdsourcing. “We’re still a little bit far away in society [to] understanding the true costs of unethical crowdsourcing, and the extent that it exists.”

Tapan Parikh was one of Kulkarni’s professors at the UC Berkeley School of Information, and he’s also been trying to examine and improve upon crowdsourcing as it currently exists. “As technologists, we’re so trained with an efficiency-first mind-set, so that immediately becomes the first and most important metric for us,” he said. “It’s a blind spot, and a limitation of the field.”

Part of what Parikh’s getting at is this: The internet is a utility, but it’s also a business–one that strives, like all others, to keep costs down and profits up, and which is faced with what’s increasingly looking like an unrealistic consumer expectation about access and price. It’s something of a new twist on the age-old adage that information wants to be free–as in, not just liberated, but gratis. We find ways to circumvent newspaper paywalls, pirate all our music, and borrow our friends’ HBOGO passwords in order to watch premium shows for free. And that attitude, coupled with a global workforce that’s willing to work for a pittance, has spawned a system in which aggressive cost-cutting is a foregone conclusion.

At the end of the day, tech companies are just that: companies, with bottom lines to attend to and, often, shareholders or investors to please. But unlike many other companies, they’re sometimes more like bureaucracies than businesses, and their practices can and do have implications for the global economy, whether the companies realize it or not. Just as when Yahoo justified hacking Chinese activists’ email accounts by arguing that this was, essentially, the cost of doing business with China, maybe there’s an allegory to be made here: In a market that seems pitched toward an inexorable race to the bottom, using cheaper-than-cheap labor is simple cost-effectiveness.

“There are surprisingly few conversations about ethics happening around here,” said one employee of a local tech company that works with crowdsourced labor. “You’d be shocked.”

If microtasking could be said to have a celebrity spokesman, it would probably be Lukas Biewald. Young, attractive, and charismatic, Biewald’s something of a Silicon Valley dream: He’s got a quick smile, perpetually mussed hair–and an armful of patents to his name.

Biewald talks about this stuff with the ease of someone who is either strongly invested in what he’s doing or is used to defending himself–or, perhaps most accurately, both. “Nobody’s getting tricked here,” he said. “It’s very, very clear what’s going on with this kind of work. And it’s really hard to coerce people to do something through a computer screen. People are choosing to do this.” CrowdFlower conducts worker-satisfaction studies regularly, he said–“and we constantly find that over 90 percent of them are satisfied with what they’re doing.”

The counterargument to that is, of course, that a job doesn’t need to be involuntary to be exploitative, but Biewald has a bigger point. “I find it interesting that the people who complain”–meaning, mostly, academics, activists, and tech-industry observers–“are not the workers,” Biewald said. “And the workers would actually be pretty pissed off if the people complaining were successful.”

That’s perhaps what’s most unsettling about all of this: The workers aren’t complaining–at least not publicly. If they did, it’d be easier for lawyers like Felstiner to regulate; easier for labor activists like Jones to decry; easier for all of us to wrap our heads around. If it’s hard to understand that people across the world and in our own backyards are doing this kind of labor for these kinds of wages, it’s even harder to understand why many of them don’t seem to mind. In a stratified world, exploitation is relative, and what’s appalling to progressive-minded Americans may be a godsend for people in developing nations.

That’s the point you’re most likely to hear from requesters: William Franceschine of, which pays workers to test new online games, brought it up almost immediately when I asked him if he had any moral qualms with using Turk. And at the same time, Samasource, a nonprofit that brings crowdsourcing and other computer jobs to people living in poverty, has furthered the idea of crowdsourcing as a social good, or at least a more productive kind of charity: On its homepage, overlaid over a picture of two smiling Southeast Asian women, neat font implores visitors to “save your business time and money while doing good.”

It’s an attractive narrative, to be sure, but it’s not so simple. As Irani pointed out, only certain segments of the population can participate in crowdsourced labor since it requires English-language skills, a computer (or, in some cases, a mobile phone), an internet connection, and enough Western-cultural capital to be able to say, for example, what the difference between a sneaker and a dress shoe is. “Whenever people talk about Amazon Mechanical Turk, they’ll say that Americans do this for fun and poor people in the Third World do this because it’s a good salary,” said Irani. “And both of these things are meant to stop questions about what the actual working conditions of actual people are.”

On the eleventh floor of an office building in downtown Berkeley, a few UC Berkeley School of Information alumni think they may have found a different way. MobileWorks isn’t, according to Kulkarni, the anti-Turk–it’s more like Turk 2.0. Kulkarni and his cofounders are mostly computer scientists by training, and they’re big believers in both the promise and the inevitability of crowdsourced work–they just think it can be better. “Turk was a great idea, but it was a nightmare to use,” explained Kulkarni–from both sides of the equation.

A couple years ago, Kulkarni himself went on Turk, just to see how long it would take him to earn a dollar. It took more than an hour. “It was incredibly painful,” he said. “It’s so difficult to earn a living wage.” The irony is that requesters aren’t even necessarily getting what they want out of it either. “You’d get right answers sometimes, wrong answers sometimes, and you’d never know if your results would get completed,” said Kulkarni.

But Kulkarni and his co-workers think they’ve found a solution to that, and it’s a simple one: “We believe that as soon as you’re paying people a fair living wage, they don’t need to cheat the responses,” he said. “They’re just a lot less likely to scam the system.” MobileWorks isn’t a charity, like Samasource, which means it’s intended to work within the system, rather than alongside it. Employees work from mobile phones and cheap laptops; the majority of them are women, and all of them are based in the developing world. They get paid roughly twice as much as Turkers do, according to the company.

As Irani and others have pointed out, relegating crowdsourcing to other parts of the world isn’t as easy as it sounds, and it isn’t an automatic absolution. The cost of living in Pakistan may be much lower than it is here, but it’s not free, and low wages are still low wages. Furthermore, relying on organizational apparatuses in far-flung countries with divergent regulatory practices can open a company up to all kinds of abuses. And finally, there’s the question of sustainability. “One of the big questions is, is this a viable career trajectory?” asked Parikh. “Will people be able to make careers out of crowdsourced work?” Parikh has acted as an adviser to Kulkarni and his cofounders, and he believes MobileWorks and others like it are a start.

Like many in the field, Kulkarni and Parikh take it as a given that the microtasking industry will only increase in scope and size. And in a weird way, the very things that make it so scary–its ubiquity, its efficiency–may be what prompt a shift in the field. After all, it may not be long before microtasking isn’t just relegated to the developing world and pockets of out-of-work Americans, but to all of us. Crowdsourcing, broadly defined, has already spread to more skilled labor sectors, like computer programming, writing, and web design, and Kulkarni–as well as most of the people I spoke to–assumes that trend will continue. At this point, it seems, the question shouldn’t be whether crowdsourcing should be done but rather how it should be done.

Ellen Cushing is a Staff Writer and Web Editor of East Bay Express. Excerpted from East Bay Express (August 1, 2012), an Oakland, California-based alternative weekly newspaper covering news, arts, and dining for Oakland, Berkeley, and the greater East Bay.

  • Published on Dec 7, 2012
In-depth coverage of eye-opening issues that affect your life.