The “Invisible Army” of Developers: How Thousands of Freelancers Around the World Teach Claude Code to Write Code Correctly
To improve the performance of Claude Code, Anthropic uses feedback from nearly 1,000 software engineers.
To improve the performance of Claude Code, Anthropic uses feedback from nearly 1,000 software engineers.
AI companies like Anthropic often outsource their data work to third-party contractors, who in turn hire experts to teach the AI various specialized subjects and perform other tasks to improve the models. Business Insider took a look behind the scenes at how developer Claude Code's "invisible army" operates around the world.
Two contractors working on the Anthropic project told Business Insider that they are paid $280 for each task of creating prototypes and reviewing code. According to them, each task takes about an hour, although some materials required additional approvals during the approval phase with Snorkel, a company that creates datasets to improve AI models and tests for chatbots.
The project, called Marlin within Snorkel, focuses on fine-tuning Claude Code's responses so that it can accurately mimic the work of a professional developer.
According to Snorkel’s instructions, the freelancers (all of whom have experience in software development) were tasked with A/B testing code written by two different models. In the process, they compared the performance of both models and chose the best one. One of the contractors noted that the goal of the project was to ensure that the model was capable of producing code with the required level of detail specified in the prompt. In effect, this was teaching Claude Code how to write simpler, more maintainable code.
The project is still ongoing. The freelancers themselves do not know which versions of the models they are evaluating.
As AI becomes smarter and more capable, data-marking platforms are moving from general tasks to increasingly specialized ones that require expert knowledge in a specific field or even degrees. Snorkel’s website states that the company works with professionals who have advanced degrees and degrees (such as Ph.D.s, M.D.s, or J.D.s) or equivalent experience. According to the company, top experts earn more than $3,000 per week.
In addition to Snorkel, platforms like Scale AI and Mercor are also willing to pay software engineers up to $110 per hour.
The Marlin project participants were to create a series of scenarios in which software developers could use Claude Code.
The contractors were tasked with choosing a GitHub repository from a list of thousands of options. They then had to create a Pull Request, a stage where a developer proposes changes to the code, such as new features or bug fixes. The contractors also had to write a prompt, a series of questions explaining exactly what they expected from the model.
In one of the tasks, the contractor asked the model to reorganize how the system stores and processes “execution metadata”—additional information about how processes are run. The goal was to make the code clearer and more user-friendly for developers, without changing the logic of the product itself or its functionality.
The model produced two versions of the code, after which the contractor chose the one he thought was more effective. According to the project instructions, participants were also required to send clarification prompts to “check how the models work with the context of the conversation.”
As part of another task, the executor created a security patch request for the model. It concerned the way MLFlow (an open source machine learning platform) loads Python packages when loading some models.
The task instructions required the following from the contractor: "Evaluate the production-ready code based on its correctness, security, reliability, and maintainability. The fix should effectively block command injection attempts while maintaining access for all legitimate whitelisted pip installer options."
Anthropic releases Opus 4.8 AI model with four times better bug detection
Anthropic releases Opus 4.8 AI model with four times better bug detection
Anthropic raises $65 billion and approaches a valuation of almost $1 trillion
Anthropic raises $65 billion and approaches a valuation of almost $1 trillion
Read the country's main IT news in our Telegram
Read the country's main IT news in our Telegram
