It’s been nearly two years since Microsoft CEO Satya Nadella predicted that knowledge work (the white-collar jobs held by lawyers, investment bankers, librarians, accountants, IT, etc.) would be replaced by AI.
However, despite the great advances made with basic models, changes in knowledge work have been slow to emerge. Models have mastered thorough research and agency planning, but for some reason, most white-collar jobs remain relatively untouched.
This is one of the biggest mysteries in AI, and thanks to new research from training data giant Mercor, we finally have some answers.
New research examines how leading AI models drawn from consulting, investment banking, and law hold up to performing real-world white-collar work. The result is a new benchmark called APEX-Agents, which so far has given every AI lab a failing grade. When faced with questions from real experts, even the best models struggled to get more than a quarter of the questions correct. Most of the time, the model returned a wrong answer or no answer at all.
Mercor CEO Brendan Foody, who helped write the paper, said the model’s biggest stumbling block was tracking information across multiple domains, which is essential for most human knowledge tasks.
“One of the big changes in this benchmark is that we modeled the entire environment after real-world professional services,” Foody told TechCrunch. “The way we work is not one person providing all the context in one place. We actually work across Slack and Google Drive and all these other tools.” For many agent AI models, this kind of multi-domain reasoning remains hit-or-miss.
All scenarios were drawn from real experts from Mercor’s expert marketplace who posed queries and set criteria for successful responses. If you look through the questions published on Hugging Face, you’ll see how complex the task can be.
tech crunch event
san francisco
|
October 13-15, 2026
One of the questions in the “Legal” section is:
During the first 48 minutes of the EU production shutdown, Northstar’s engineering team exported one or two bundled sets of EU production event logs containing personal data to a U.S. analytics vendor. Based on Northstar’s own policies, could the export of one or two logs be reasonably treated as consistent with Section 49?
The correct answer is yes, but getting there requires a detailed assessment of a company’s own policies and relevant EU privacy laws.
This can be confusing to even the most informed people, but the researchers were trying to model work done by experts in the field. If LLMs can reliably answer these questions, they could effectively replace many of the lawyers currently working. “I think this is probably the most important topic in economics,” Foody told TechCrunch. “The benchmarks are very reflective of the actual work of these people.”
OpenAI also attempted to measure specialized skills with the GDPval benchmark, but the APEX-Agents test differs in important ways. While GDPval tests general knowledge across a wide range of professions, the APEX-Agents benchmark measures a system’s ability to perform continuous tasks in a limited number of high-value professions. The consequences are more difficult for models, but also more closely related to whether these jobs can be automated.
Although none of the models proved ready to take over the position of investment banker, a few clearly came closer to the goal. Gemini 3 Flash performed best in the group with 24% one-shot accuracy, closely followed by GPT-5.2 at 23%. Below that, the Opus 4.5, Gemini 3 Pro, and GPT-5 all scored around 18%.
Although early results are lacking, the AI field has a history of breaking through difficult benchmarks. Now that the APEX-Agents test has been published, this is an open challenge for AI Labs that believes it can do better, and Foody fully expects to do so in the coming months.
“It’s improving really quickly,” he told TechCrunch. “Right now, we’d say the interns were getting it right one in four of the time, whereas last year they were getting it right 5 to 10 percent of the time. Year-on-year improvements like this can have an impact very quickly.”
