Amazon CEO Andy Jassy speaks at a company event in New York on February 26, 2025.
Michael Nagle | Bloomberg | Getty Images
Amazon laid out AI usage guidelines for software engineers in a memo to staff, discouraging employees from relying on “token maxing” to drive adoption of artificial intelligence tools.
The memo, which was circulated internally in September and posted on the company’s website on Tuesday, amends how the company defines the roles of software development engineers and managers in the AI era.
“We are still at the beginning of AI’s impact on software engineering,” the company wrote. “We have a vision of what the future will hold, but there is still much that remains to be seen. Whatever future best practices are, we are confident that AI will be central to software engineering practices.”
Under CEO Andy Jassy, Amazon rapidly integrated AI across its vast business operations. Jassy previously predicted that AI would “change the scope and speed” of daily tasks and that automation could reduce companies’ workforces in the coming years, but also noted that it would also create “other jobs.”
As employees rush to adopt AI tools, companies are grappling with the rising costs of technology and how to reliably measure whether technology leads to productivity gains and other valuable returns on investment.
Companies have recently cracked down on token maxing, where developers maximize the use of AI with little regard for the output. Some companies have placed limits on the use of tokens by employees, or units of data processed and generated by AI models, to limit the rise in AI fees. Some have turned to cheaper open source models for certain tasks.
An Amazon spokesperson said the company does not impose a cap on the amount of tokens employees can use. In May, Amazon shut down its internal leaderboards after discovering that employees were token-maxing to improve their rankings.
Amazon said in a note posted Tuesday that metrics such as token usage, AI tool adoption rates, and code changes and reviews remain “significantly flawed” ways to measure teams and organizations’ use of AI, adding that they can lead to higher costs and undue complexity.
But they stopped short of telling their staff to ignore these metrics completely. Token use and code generation should be considered “with caution and judgment” in conjunction with other data, Amazon said.
“We believe that long-term outcomes are best measured and measured by long-term outcomes,” the company wrote.
Mark Brooker, vice president and distinguished engineer at Amazon Web Services, said in an interview with CNBC that the company is looking at metrics such as “feature velocity” to measure how AI is improving the productivity of its engineers.
He said the metrics look at things like “how many releases we’re making and how many changes are successful in production.”
“Broadly speaking, we think of this as a return on investment,” Brooker said. “If there are important problems to solve for our customers or important problems to solve for our business, we will invest tools, time, tokens, human effort, etc. into them.”
He added: “What we don’t want to see is wasted effort spending tokens on solving problems that don’t matter, and sometimes it’s hard to know, so it requires some willingness to experiment.”
Amazon also discouraged employees from using AI if it didn’t make sense for a particular task.
“If your work doesn’t benefit from today’s AI tools, it’s a discovery, not a failure,” Amazon said in the memo.
The company added, “Engineers who can explain where AI is not yet useful in their work demonstrate the judgment we are looking for.”
