Amazon intensified its effort to design cheaper AI processors for Amazon Web Services as demand for Nvidia hardware increased the cost of cloud computing. Engineers at its chip laboratory in Austin, Texas, were testing server designs built around Amazon’s own accelerators in July 2024.
The strategy was aimed at reducing reliance on a supplier that also powered a large part of AWS’s fastest-growing workloads. Amazon still used Nvidia extensively, but it wanted Trainium and Inferentia to give customers another option for training and running models.
Custom silicon also offered Amazon greater control over the economics of its cloud. If its processors delivered competitive performance at a lower price, AWS could reduce what executives described as the “Nvidia tax,” protect margins and make demanding AI workloads more affordable for customers.
The work was visible in a new server design being tested by engineers at the Austin facility on 26 July. That hardware brought Amazon’s chip strategy into a complete server system rather than treating the accelerator as an isolated component.
Server-level testing also allowed engineers to examine how the components performed together.
Annapurna Labs Gave Amazon a Chip Foundation
Amazon’s development programme built on its 2015 acquisition of Annapurna Labs. The unit had spent nearly a decade refining processors for mainstream cloud workloads, including Graviton, Amazon’s general-purpose server chip, which had reached its fourth generation.
That experience provided design expertise, software integration and a route to deploy hardware across AWS. Trainium was designed for model training, while Inferentia targeted inference—the process of operating a trained model to answer requests. Both were still described as early-stage products compared with Amazon’s more established processors.
Rami Sinno, engineering director at Annapurna Labs, said customers were increasingly asking for more economical alternatives to Nvidia. For AWS, responding to that demand was both a product opportunity and a defensive move as Microsoft and Alphabet also developed custom chips for their cloud platforms.
Price-Performance Was the Central Promise
AWS executives said Amazon’s in-house chips could offer a 40% to 50% improvement in price-performance compared with Nvidia-based alternatives. If achieved across real customer workloads, that difference could translate into substantial savings for organisations training or serving AI models at scale.
The claim required context. Price-performance could vary with the model, software and utilisation, and Amazon’s chips did not yet match Nvidia’s breadth of adoption. Customers would evaluate not only the accelerator but also developer tools, compatibility and how easily workloads could move from existing systems.
Amazon could nevertheless use its ownership of the cloud platform to optimise hardware, servers and services together. That vertical integration made it possible to design processors around common AWS workloads and offer them directly through services already used by enterprise customers.
The approach also gave AWS a commercial lever. It could set prices for its own processors, make capacity available within its regions and connect the hardware with managed AI services. Those advantages could make an alternative attractive even before it matched every feature in Nvidia’s ecosystem.
AWS Made Custom Chips a Strategic Priority
AWS generated $25 billion in first-quarter revenue, an increase of 17% from the previous year, and accounted for slightly less than one fifth of Amazon’s total sales. The division held roughly one third of the global cloud market, compared with about one quarter for Microsoft Azure.
Those figures explained why AI infrastructure mattered far beyond a laboratory experiment. Cloud computing was a central growth engine for Amazon, and the cost and availability of accelerators could influence how quickly AWS expanded AI services.
Amazon also demonstrated its ability to deploy custom hardware internally. During Prime Day it used 250,000 Graviton chips and 80,000 of its AI processors to support shopping, streaming and other services. The event generated a record $14.2 billion in sales according to Adobe Analytics, providing a large operational test for Amazon-designed silicon.
Nvidia Continued to Raise the Competitive Bar
Amazon’s programme did not leave Nvidia standing still. Jensen Huang had announced the Blackwell generation, expected later in 2024, and said it would deliver twice the performance for model training and five times the speed for inference.
Nvidia’s position was supported by a customer list that included Amazon, Google, Microsoft, OpenAI and Meta. Its AI focus had helped push the company’s market value to around $2 trillion, behind only Microsoft and Apple at the time. It was also expanding software and developing specialised chips for applications such as vehicle assistants and humanoid robots.
The competition was therefore not a simple replacement story. Amazon remained an Nvidia customer while simultaneously building alternatives. Nvidia offered a mature, widely supported platform; Amazon could offer closer AWS integration and potentially lower prices.
For cloud customers, that rivalry created useful choice. Amazon’s chips could put downward pressure on costs and reduce exposure to shortages, while Nvidia’s rapid product cycle forced custom-chip teams to keep improving. The success of Trainium and Inferentia would depend on whether AWS could convert promising price-performance claims into reliable advantages across real AI workloads.
