You can now rent Google’s most powerful AI chip: Trillium TPU underpins Gemini 2.0 and will put AMD and Nvidia on high alert

LovabledanielsDecember 27, 2024123 Views

Trillium has hit general availability just months after preview release
Powerful AI chip offers more than four times the training performance
Google uses it to train Gemini 2.0, the company’s advanced AI model

Google has been developing Tensor Processing Units (TPUs), its custom AI accelerators, for over a decade, and a few months after being made available in preview, has announced that its sixth-generation TPU has reached general availability and is now available for rent.

Trillium doubles both the HBM capacity and the Interchip Interconnect bandwidth, and was was used to train Gemini 2.0, the tech giant’s flagship AI model.

Google reports it offers up to a 2.5x improvement in training performance per dollar compared to prior TPU generations, making it an appealing option for enterprises seeking efficient AI infrastructure.

Google Cloud’s AI Hypercomputer

Trillium delivers a range of other improvements over its predecessor, including more than four times the training performance. Energy efficiency has been increased by 67%, while peak compute performance per chip has risen by a factor of 4.7.

Trillium naturally improves inference performance as well. Google’s tests indicate over three times higher throughput for image generation models such as Stable Diffusion XL and nearly twice the throughput for large language models compared to earlier TPU generations.

The chip is also optimized for embedding-intensive models, with its third-generation SparseCore providing better performance for dynamic and data-dependent operations.

Trillium TPU also forms the foundation of Google Cloud’s AI Hypercomputer. This system features over 100,000 Trillium chips connected via a Jupiter network fabric delivering 13 Petabits/sec of bandwidth. It integrates optimized hardware, open software, and popular machine learning frameworks, including JAX, PyTorch, and TensorFlow.

With Trillium now generally available, Google Cloud customers have the opportunity to access the same hardware used to train Gemini 2.0, making high-performance AI infrastructure more accessible for a wide range of applications.

Trillium TPU, built to power the future of AI – YouTube