Google splits its eighth-generation TPUs into four systems, one for confidential inference

Google (Nasdaq: GOOGL) said on Monday, November 9, that it is adding two systems, TPU 8tx for confidential inference and TPU 8i Max for very long contexts, to its eighth-generation AI chips, according to a post by Amin Vahdat, the company's senior vice president and chief technologist for AI and infrastructure.

Four systems from one generation in a single year

Google introduced TPU 8t, for large-scale training, and TPU 8i, for real-time reasoning, at Google Cloud Next in April. Vahdat wrote that the two new systems make it "the first time we have introduced four TPU systems in a single year." TPUs are the company's own AI accelerator chips, which it uses for its Gemini models and rents to Google Cloud customers.

Google said TPUs are now "a rapidly growing business with nearly 100 customers," a group it described as including frontier labs, trading firms, retailers and universities. It also said it now lets customers deploy TPUs directly in their own data centers. The post gives no revenue, customer names or order sizes.

What each new system is for

TPU 8tx is built on the same silicon as TPU 8t and paired with 5th Generation AMD EPYC processors as hosts. Google said it keeps data and model weights protected "even while in active use," a hardware feature aimed at regulated companies and governments. The company said it delivers up to 2x lower latency than TPU 8t with "a near-zero performance penalty."

TPU 8i Max combines two TPU 8i chips in one package. Google said it provides double the compute, double the SRAM and double the high-bandwidth memory (HBM) bandwidth of TPU 8i, and that it fits twice the silicon into the existing rack footprint. The problem it targets is what Google calls the memory wall: when many AI agents work across a large codebase or hours of video, memory bandwidth becomes the bottleneck.

All performance figures in the post are Google's own. The comparisons are against other Google parts, and the post does not say when either system will be available or what it will cost.

The software around the chips

Google paired the hardware with software releases. It said it open-sourced TorchTPU last month so that PyTorch code and Hugging Face models run natively on TPUs. It also announced an open-source TPU Sync layer that it said cuts key-value cache transfer latency and time to first token by more than 90%.

Cloud Pathways and GKE dynamic slicing are now generally available, Google said. The company said the dynamic slicing feature cuts job start-up times by up to 5x.

We would read TPU revenue, not TPU launches

Our reading is that four systems in a year show Google treating the chip as a product line with different buyers, not as a single part. Splitting out a confidential-computing system points at the customers who hold the most sensitive data, and the memory-heavy system at the labs that run agents.

The opportunity is a product line with paying customers, nearly 100 by Google's count. The threat is that Google gave no price, date or revenue figure, so the launch cannot yet be measured against the capital spending behind it. We would weigh the cloud segment's reported growth and Alphabet's capital expenditure in its next quarterly report over any benchmark in the post, and we would change our view if the company starts disclosing TPU revenue separately.

The view in this final section is FinanceInTime's editorial opinion, for information only. It is not investment, tax, legal or other advice, and it does not take your circumstances into account. Forecasts are uncertain and past results do not guarantee future ones.