This report is written in the voice and perspective of Andrew Carnegie (AI) — an industrial strategist assessing Google's custom silicon ambitions through the lens of vertical integration, cost curves, and competitive moats. The analysis reflects a historically grounded, capital-focused perspective on Alphabet's eighth-generation TPU strategy.
1. Overview
In April 2026, Google made a decision that separates the builders from the speculators in the AI infrastructure race. The company unveiled its eighth-generation Tensor Processing Units — not as a single workhorse, but as two purpose-built machines: TPU 8t for training, TPU 8i for inference. After twelve years and seven generations of unified silicon, this split signals something the market has been slow to acknowledge: the era of general-purpose AI accelerators is giving way to workload-specific architecture as the dominant paradigm.
Announced at Google Cloud Next '26 on April 22–23, the eighth-generation TPU line delivers generational leaps in compute, efficiency, and cost-effectiveness while expanding Google's addressable market in external cloud AI hardware. But this is more than a product launch. The TPU 8t and 8i embody a thesis that I recognize from the steel era: he who controls the means of production — the raw materials, the mills, the transport, and the distribution — holds the decisive advantage. Google calls this "infrastructure sovereignty." It is vertical integration by another name, and it is the real story here.
2. Key Insights
2.1 The Strategic Shift to Workload-Specific Silicon
The defining architectural decision of the eighth-generation TPU is the bifurcation of what was previously a unified accelerator into two specialized chips. The TPU 8t is optimized for training workloads, while the TPU 8i is optimized for inference. This is a deliberate departure from Google's prior approach of offering a single TPU design that handled both phases of the AI lifecycle.
The rationale is grounded in fundamentally different computational requirements:
- Training demands sustained, high-throughput matrix arithmetic across massive datasets
- Inference — particularly for autonomous AI agents — demands low latency, high memory bandwidth, and efficient handling of key-value caches for reasoning chains
By disaggregating these use cases, Google can optimize each chip's microarchitecture, memory hierarchy, and interconnect topology for its specific task. This split reflects a broader industry transition away from general-purpose processors toward domain-specific architectures and custom ASICs for AI workloads.
The hyperscaler custom-silicon movement — encompassing Google's TPUs, Amazon's Trainium and Inferentia, and Microsoft's Maia chips — is creating a structural disruption to the GPU-centric compute model that has dominated AI since the deep-learning renaissance.
2.2 Performance: Generational Leaps Across Multiple Dimensions
The performance claims for TPU 8t and TPU 8i are unusually well-corroborated, with multiple independent sources reporting consistent metrics.
Compute Performance
- The TPU 8t delivers approximately 3x compute improvement per pod compared to the seventh-generation Ironwood TPU
- A single TPU 8t superpod (9,600 chips) achieves 121 ExaFlops of compute performance
- The eighth generation delivers 6x more computing power per unit of electricity compared with TPU technology from five years earlier
Performance per Watt
- The TPU 8t/8i generation achieves up to 2x performance-per-watt versus the Ironwood generation
- The generation delivers 20% more energy efficiency compared to the prior generation
Training Price-Performance
- The TPU 8t delivers approximately 2.7x better training price-performance compared to Ironwood
- LLM development is 36% faster using TPU 8t compared with previous generations
Inference Performance and Cost Efficiency
- The TPU 8i delivers 80% better inference performance-per-dollar versus the prior generation
- 80% cost efficiency improvement for inference workloads
Memory Architecture
- The TPU 8i triples on-chip SRAM to 384MB
- Paired with 288 GB of high-bandwidth memory (HBM)
- The memory expansion is specifically designed to accommodate larger KV caches for reasoning and multi-turn agentic workflows
Interconnect Bandwidth
- The TPU 8i doubles Inter-Chip Interconnect (ICI) bandwidth to 19.2 Tb/s
- The TPU 8t achieves 2x ICI bandwidth improvement versus Ironwood
- Data center network bandwidth increases 4x
Fabric Latency and Goodput
- The TPU 8i introduces a Collective Acceleration Engine (CAE) that reduces on-chip latency by 5x
- Communication latency improves by up to 50% versus Ironwood
- ICI network diameter is reduced by over 50%
- The TPU 8t achieves 97% goodput (productive compute time) via automated fault detection and Optical Circuit Switching
- Unloaded fabric latency is reduced by 40%
Storage Access
- The TPU 8t delivers 10x faster storage access compared to Ironwood
These numbers are consistent across independent sources — a rare signal in a market awash in unsubstantiated claims.
2.3 The Full-Stack Integration Thesis
The most strategically significant insight is the degree to which Google's TPU strategy is inseparable from its broader full-stack vertical integration:
- Google designs its own TPU silicon in-house
- Integrates its own Axion Arm-based CPUs as TPU hosts for the first time in the eighth generation
- Deploys its Virgo network, JAX, and Pathways software layers to enable near-linear scaling up to 1 million chips in a single logical cluster
- Supports the PyTorch, JAX, MaxText, SGLang, and vLLM frameworks natively
This vertical integration creates a virtuous feedback loop. Google gains firsthand insights into AI model behavior through its own development of frontier models (e.g., Gemini), and those insights directly inform TPU design decisions. The company's design philosophy rests on "three pillars: scalability, reliability, and efficiency."
The result is a full-stack capability — custom TPUs, proprietary AI models, data infrastructure, and cloud services — that competitors without equivalent in-house silicon cannot replicate. The strategic intent is explicit: "infrastructure sovereignty." Google wants control over the entire compute stack, reducing dependence on external suppliers like NVIDIA for GPUs, Intel and AMD for CPU hosts, and Broadcom for design services.
This is the modern equivalent of what was done in steel: owning the mines, the mills, the railroads, and the distribution network. The technologies change; the dynamics rhyme.
2.4 Supply Chain, Partnerships, and the Ecosystem
The TPU program involves a complex web of supply-chain relationships:
-
Intel Collaboration: Google and Intel are co-developing custom Infrastructure Processing Units (IPUs) for AI workloads. These IPUs appear to handle infrastructure acceleration tasks distinct from the TPU's core AI compute role.
-
Broadcom Relationship: Broadcom has served as Google's co-designer for TPU ASICs, with TPU 8i design wins benefiting Broadcom's custom silicon revenue line.
-
MediaTek Transition: Google is moving to MediaTek for the V8 TPU, suggesting a shift in design-services partners from Broadcom.
-
Marvell Involvement: Marvell Technology is serving as the "design-services partner" for one of the two custom AI chips, with involvement in the "remaining portions of the V8 Inference TPU design."
-
TSMC Manufacturing: The TPU 8 program uses Taiwan Semiconductor Manufacturing Company's 3nm packaging.
-
Broader Supply Chain: Component categories supporting Google TPU deployments include optical modules, high-speed interconnect chips, optical circuit switching, server power supplies, PCBs, HBM and NAND flash memory, liquid cooling systems, and ARM-based CPUs.
The AI infrastructure ecosystem around Google's TPUs is characterized as a multi-layered "picks-and-shovels" opportunity, suggesting that the investment implications extend beyond Alphabet itself to the broader supply chain.
2.5 Competitive Positioning and Risks
Versus NVIDIA
The competitive dynamic with NVIDIA is the most frequently cited tension. Google continues to rely on NVIDIA for GPU instances even as it deploys custom TPUs, suggesting a multi-sourcing strategy rather than a complete replacement. NVIDIA CEO Jensen Huang asserts that custom accelerators like Google's TPUs "perform well in controlled hyperscaler environments but lack the broad ecosystem and cost advantages of NVIDIA's general-purpose GPUs."
Conversely, other claims argue that:
- TPUs are "designed for deep learning and offer better performance for training large models than NVIDIA GPUs"
- TPUs are "approximately 2x cheaper than NVIDIA GPU alternatives for well-defined, cost-optimized use cases"
- Google's TPU-plus-custom-networking stack could pose "technology disruption risk" to NVIDIA
Cost Advantage
TPUs are positioned as offering a 2x cost advantage over NVIDIA for suitable workloads, with the 80% performance-per-dollar improvement over Ironwood further widening the moat.
Obsolescence Risk
Multiple claims flag a significant strategic risk: TPUs typically take about three years to develop from start to finish, while AI models are evolving much faster. This raises the specter that TPU architecture "may not keep pace with rapid AI model evolution, creating a risk of technological obsolescence." Another source warns that "current Nvidia and Google TPU infrastructure may become obsolete before achieving a return on investment."
Customer Conflict
Google's own AI services compete with customers who use the same TPU supply. This "co-opetition" dynamic could become more acute as external TPU availability expands.
External Availability and Market Expansion
TPU 8t and TPU 8i will become generally available to external cloud customers later in 2026. This marks a continuation of Google's strategy to make its custom silicon available externally. The TPU 8i is the first TPU to offer bare-metal access, and both chips support bare-metal configurations. The external availability of TPUs is expanding the range of hardware options available to builders of agentic AI systems.
3. Analysis and Significance
3.1 The Split Architecture as a Competitive Moat
The decision to split training and inference silicon is the single most consequential insight. It signals that Google believes AI workloads have diverged sufficiently to warrant separate silicon — a view that, if validated by market adoption, could reshape the competitive landscape.
- Training chips optimize for compute density and scale-out efficiency
- Inference chips optimize for memory bandwidth, latency, and cost per transaction
By designing two chips, Google can push the frontier on both dimensions simultaneously, rather than compromising on a unified design. This is the kind of focused, capital-efficient thinking recognized in the best-run industrial enterprises.
This is particularly significant for the inference market, which is still in its early growth phase. As AI models move from training to production deployment at scale, inference workloads will dominate total AI compute demand. The TPU 8i's tripled SRAM, 80% better cost efficiency, and explicit optimization for "autonomous AI agents requiring reasoning, planning, and multi-step workflows" position Google to capture a disproportionate share of this emerging market. The TPU 8t's 97% goodput and near-linear scaling to 1 million chips address the training side equally aggressively.
3.2 The Axion CPU Integration: Completing the Vertical Stack
The integration of Google's custom Axion Arm-based CPU into TPU hosts for the first time is a subtler but equally important development. It addresses the "host bottleneck" caused by data preparation latency and reduces dependence on Intel and AMD x86 CPUs.
If this transition is validated, it represents a direct competitive threat to Intel and AMD's data-center CPU businesses and further evidence of the industry-wide shift to ARM-based server processors. In the steel business, controlling the supply of coke and iron ore gave pricing power over competitors who had to buy from the open market. Google is applying the same logic to the compute stack.
3.3 The Picks-and-Shovels Investment Thesis
The detailed characterization of the TPU supply chain suggests that Google's custom silicon strategy creates meaningful downstream investment opportunities. The component categories identified — optical modules, interconnect chips, optical circuit switching, HBM, liquid cooling, ARM-based CPUs, advanced packaging — each represent potential beneficiaries of Google's aggressive infrastructure scaling.
Google's TPU cluster sizes have reached the "gigawatt level," making system design, power delivery, and cooling "the principal bottlenecks in AI scaling." The fourth-generation liquid cooling technology deployed to support TPU infrastructure underscores the intensifying physical constraints. These are not abstract concerns — they are real engineering limits that create real opportunities for the suppliers who can solve them.
3.4 Risk Assessment
The most significant risk identified across the claims is technology obsolescence. The three-year TPU development cycle versus the rapid evolution of AI model architectures creates a structural mismatch. Google is already planning TPU v9 trials for 2027, suggesting the company is aware of the need to accelerate its cadence. However, the claim that TPU v5e utilization increased by 72% between October 2024 and January 2026 suggests that existing generations are still being absorbed efficiently, which may partially mitigate the obsolescence concern.
A secondary risk is competitive response. NVIDIA is not standing still, and the claim that Google continues to rely on NVIDIA GPUs suggests the custom silicon is complementary rather than fully substitutive, at least for now. If NVIDIA's next-generation GPUs outperform TPUs on key workloads, or if rival custom chips (AWS Trainium, Microsoft Maia) gain traction, Google's investment thesis could be challenged.
3.5 Implications for the AI Semiconductor Landscape
The collective weight of the claims supports the thesis that custom silicon is becoming the dominant paradigm for hyperscale AI compute. Google's TPU 8t/8i split is not an isolated product decision but a leading indicator of a structural shift away from general-purpose GPUs toward workload-specific ASICs. Qualcomm's reported entry into custom hyperscale silicon for agentic AI workloads further validates this trend.
For investors, the implication is that the AI semiconductor opportunity is broadening beyond NVIDIA. The total addressable market for AI accelerators is expanding as hyperscalers internalize chip design, creating opportunities for:
- Design-services partners (Broadcom, Marvell, MediaTek)
- Foundry partners (TSMC)
- Memory suppliers (HBM)
- The broader interconnect and cooling ecosystem
Even as it introduces competitive pressure on incumbent GPU suppliers.
4. Key Takeaways
The TPU 8t/8i Split is a Defining Strategic Inflection Point for Alphabet
By disaggregating training and inference into purpose-built chips for the first time, Google has positioned itself to optimize for two rapidly diverging workloads simultaneously. The 3x compute improvement (training), 80% better inference cost efficiency, and 2x performance-per-watt are well-corroborated metrics that give investors confidence in the product's competitiveness. The explicit targeting of agentic AI workloads through the TPU 8i's enhanced memory and latency profile positions Google to capture the next wave of inference demand.
Vertical Integration is Google's Core Competitive Moat, and It is Deepening
The Axion ARM CPU integration, Virgo networking, Pathways/JAX software stack, fourth-generation liquid cooling, and near-linear scaling to 1 million chips represent a full-stack capability that no other cloud provider — and certainly no GPU-only supplier — can match. Investors should monitor the pace at which Google can bring this integrated stack to external cloud customers, as that will determine whether TPU monetization scales beyond Google's own AI workloads.
Technology Obsolescence is the Primary Risk, and the Development Cycle Mismatch is Structural
The three-year TPU development cycle versus accelerating AI model evolution creates genuine risk that TPU architectures may lag behind emerging model requirements. Google's planned 2027 v9 trials suggest awareness, but investors should watch for signs that Google is shortening its silicon iteration cycle or adopting more flexible chiplet-based architectures. The counterargument is that TPU v5e utilization is still ramping (72% increase), suggesting existing capacity is not yet fully monetized.
The TPU Supply Chain Creates Material Picks-and-Shovels Investment Opportunities Beyond Alphabet
The detailed bill of materials for TPU infrastructure — optical modules, interconnect ASICs, HBM, advanced packaging (TSMC 3nm), liquid cooling, and ARM CPUs — identifies specific beneficiaries of Google's infrastructure scaling. As TPU cluster sizes reach gigawatt scale, companies providing interconnect, cooling, and memory solutions stand to benefit disproportionately from the buildout, independent of whether Google's custom silicon ultimately displaces NVIDIA's GPUs at the architectural level.
