TB: AI Infrastructure and Scaling

• Pre-training: larger models + more data = smarter AI; synthetic data generation removes data-scarcity constraint; compute is the sole bottleneck
• Post-training: reinforcement learning and fine-tuning extend gains beyond pre-training; this dimension was underestimated
• Test-time scaling: inference requires intensive reasoning and search — "inference is thinking, and thinking is hard"; industry initially dismissed this as easy; inference market will eventually exceed training
• Agentic scaling: agents spawn sub-agents, access tools and databases; experience feeds back into retraining; the feedback loop makes capability growth self-reinforcing

• Problems that require million-fold speedup across 10,000+ machines create bottlenecks in networking, memory, power, and cooling that must be solved simultaneously — Amdahl's Law means optimizing only compute yields minimal gains if networking is 50% of latency
• Vera-Rubin pods (2026): 7 chip types, 40 racks, 60 exaflops, 10 PB/s bandwidth, 1.2 quadrillion transistors, ~200 pods/week production target
• NVLink went from 8 to 72 connections per rack (Grace-Blackwell → Vera-Rubin) driven entirely by anticipated agentic workloads requiring tool access and database lookups — the hardware change preceded the software maturity

• NVIDIA educates upstream CEOs (TSMC, SK Hynix, ASML) about multi-year roadmaps to enable aligned capital investment; the supply chain manufactures complete supercomputer racks, not components
• Memory evolution: convinced three 45-year-old DRAM manufacturers to shift DDR → HBM and LPDDR5; required explaining why "cell phone memory for supercomputers" made economic sense
• Scale of manufacturing complexity: 1.3 million components per NVLink-72 rack from 200+ suppliers

• Power grids operate at ~60% capacity 99% of the time; the 40% reserve for peak demand is idle capacity that could serve AI workloads through demand-flexible contracts with tiered SLAs
• NVIDIA improved computing efficiency 1 million-fold over 10 years (vs. ~100x from Moore's Law alone) — the additional 10,000x came from co-design across the full stack
• Tokens-per-second-per-watt is the emerging figure of merit for sustainable AI scaling

• x86 won despite poor architecture; RISC failed despite elegance; CUDA won through ubiquity — install base on GeForce gaming PCs pre-installed CUDA on millions of researchers' machines
• The 2006 CUDA bet: adding CUDA to consumer GPUs dropped gross margins from ~35% to unsustainable levels; market cap fell from ~$7-8B to $1.5B; took a decade to recover; but established the developer ecosystem that made the AI revolution NVIDIA's to win
• Architectural decisions evaluated against physical limits first ("speed-of-light thinking"), not incremental improvement from existing baselines

• NVIDIA open-sources models (Nemotron 3 Super: 120B parameter transformer+SSM hybrid) to: (1) gain first-person visibility into what future workloads need to accelerate, (2) enable every industry/researcher/country to join the AI revolution, (3) diffuse AI into non-language domains (biology, chemistry, physics)
• Building open models is not altruism — it's co-design intelligence: understanding algorithmic trends is prerequisite to anticipating hardware needs

The ultimate limit on AI scaling through 2030 is not capital, energy, or talent — it is ASML's EUV lithography output (~70 tools/year, scaling to ~100 by 2030). One gigawatt of AI compute capacity requires approximately 3.5 EUV tools. With ~700 EUV tools installed globally by 2030, the ceiling is roughly 200 GW of AI chip production capacity — sufficient for Sam Altman's 52 GW/year target only if Nvidia receives ~25% of all global EUV allocation.

Key sub-constraints:

• HBM memory: the second most severe bottleneck; prices set to double or triple again; memory vendors signing long-term deals that lock out spot-market buyers
• Logic (advanced nodes): labs that locked in TSMC N3 contracts early (Nvidia, OpenAI/Microsoft) have structural advantages that cannot be purchased at any price now
• Power: counterintuitively not a near-term US constraint — grid reserve capacity (~40% idle) can be absorbed via demand-flexible SLAs

The "production hell" gap: ASML had working EUV lab tools in the early 2010s but took 5–7 years to reach volume manufacturing. China's indigenous EUV program faces the same ramp — prototypes by 2030 are plausible; volume production is not. This asymmetry favors the US in fast-timeline AGI scenarios ("fast timelines, the US wins; long timelines, China wins").

The Hopper→Blackwell lesson: real-world inference performance improved ~20x despite only ~3x raw FLOPS improvement, due to NVLink bandwidth, advanced packaging, and architectural changes incompatible with older nodes. Falling back to 7nm multi-patterning carries severe performance penalties beyond the headline FLOP count.

Anthropic's compute paradox: $30B raised but conservative contracting left them buying H100 spot capacity at $2.40/hr vs. $1.40 historical manufacturing cost — a cautionary example of mismatched growth and infrastructure planning.

The Alchian-Allen effect on model quality: as GPU spot prices rise (scarcity), the relative premium for frontier models shrinks, shifting demand toward best-available models. Margin accrues to whoever controls allocations (foundries, memory vendors, toolmakers) — not necessarily inference providers.