GROQ
LPU // Language Processing Unit
The inference speed champion. Groq's revolutionary LPU architecture delivers deterministic latency and unprecedented throughput for AI inference. Founded by the architect of Google's TPU, Groq proved that purpose-built hardware could dramatically outperform GPUs—so much so that NVIDIA paid $20 billion to acquire their technology in December 2025.
LPU Architecture
Why Groq is fundamentally different from GPUs
Tensor Streaming Processor (TSP)
Unlike GPUs with thousands of cores fighting for memory access, Groq's LPU uses a single stream of execution. Data flows through the chip in a predetermined path, eliminating the scheduling overhead and memory contention that plague GPU inference. The result: perfectly predictable performance every single time.
SRAM-First Design
Each LPU contains 230MB of on-chip SRAM with 80 TB/s of internal bandwidth. By storing model weights in SRAM rather than external HBM, Groq eliminates the memory bandwidth bottleneck that limits GPU inference. This is why NVIDIA's acquisition addresses their own HBM supply chain vulnerability.
Static Scheduling
The compiler determines exactly when every operation will execute before runtime. There's no dynamic scheduling, no kernel launches, no runtime decisions. This eliminates jitter entirely—if Groq says inference takes 5ms, it takes exactly 5ms, every time. Critical for real-time applications like autonomous vehicles and live AI assistants.
TruePoint Numerics
Groq's proprietary numerical format stores 100 bits of intermediate accumulation, preserving quality while enabling aggressive optimization. Unlike GPU quantization that sacrifices accuracy for speed, TruePoint reduces precision only where it doesn't affect output quality.
The NVIDIA Deal
Why Jensen Huang paid $20 billion for Groq
Technology Acquisition
NVIDIA gains Groq's inference-optimized architecture and TruePoint numerics. The LPU technology will be integrated into CUDA, creating a "dual-stack" for training on GPUs and inference on LPU-enhanced hardware.
Talent Capture
Jonathan Ross (TPU architect) and Sunny Madra join NVIDIA leadership. The engineering team that built the fastest inference chip joins the dominant AI hardware company.
HBM Hedge
Groq's SRAM-based approach reduces NVIDIA's dependency on constrained HBM supply chains. With HBM production frequently bottlenecked, this strategic alternative is invaluable.
Technical Specifications
LPU v2 Performance Metrics
Company Timeline
From Google TPU to NVIDIA acquisition
Strategic Analysis
What the NVIDIA deal means for AI infrastructure
✦ Why Groq Succeeded
- Founder pedigree: Jonathan Ross literally designed Google's TPU
- GroqCloud viral moment created massive developer awareness
- Deterministic latency solved real problems for production AI
- 75% of Fortune 100 had accounts on the platform
- Aggressive global expansion with 12+ data centers
- Clear use case: real-time inference where latency matters
⚠ Why They Sold
- Manufacturing on GlobalFoundries 14nm limited competitiveness
- $20B represented 3x last private valuation—exceptional return
- Competing with NVIDIA long-term requires massive capital
- Integration into CUDA ecosystem amplifies technology impact
- NVIDIA's resources accelerate LPU v3+ development
- Strategic acqui-hire secures team's financial outcomes
◎ Industry Impact
- Validates inference-specific hardware as strategic priority
- NVIDIA now controls both training AND inference innovation
- Remaining startups (Cerebras, Tenstorrent) become more valuable
- Regulators may scrutinize deal structure
- Sets precedent for AI chip startup exits
- Accelerates "Inference Era" investment thesis
⟁ Future Outlook
- CUDA + LPU "dual-stack" strategy emerges
- Groq technology likely appears in NVIDIA products by 2027
- GroqCloud may continue operating under NVIDIA umbrella
- Jonathan Ross positioned for major NVIDIA leadership role
- Inference hardware market consolidates around NVIDIA
- Alternative chip startups face higher bar for success