Okay, here is the abstract and summary for the provided Tech Poutine transcript:
Abstract:
This episode of the Tech Poutine podcast, hosted by Ian Cutress (TechTechPotato) and George Almasi (Chips and Cheese), delves into a wide array of recent semiconductor and technology news. A significant portion discusses the complexities and implications of newly announced US tariffs, analyzing exemptions (particularly for Mexican assembly impacting server builds), the impact on data center construction costs, and the semiconductor equipment supply chain, referencing a detailed SemiAnalysis article. The hosts cover the Tens Torrent Developer Day, detailing the new Black Hole AI accelerator chip (architecture, GDDR6 memory, Ethernet scaling), the Ascalon RISC-V core roadmap, their open-source software stack, and the Galaxy server system. They also discuss Google's TPUv7 (Ironwood) announcement, its chiplet design and specifications, while critiquing the misleading performance comparison made against the El Capitan supercomputer. The launch of IBM's Z17 mainframe system is explored, including the Telum 2 processor with its integrated DPU, the Spire AI accelerator card, the system's high-availability focus, surprisingly thick PCB design, and potential use cases like Pokémon Go. Other topics include highlights from the OFC conference on optical networking and compute, the announcement of the UALink interconnect standard, Llama 4 AI models, Nintendo Switch 2 details, upcoming AMD events, and Lip-Bu Tan's keynote at Intel Vision.
Tech Poutine Episode 22: Tariffs, Tens Torrent, TPUv7, Z17, and More
0:00:05 Jim Keller Intro: The show opens with a clip of Jim Keller (CEO of Tens Torrent) agreeing to say "Don't forget to like and subscribe" for the channel, obtained during the Tens Torrent Dev Day.
0:02:06 Show Intro & Location: Ian and George introduce the show. George is broadcasting from Washington, attending the PDXLAN event.
0:04:47 Upcoming Events: Mention of upcoming industry events: TSMC Foundry (Apr 23), Intel Foundry (Apr 29), Andes Con (Apr 29), followed later by Samsung and GlobalFoundries events. An interview with the new GlobalFoundries CEO is anticipated around August.
0:06:47 Meta MTIA Chip on eBay: Discussion about Meta's MTIA AI accelerator engineering samples appearing on eBay, listed under a different name, and confirmation via heatsink removal analysis by others (Andy from Chips and Cheese acquired one).
0:09:04 Planned AI/NPU Testing: Chips and Cheese is planning an AI/NPU roundup, testing various cards (PCIe/M.2 like Halo, Axelera) and exploring the challenges of benchmarking different AI architectures (CNNs vs LLMs).
0:12:26 Recent Event Roundup: A quick recap of recent significant events covered or attended: Synopsis User Group (Snug) focusing on HBM4, the IBM Infrastructure Summit (Z17 pre-brief), Intel Vision, and the Optical Fiber Communications Conference (OFC).
0:14:50 Optical Compute Demo: Ian shows a video (originally from Supercomputing '23, now publishable) of an optical processor from Salience Labs playing an Atari game using a neural network, demonstrating optical compute capabilities.
0:16:13 More Events & Elon Musk: Mentions the Tens Torrent Dev Day again and briefly touches on Elon Musk's recent game streaming attempts and associated technical issues/lag claims.
0:18:48 US Tariffs Deep Dive: Begins an extensive discussion on the recently announced US import tariffs, referencing a SemiAnalysis article. Covers the complexity, varying rates by country (including reciprocal tariffs making China over 100%), the calculation methods (possibly involving LLMs leading to oddities like tariffs on penguin islands), and the 90-day pause (excluding China).
0:27:07 Tariff Exemptions & Loopholes: Details exemptions: Canada/Mexico (existing deals), steel/aluminum (with nuance on machined goods), products with >20% US content (tariff only on foreign value-add), and specific items in Annex 2 (pharma, lumber, some semiconductors). Critically, the $800 de minimis threshold for duty-free personal imports will no longer apply to China.
0:31:45 Tariffs: Impact on GPUs/Servers: Explains the significant loophole: server/GPU assembly in Mexico (used by Foxconn, Wistron, Quanta, Supermicro for cloud providers like Amazon, Google) largely bypasses the tariffs for major components like GPUs/CPUs. Estimates tariffs only apply to ~8% of a GB200 server's Bill of Materials cost. Consumer GPUs assembled elsewhere (e.g., Taiwan) would be hit.
0:36:38 Tariffs: Data Center & Networking: Discusses the knock-on effects for data center construction (power delivery, cooling, generators, lighting) and networking gear (optical modules), which have higher exposure due to manufacturing predominantly in Asia (excluding Mexico/EU builds).
0:39:26 Tariffs: Semiconductor Equipment: Highlights the major impact on semiconductor manufacturing equipment (ASML, Tokyo Electron, Applied Materials) due to global supply chains for components and raw materials (wafers, photoresists). This is expected to increase US fab buildout costs significantly, potentially reducing the impact of CHIPS Act funding by ~25%.
0:48:35 Tens Torrent Dev Day: Transitions to covering the Tens Torrent developer day event in San Francisco, introducing their next-gen AI chip, Black Hole.
0:50:55 Black Hole Architecture & Scaling: Details the Black Hole chip architecture: a 2D grid with 140 "10six++" AI cores, 8 SciFive X280 RISC-V cores, 32GB GDDR6 memory (512 GB/s), 210MB on-chip SRAM, PCIe Gen 6, and up to 10x 400GbE ports. The strategy uses Ethernet for scale-up, connecting potentially thousands of chips without large external switches.
0:53:19 Tens Torrent Products: Outlines the product lineup based on Black Hole: P100A (binned down, active cooling, fewer cores/memory/Ethernet), P150A/B (full chip, active/passive, 4x 800G ports - initially limited to TT interconnect), the liquid-cooled 4-card Galaxy server box (~$12k), and the upcoming dual-chip P300A/B card.
0:58:30 Ascalon RISC-V Core: Discusses Tens Torrent's custom RISC-V CPU core, Ascalon (RVA23, ~2.5 GHz, targeting ~18 SpecINT2k6/GHz). It will be available for licensing and used in their own CPU chiplets. It's expected to be a high-performance RISC-V core but competes in a different class than current high-end x86.
1:01:05 Tens Torrent Software & Ecosystem: Covers their open-source software stack (PyTorch down to low-level kernel - LLK), mentioning another AI company is using their compiler. Jim Keller is open-sourcing Tens Torrent patents. They have partnerships for cards, workstations (e.g., Falcon Northwest), cloud providers, and full systems. LG is a known chiplet customer.
1:03:40 Galaxy Server Hardware: Shows the hardware demo of the Galaxy server, noting the move to a more traditional server design (away from early 48V DC challenges). It's a 6U chassis with four 1.5U trays, each holding eight Black Hole chips, potentially host-less due to on-chip CPUs.
1:07:10 Tens Torrent Portfolio & Scorecard: Summarizes their business model: selling hardware (cards, systems), chiplets (CPU, AI, customer-specific), and IP. Jim Keller's self-assessment highlights openness and price as strengths, with performance, compatibility, and scale needing work. They are exploring training/fine-tuning workloads on the GDDR6-based cards.
1:12:13 Networking Standards (200G SerDes): Discussion on the IEEE standards process, specifically the split and delay in ratifying the 200G SerDes spec needed for 800G/1.6T Ethernet, leading industry groups (like UALink) to define their own specs in the interim.
1:21:17 Google TPUv7 (Ironwood) Announced: Google announced their next-gen TPU, seemingly unifying the training (P) and inference (E) lines. It features a chiplet design (appears to be 2 compute dies, 1 IO die).
1:23:31 TPUv7 Specs & Scaling: Provides specs: ~4.6 PFLOPS FP8 peak compute, 192GB HBM3e @ 7.2 GT/s (7.2 TB/s bandwidth - slightly lower speed likely for multi-vendor sourcing), scaling up to 9,216 TPUs per pod (~10 MW est.), and increased interconnect bandwidth (1.2 Tbps bidirectional).
1:27:15 TPUv7 vs El Capitan Controversy: Critiques Google's marketing comparison claiming 24x more compute than El Capitan. Highlights the flaws: comparing TPUv7's peak FP8 theoretical flops to El Capitan's FP64 Linpack (Rmax, not Rpeak) actual workload performance. A more apples-to-apples (though still estimated) FP8 comparison suggests El Capitan (based on older MI300A) is significantly more powerful, making TPUv7 seem less efficient than expected for a new, specialized chip.
1:38:07 TPUv7 Sparse Core & Pathways: Briefly mentions continued support for Sparse Core (introduced in v6/Trillium) for recommendation workloads and the Pathways runtime environment.
1:40:33 IBM Z17 System Launch: Focus shifts to the IBM Z17 mainframe system launch, built around the Telum 2 processor (detailed last year at Hot Chips). Telum 2 maintains 8 CPU cores but increases L3 cache (360MB total) and adds an on-die Data Processing Unit (DPU).
1:42:37 Z17 DPU Explained: Details the function of the new DPU: it's not primarily for throughput like typical DPUs, but rather to abstract legacy Z-architecture IO channel complexities, reducing friction and simplifying the implementation of modern IO like PCIe and Ethernet, improving latency and power efficiency.
1:47:04 Spire AI Accelerator Integration: Discusses the integration of the IBM Spire AI accelerator card (announced at Hot Chips) into the Z17 system (up to 192 cards per system). Each card has 128GB LPDDR5. While technically supporting x86 (used for bring-up), IBM is unlikely to sell the card standalone due to their systems/consulting business model. The card leverages IBM's extensive RAS features.
1:49:54 Z Architecture Use Cases & Evolution: Addresses the relevance of mainframes today. Z systems still handle traditional banking/airline transactions but have evolved with Linux One versions for standard enterprise workloads. Key differentiators are extreme high availability (RAS features) and virtualization. A surprising potential customer discussed is Niantic (Pokémon Go) for managing millions of real-time players, transactions, low latency interactions, and requiring high uptime.
2:02:02 IBM Z Momentum & Roadmap: IBM shows Z platform compute capacity (measured in MIPS) sales are higher than ever. Their roadmap extends to ZNext+2 by ~2030, moving to 3nm and 2nm nodes (likely Samsung), strongly hinting at a move to chiplet-based designs, mirroring the Power roadmap.
2:08:05 IBM Z17 Hardware Details: Shows hardware components: the Dual Chip Module (DCM) package, the OMI-based DDIMM memory modules (serialized memory for density and signal integrity), a removable power delivery card, and the exceptionally thick main system PCB (estimated 64+ layers), highlighting IBM's focus on reliability and serviceability.
2:17:36 OFC Optical Tech Roundup: Quick showcase of various optical technologies seen at OFC: IMEC's 300mm photonics wafer, Intel's co-packaged optics (CPO) concepts and optical chiplets, optical circuit switches, Broadcom's CPO roadmap, Ayar Labs' TeraPHY optical chiplet and external laser source box (ELS), Lightmatter's optical interposer (Passage) and Envize optical compute chip, and CogniFiber's concept for compute-in-fiber using light interference.
2:37:00 AMD Advancing AI Event: Notes the upcoming AMD Advancing AI event on June 12th, where the next-gen Instinct MI350X GPU is expected to be announced. Also mentions the completion of AMD's acquisition of ZT Systems (for rack-scale design capabilities, manufacturing side to be divested).
2:41:03 Nintendo Switch 2 Rumors & Details: Discusses the Switch 2: $450 price point, rumored specs (enhanced Tegra, Ampere GPU cores, RT cores for upscaling), backward compatibility issues (some games may not work, others might require a fee for enablement on Switch 2), potential game price increase to $80, and US pre-orders being delayed possibly due to tariff uncertainties impacting stock allocation.
2:44:38 Llama 4 Models: Meta launched Llama 4 models (Scout ~100B, Maverick ~400B, Behemoth ~1.6T parameters). These are now Mixture-of-Experts (MoE) models (e.g., Maverick has 17B active parameters), making inference faster but requiring large memory capacity. Early reports suggest good performance but potentially lower accuracy compared to previous dense models.
2:45:59 UALink 1.0 Spec Announced: The Ultra Accelerator Link (UALink) 1.0 specification was announced by the consortium (AMD, Intel, Google, Microsoft, Meta, Cisco, Broadcom rejoined, Apple, Tenstorrent etc.). It's an industry standard alternative to NVLink for scale-up interconnect, derived from AMD's Infinity Fabric/XGMI. Supports up to 1024 accelerators, uses 200G SerDes (ahead of IEEE ratification), targeting high-bandwidth, low-latency communication within a rack/pod. First switch chips expected ~2027 publicly, potentially earlier internally.
2:52:50 Intel Vision Keynote (Lip-Bu Tan): Recap of the new Intel Chair Lip-Bu Tan's keynote. Focused on establishing his credentials (long tenure as Cadence CEO, successful VC investor via Walden International), acknowledging Intel's core strengths (client, enterprise, foundry), and aiming to foster innovation/startup mentality. His extensive board positions at other tech companies (some potentially competing with Intel niches like Celestial.ai) were noted.
3:00:31 Intel Livestream Controversy: Mentions an Intel product announcement livestream that was temporarily privated after a partner from a defense AI startup made a controversial comment about military decision-making ("better to make the wrong decision immediately than the right one later").
3:04:24 Q&A & Closing: Addresses a viewer question about the CHIPS Act status (still law). Ian explains his absence from PDXLAN due to extensive travel and family commitments. Discussion shifts to Computex plans, including hotel bookings and favourite ramen spots (Ramen Ultra).
3:10:49 Outro: Standard sign-off, promoting their respective channels (TechTechPotato, Chips and Cheese), Substacks (MoreThanMoore, Chips and Cheese), and merchandise.