About this track
InferenceX, formerly InferenceMAX: https://inferencex.com/ Article (latest): https://newsletter.semianalysis.com/p/inferencex-v2-nvidia-blackwell-vs GitHub https://github.com/SemiAnalysisAI/InferenceX Article (original) https://newsletter.semianalysis.com/p/inferencemax-open-source-inference 00:00 Introduction to InferenceX 02:52 Evolution from InferenceMAX to InferenceX 06:06 Benchmarking and Performance Insights 08:43 The Scale of Benchmarking Work 11:39 Collaboration with AMD and Nvidia 14:52 The Evolution of Inference Benchmarking 17:34 Optimizations and Their Impact 20:47 Challenges in Composability 23:51 Multi-Token Prediction Explained 26:52 Cost Implications of Optimizations 31:06 Understanding Inference Workloads and Benchmarks 33:44 Future Plans for Inference Optimization 37:16 Roadmap for New Models and Data Sets 39:03 Challenges in Benchmarking Multi-Turn and Multi-Modal Data 42:44 Experiences with AI Models and Their Limitations 48:43 Skepticism About Future AI Improvements
More By Jordan Nanos, Doug O'Laughlin...
Ep. 010 - How Much Do GPUs Really Cost, and Where Does the Value Go? (AI Cloud TCO) | Jordan Nanos, Dan Nishball, Kang Wen Cheang, Zane FongJordan Nanos, Doug O'Laughlin...Operationtag/External ChartsEp. 019 - Inside the STEEL Lab: From Package to Transistor (Teardown Lab) | Afzal Ahmad, Andrew Wagner, Jordan NanosJordan Nanos, Doug O'Laughlin...Operationtag/External ChartsEp. 016 - What Unitree's Evolution Means For Robotics (Robotics) | Jordan Nanos, Reyk Knuhtsen, Niko CiminelliJordan Nanos, Doug O'Laughlin...Operationtag/External ChartsEp. 020 - Anthropic vs OpenAI Usage, Margins, Meta Compute, Future of MSL (Tokenomics) | Crystual Huang, Max Kan, Joey Brookhart, Jordan NanosJordan Nanos, Doug O'Laughlin...Operationtag/External ChartsEp. 011 - GPT 5.5 vs Claude 4.7: OpenAI's Comeback From the Brink (Tokenomics) | Jordan Nanos, Dylan Patel, Doug O'Laughlin, Max KanJordan Nanos, Doug O'Laughlin...Operationtag/External ChartsEp. 015 - DG Matrix Explains 800V DC vs Legacy AC Distribution (Datacenter, Energy) | Jordan Nanos, Jeremie Eliahou Ontiveros, Nicolas Bontigui, Haroon InamJordan Nanos, Doug O'Laughlin...Operationtag/External ChartsEp. 006 - The AI Silicon Shortage Explained (AI Supply Chain & Fabs) | Sravan Kundojjala, Ivan Chiam, Jordan NanosJordan Nanos, Doug O'Laughlin...Operationtag/External ChartsEp. 012 - Cerebras IPO: 1,100 Tokens Per Second vs GPU's 500 (AI Cloud TCO) | Jordan Nanos, Howie , Myron XieJordan Nanos, Doug O'Laughlin...Operationtag/External ChartsEp. 008 Claude Code Psychosis: How SemiAnalysis Is Token Mogging Meta | Dan Nishball, Sam Harshe, Jordan NanosJordan Nanos, Doug O'Laughlin...Operationtag/External ChartsSemiAnalysis WeeklyJordan Nanos, Doug O'Laughlin...Operationtag/External Charts