Ep. 002 - InferenceX 2.0 Release (Technical Staff) | Cam Quilici, Bryan Shan, Doug O'Laughlin, Jordan NanosTRACK

Sound

Ep. 002 - InferenceX 2.0 Release (Technical Staff) | Cam Quilici, Bryan Shan, Doug O'Laughlin, Jordan Nanos

51:22 duration0 plays0 favoritesMedium quality
Play now

About this track

InferenceX, formerly InferenceMAX: https://inferencex.com/ Article (latest): https://newsletter.semianalysis.com/p/inferencex-v2-nvidia-blackwell-vs GitHub https://github.com/SemiAnalysisAI/InferenceX Article (original) https://newsletter.semianalysis.com/p/inferencemax-open-source-inference 00:00 Introduction to InferenceX 02:52 Evolution from InferenceMAX to InferenceX 06:06 Benchmarking and Performance Insights 08:43 The Scale of Benchmarking Work 11:39 Collaboration with AMD and Nvidia 14:52 The Evolution of Inference Benchmarking 17:34 Optimizations and Their Impact 20:47 Challenges in Composability 23:51 Multi-Token Prediction Explained 26:52 Cost Implications of Optimizations 31:06 Understanding Inference Workloads and Benchmarks 33:44 Future Plans for Inference Optimization 37:16 Roadmap for New Models and Data Sets 39:03 Challenges in Benchmarking Multi-Turn and Multi-Modal Data 42:44 Experiences with AI Models and Their Limitations 48:43 Skepticism About Future AI Improvements

From the album

SemiAnalysis Weekly album coverSemiAnalysis Weekly22 tracks · Track 2

Everything semiconductors and AICovering the spectrum...

→

More By Jordan Nanos, Doug O'Laughlin...

You may also like

Hygge cover art
HyggeStaring at the SunMusic