SemiAnalysis — how mixture-of-experts models map onto inference hardware
pick
SemiAnalysis — how mixture-of-experts models map onto inference hardware
· SemiAnalysis
Source: View source · SemiAnalysis · web
Why it's worth your time
A structural piece on computation and data movement when serving MoE models — the part of the AI stack where memory bandwidth and interconnect, not raw compute, set the cost. Worth the read for anyone holding a view on memory or interconnect demand from inference rather than training.
Related
10 eventsSource links
Tweets, videos, filings, articles, and source pages tied to this read.