SambaNova demonstrates first disaggregated inference for AI agents
HIGHLIGHTS
- •SambaNova Systems demonstrated the first disaggregated inference for AI agents
- •Using Nvidia B200 for prefill and SN40 RDU for decode, achieving 2x faster speeds than B200-only configurations, and operational at Vector Core Compute's (VC2) data center with Together.ai as the first commercial customer.
- •This new architecture targets 10x throughput at 500 tokens per second per user on MiniMax M2.7, expected in the second half of the year.
Published:
Official Source (Company Blog)