SemiAnalysis: Kimi K3 Reduces KV Transmission Bandwidth, AI Network Demand Remains Strong

On July 19, SemiAnalysis reported that approximately three-quarters of the Kimi K3's network layer utilizes KDA, allowing KV cache transmission bandwidth to be reduced by up to 10 times. However, this will not lead to a significant contraction in the AI switch market size. The Kimi K3 features 28 trillion parameters and requires about 1.5TB of HBM bandwidth for each forward computation when using MXFP4. To maintain reasonable interaction speeds and achieve profitable deployments, it is still necessary to connect a large number of chips through high-bandwidth networks like GB300 NVL72 and rely on WideEP expansion services. WideEP distributes 896 expert models across multiple GPUs and executes token distribution and result merging twice in each layer and for each forward computation, requiring over 120 executions per single forward computation. In contrast, KV cache transmission occurs only once per round of dialogue, meaning the bandwidth savings from KDA may be significantly less than the expanded network demand brought about by large-scale expert models. SemiAnalysis believes that more efficient attention mechanisms could push the context length from 1 million tokens to over 5 million tokens, and the efficiency gains may expand AI usage, further increasing network demand.

Disclaimer: This content is provided for general branding and informational purposes only and doesn't constitute financial, investment, legal, or tax advice. Any events, rewards, online events, or related information mentioned herein should not be considered a recommendation, solicitation, or invitation to purchase, sell, trade, or otherwise deal in any crypto assets or to use any services. Crypto assets are highly volatile and may result in loss. WEEX services and online events may not be available in all regions and are subject to applicable laws, regulations, and eligibility requirements. You are responsible for ensuring that your use of WEEX services complies with local laws and for carefully assessing the risks before participating in any crypto-related activities.