On July 16, 2026, Moonshot AI introduced Kimi K3, a significant advancement in artificial intelligence technology. This model is designed with 2.8 trillion parameters, making it one of the largest open Mixture of Experts (MoE) models available.
Kimi K3 utilizes Kimi Delta Attention and Attention Residuals, which enhance its performance by activating 16 of the 896 available experts. This innovative approach allows for more efficient processing and improved accuracy in various AI applications.
With a context length of 1 million tokens, Kimi K3 is poised to handle complex tasks and large datasets, setting a new standard in the field of AI and machine learning.