DeepSeek's surprisingly inexpensive AI model challenges industry giants. The company claims to have trained its powerful DeepSeek V3 neural network for a mere $6 million using only 2048 GPUs, significantly undercutting competitors. However, this figure is misleading.
Image: ensigame.com
DeepSeek V3 leverages innovative technologies: Multi-token Prediction (MTP) for enhanced accuracy and efficiency; Mixture of Experts (MoE), utilizing 256 neural networks, to accelerate training and improve performance; and Multi-head Latent Attention (MLA) to focus on crucial sentence elements, minimizing information loss.
Image: ensigame.com
Contrary to their initial claim, SemiAnalysis revealed DeepSeek's extensive infrastructure, comprising approximately 50,000 Nvidia Hopper GPUs across multiple data centers, representing a total investment of roughly $1.6 billion and operational costs of $944 million. This includes substantial investment in personnel, with some researchers earning over $1.3 million annually.
Image: ensigame.com
DeepSeek, a subsidiary of High-Flyer, a Chinese hedge fund, owns its data centers, providing greater control and faster innovation implementation. This self-funded approach contributes to its agility. The $6 million figure only reflects pre-training GPU costs, excluding research, refinement, data processing, and infrastructure. DeepSeek's actual investment in AI development exceeds $500 million.
Image: ensigame.com
While DeepSeek's success stems from substantial investment, technological advancements, and a skilled team, its "budget-friendly" narrative is an oversimplification. However, even with the corrected figures, DeepSeek's costs still pale in comparison to competitors. For example, DeepSeek's R1 model cost $5 million to train, while ChatGPT-4 cost $100 million. The company's efficiency and focused structure, however, allow it to compete effectively with industry giants, despite the substantial investment involved.
