1、arXiv:2505.09343v1 cs.DC 14 May 2025This is the authors version of the work.It is posted here for your personal use.Not for redistribution.The definitive version will appear as part of the Industry Track in Proceedings of the 52nd Annual International Symposium on Computer Architecture(ISCA 25).Insi

2、ghts into DeepSeek-V3:Scaling Challenges and Reflections onHardware for AI ArchitecturesChenggang Zhao,Chengqi Deng,Chong Ruan,Damai Dai,Huazuo Gao,Jiashi Li,Liyue Zhang,PanpanHuang,Shangyan Zhou,Shirong Ma,Wenfeng Liang,Ying He,Yuqing Wang,Yuxuan Liu,Y.X.WeiDeepSeek-AIBeijing,ChinaAbstractThe rapid

3、 scaling of large language models(LLMs)has unveiledcritical limitations in current hardware architectures,including con-straints in memory capacity,computational efficiency,and intercon-nection bandwidth.DeepSeek-V3,trained on 2,048 NVIDIA H800GPUs,demonstrates how hardware-aware model co-design can

4、effectively address these challenges,enabling cost-efficient trainingand inference at scale.This paper presents an in-depth analysis ofthe DeepSeek-V3/R1 model architecture and its AI infrastructure,highlighting key innovations such as Multi-head Latent Attention(MLA)for enhanced memory efficiency,M

5、ixture of Experts(MoE)architectures for optimized computation-communication trade-offs,FP8 mixed-precision training to unlock the full potential of hard-ware capabilities,and a Multi-Plane Network Topology to minimizecluster-level network overhead.Building on the hardware bottle-necks encountered du

6、ring DeepSeek-V3s development,we engagein a broader discussion with academic and industry peers on po-tential future hardware directions,including precise low-precisioncomputation units,scale-up and scale-out convergence,and in-novations in low-latency communication fabrics.These insightsunderscore