constructing-the-10x-efficiency-of-cloud-native-ai-infrastructure-matsu-zha-ai-xia-10-dyags-peter-pan-daocloud-xie-zuo-daocloud.pdf

1、Constructing10 x Efficiency of Cloud-Native AI InfrastructureQiuping DaiPeter PanConstructing10 x Efficiency of Cloud-Native AI InfrastructureQiuping DaiPeter PanAbout usQiuping DaiPeter PanProduct Manager of Cloud Native was major designer of d.runskilled at network,storage,GPU areaCloud Native Dev

2、 Lead employed by DaoCloudopen source advocateBackground工业和信息化部、国务院国资委等6部门联合印发算力基础设施高质量发展行动计划，提出到建设规划：2025年算力规模超过300EFLOPS，智能算力占比达到35%，By 智研瞻：中国人工智能数据中心（AIDC，智算中心）行业发展前景与投资战略规划分析报告MIIT+State Council Action Plan for the High-quality Development of Computing Power Infrastructure By 2025,up to 300EFLop

3、s,AI computing reaches 35%,Analytical report AIDC industrial trend in ChinaAIDC 50%growth rate2024202520262027202820292030AIDC Market scale(100 millon RMB)Increase ratio(%)Government IndustryChallengeOpportunityTechnical ObstacleGPU Supply-Demand ImbalanceLow Utilization and efficiency Immature Oper

4、ating ExperienceChallenges-GPU Supply&UtilizationGPU Supply ShortageNorth Beta Labs400300050100150200250300350400450GPU Supply in 2024（10,000 Cards）Demand in TrainGrowth of Train GPUs2500369050010001500200025003000GPU Supply in 2024（10,000 Cards）Demand in inferenceGrowth of inference GPUsGPU Power U

5、sageBatch sizeGPU Mem Usage（MiB）Batch sizeLow UtilizationLow Efficiency-Reasons1.Network Bottom Neck-Data communication in low efficiency2.Storage Read&Write Efficiency-Dataset load is slow-checkpoint saving is slow3.GPU Allocation is not Optimal-GPU Scheduling is not matching the AI scenario-Alloca

6、ted GPU is more than required in Inference scenario4.GPU Fault waste Training time Success Storyd.run as a computing-hub for AIDC across China.empower tens of AIDCBoth SaaS and On-PremisesGPU cost:48%savingGPU avg utilization:25%-54%11 based on statistic of two A100 clusters managed by d.runOptimiza