冷却人工智能！：比较液冷和风冷人工智能节点的功耗和吞吐量性能.pdf

1、Imran Latif,Muhammad Ali ShafiqueCool IT!:Comparing Power and Throughput Performance of Liquid and Air-cooled AI nodesCool IT!:Comparing Power and Throughput Performance of Liquid and Air-cooled AI nodesImran Latif,Muhammad Ali ShafiqueOCP SPECIAL FOCUS:ARTIFICIAL INTELLIGENCE(AI)Intensive Compute N

2、eeds:Training&operating LLMs requires massive GPUsRapid Demand Growth:Data center power demand to rise 165%by 2030(Goldman Sachs Research)National Impact:U.S.data centers may consume 6.7%12%of total electricity by 2028(Shehabi et al.2024)Soaring AI Energy ConsumptionAddressing the challengesAir-Cool

3、ed vs Liquid-Cooling:Impact on ITDirect-to-chip(DTC)liquid cooling reduces facility energy,but how does it affect the IT?Tested identical workloads on two 8-H100 nodes,one DTC liquid and one air cooledWorkloads included Stress tests,VLM pretrainingLLM fine tuningExperimental Setup-MetricsCore Findin

4、gs GPU Burn TestMetricAir-CooledDTC Liquid-CooledIDLE POWER(W)2,1741,776GPU BURN NODE POWER(W)8,1616,992Avg.TFLOPS4654Avg.TEMP(C)62.545.8Core Findings GPU Temperature StabilityLiquid-cooled(left)and air-cooled(right)GPU temperature training Vita-CLIPCore Findings Reduced Node Power17.7%19.7%19.8%18.

5、6%17.0%1.40%8.80%20.4%15.4%Core Findings Reduced Node Power5,000 nodes$11.8M/year savings91 GWh annual energy savedLiquid cooling PUE:1.2 vs.air cooling:1.3Facility-Level Impact Aggressive ScenarioThe Liquid Cooling Advantage“As AI scales,so must our thermal strategy.The time to switch is now.”1.Glo

6、bal data center industry wide need to define KPIs based on compute efficiency such as TFLOPS per watt,token per watt,token per grid.2.Reference design Architectures for liquid cooled GPU based AI accelerated hardware3.Join OCP Projects on S