CXL Solutions for Key AI Problems.pdf

1、Samsung-MemVerge CollaborationCXL Solutions for Key AI ProblemsKristian Vtt,Product Manager,SamsungAccelerating AI Workloads with Shared MemoryArtificial Intelligence(AI)AIMemoryNICACCTop 3 Memory-Related Industry ChallengesMemoryNICACCMemoryNICACCMemoryCPUNICACCStorageACCACCACCStorageMemoryCPUMemor

2、ySmartNIC00.511.522.533.5420122014201620182020CPU core countMemory channel BW per coreNew memory scaling solution is neededNICACCNICACCMemoryNICACC”Up to 25%of memory is stranded”Only 40%of DRAM is usedIn production clusters”DRAM makes up 50%ofthe server cost”#1 Changing Architecture#2 Memory Bandwi

3、dth Scaling#3 Memory StrandingAI Requires Faster&Higher Capacity MemoryVarious new memory solutions are coming to market to address emerging requirements(29yr)CMM-D Memory Expander is ShippingPCIe Gen5 x8 with 32GB/s BWCXL 2.0 support(pooling,hot-plug)256GB in E3.S 2T is available nowSupporting RAS/

4、Security based on memory controllerAdditional bandwidth from CXL improves inference per minuteCMM-D Performance in AI ActionMemoryNICACCMemoryNICACC+Add instances on CXLInference per minuteNumber of instancesThroughput saturationInstances on DRAMDIMM 2CH/CXL CMM-D1.011.351.642.002.891.00 1.17 1.30 1

5、.45 1.79 DDR 2CH+1-CXL+2-CXL+4-CXLk-CXL(Theoretical)BERTNASNetTest ScenarioBenchmark ResultIntroducing CXL Memory Module Box(CMM-B)PCIe 5.0 x88x E3.SPCIe 5.0 x8Compute SystemCompute SystemCXL.memCXL.memDisaggregated memory poolScalability with CXL 1.1/2.0 for capacity&bandwidth expansionFlexible arc

6、hitecture with composable memoryPower efficient CXL SoC switchSamsung”Cognos”fabric manager for dynamic memory allocation&performance monitoringUp to 2TB memory per box(8x 256GB E3.S)Shared memory enables dynamic allocation&on-demand performance scaling CMM-B Shared Memory in ActionMemoryNICACCMemor