高效架构上的高效模型.pdf

1、AI Hardware&SystemsaiandsystemsSteven Brightfield Efficient Models on Efficient ArchitecturesChief Marketing Officer2024 BrainChip Inc.1AI Hardware&Systemsaiandsystems4 Elementsof AIDataSoftwareHardwareElectricitySoftwareDataEnergyHardwareDatasets growing exponentially and the resulting model parame

2、tersPower costs rising in cloud,hard limits on power on the edge4 Elements of AI2024 BrainChip Inc.2AI Hardware&SystemsaiandsystemsModel Execution PowerNeural Model Complexity(operations/model)Neural Model Execution(operations/watt)=The Model Efficiency Equation4 Elementsof AIDataSoftwareHardwareEle

3、ctricitySoftwareDataEnergyHardware2024 BrainChip Inc.3AI Hardware&SystemsaiandsystemsUsing Foundation ModelsPruning and distillationFine tuningTrade off quality versus model sizeUse smaller context windowsRAG AssistanceMore efficient trainingIncremental trainingRelevant Subset trainingNew Foundation

4、 ModelsNew models suited for edge use casesNeural Model Complexity4 Elementsof AIDataSoftwareHardwareElectricitySoftwareDataEnergyHardware2024 BrainChip Inc.4AI Hardware&SystemsaiandsystemsAlgorithmic Compute Efficiency=Model Metric(PESQ,Perplexity,mAP)MACs/inference(power+area)Algorithmic Memory Ef

5、ficiency=Model MetricParameters(memory movement)The Neural Model Efficiency2024 BrainChip Inc.5AI Hardware&SystemsaiandsystemsNew NPU chip architecturesReduced precisionIn-memory computeAnalog computeHigh sparsity executionEfficient scheduling compilersDedicated Transformer acceleratorsOpticalQuantu

6、mNew siliconSmaller process nodesLower voltagesBetter heat dissipationNeural Model Execution4 Elementsof AIDataSoftwareHardwareElectricitySoftwareDataEnergyHardware2024 BrainChip Inc.6AI Hardware&SystemsaiandsystemsCompute Efficiency=Actual MACs/sec ComputedTotal MACs/sec PossibleWhat percentage of