AI 模型基于信息的软件优化.pdf

1、AI Hardware&SystemsaiandsystemsInformation-BasedSoftware Optimization of AI ModelsGerald Friedland,Principal Scientist AWS|Adjunct Faculty,UC BerkeleyDisclaimer:The views and opinions expressed in this presentation are my own and do not necessarily reflect the official policy or position of Amazon o

2、r its affiliates.https:/ Hardware&SystemsaiandsystemsBook on this Topic 200 downloads per day(e-book)Spanish version soon.Jupyter Notebooks with algorithms as code:https:/ Hardware&SystemsaiandsystemsHow much hardware(money,cpu time,memory,IO)do I need to budget for my AI model?State of the Art:No a

3、nswer.For example,ImageNet models vary significantly:AlexNet:238MB model,2.27Bn OpsDarkNet:28MB model,0.96Bn OpsVGG-16:528 MB,30.94Bn OpsSource:https:/ QuestionAI Hardware&SystemsaiandsystemsThe Scientific MethodData Science:The Science of Automating the Scientific MethodAI Hardware&Systemsaiandsyst

4、emsThe Scientific Method:Traditional=2AI Hardware&Systemsaiandsystems=2The Scientific Method:NewAI Hardware&SystemsaiandsystemsAssume(binary classifier)Question:How many state transitions does M need to model the training data?,()0,1Conceptual FrameworkAI Hardware&SystemsaiandsystemsEvery row gets i

5、ts own arrow:Lookup Table/DictionaryBest accuracy.No Generalization.State Machine Models Table:OverfitAI Hardware&SystemsaiandsystemsState Machine Models Table:Over generalOne arrow:Maximum generalization.Best guess accuracy.AI Hardware&SystemsaiandsystemsOptimization Problem:How many state transiti

6、ons does M need to model the training data for maximum accuracy and best generalization?Machine LearningAI Hardware&SystemsaiandsystemsCS explains Machine Learning Intellectual Capacity:The number of unique target functions a machine learner is able to represent(as a function of the number of model