HC2022_Cerebras_Final_v02.pdf

1、 2022 Cerebras Systems Inc.All Rights Reserved 2022 Cerebras Systems Inc.All Rights ReservedCerebras SystemsSean LieCo-founder&Chief Hardware ArchitectCerebras Architecture Deep Dive:First Look Inside the HW/SW Co-Design for Deep Learning 2022 Cerebras Systems Inc.All Rights ReservedBuilding and dep

2、loying a new class of computer systemDesigned for the purpose of accelerating AI and changing the future of AI workFounded in 2016400+Engineersin 14 CountriesEngineering OfficesSilicon Valley|San Diego Toronto|BangaloreCustomersNorth America|Asia|EuropeCerebras Systems 2022 Cerebras Systems Inc.All

3、Rights ReservedCustomers:Large Enterprise,HPC;Military and IC GlaxoSmithKline,TotalEnergies,AstraZeneca,Bayer,Genentech,Tokyo Electron Devices.ANL,LLNL,NETL,PSC,NCSA,EPPC,Leibniz Supercomputing Centre.Security,e.g.DARPA,USAF,ARLSelect Cerebras Customers3 2022 Cerebras Systems Inc.All Rights Reserved

4、Over 1000 x increaseIn just 2 yearsTomorrow,multi-trillion parameter modelsExponential Growth of Neural Networks 1 10 100 1,000 10,000 100,000 1 10 100 1,000 10,000 100,000Total training compute,PFLOP-daysModel memory requirement,GBMemory and compute requirementsBERT Base(110M)BERT Large(340M)2018 1

5、 10 100 1,000 10,000 100,000 1 10 100 1,000 10,000 100,000Total training compute,PFLOP-daysModel memory requirement,GBMemory and compute requirementsBERT Base(110M)BERT Large(340M)2018GPT-2(1.5B)Megatron-LM(8B)T5(11B)2019 1 10 100 1,000 10,000 100,000 1 10 100 1,000 10,000 100,000Total training comp

6、ute,PFLOP-daysModel memory requirement,GBMemory and compute requirementsBERT Base(110M)BERT Large(340M)2018GPT-2(1.5B)Megatron-LM(8B)T5(11B)2019T-NLG(17B)GPT-3(175B)MSFT-1T(1T)2020+MT-NLG(530B)1 10 100 1,000 10,000 100,000 1 10 100 1,000 10,000 100,000Total training compute,PFLOP-daysModel memory re