深度探讨技术：退化AI？如果 GenAI 已经达到极限那么它的下一次进化是什么？.pdf

1、QATAR COMPUTING RESEARCH INSTITUTE1Dr.Sabri BoughorbelDegenerativeAI?If GenAI Has Already Hit its Limit,What is its Next Evolution?QATAR COMPUTING RESEARCH INSTITUTE2Outline2 Review of AI progress Challenges on the road of AI Promising directionshttps:/nishu- COMPUTING RESEARCH INSTITUTE3GenAI Under

2、 the Hood3QATAR COMPUTING RESEARCH INSTITUTE4AI Progress4 Compute expanding at 4x per year Fastest technological expansions in recent historyo Mobile adoption(2x/year)o Solar energy capacity(1.5x/year)o Human genome sequencing(3.3x/year)o Moores Law(1.3x/year)QATAR COMPUTING RESEARCH INSTITUTE5AI Pr

3、ogress5 AI Performance is predictableo Scaling of model,data and compute Would this lead to capability explosion?o Task automationo Fast economy growthQATAR COMPUTING RESEARCH INSTITUTE6Monetization of AI 6 Nvidia:o 100B annual revenue projected for Nvidia from data center OpenAI:o$1B revenue in Aug

4、 2023o$2B revenue in Feb 2024 o doubling every 6 months Microsoft:o$5B estimation of incremental AI revenueQATAR COMPUTING RESEARCH INSTITUTE7Models are Reaching a ceiling?7.Ilya Sutskever,co-founder of AI labs Safe Superintelligence(SSI)and OpenAI,told Reuters recently that results from scaling up

5、pre-training-the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures have plateaued.Nov 11,2024 QATAR COMPUTING RESEARCH INSTITUTE8Challenges8 Data Wall Scaling Training Runs QATAR COMPUTING RESEARCH INSTITUTE9Will We Run Out of Data

6、?9https:/epoch.ai/blog/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-dataQATAR COMPUTING RESEARCH INSTITUTE10Will we run out of data?10 Data from other modalities Enterprise data Synthetic dataQATAR COMPUTING RESEARCH INSTITUTE11Synthe