2024AI硬件和边缘AI峰会（AI Hardware &ampamp； Edge AI Summit 2024）嘉宾演讲PPT合集（共30套打包）

Mentium Technologies All Rights Reserved 9/13/20241 2024 Mentium Technologies Inc.MENTIUM HYBRID COMPUTATION FOR MISSION-CRITICAL EDGE AI,PARTNERING WITH SYNOPSYSMirko Prezioso,Co-founder&CEO,Mentium TechnologiesSeptember 10,2024 2024 Mentium Technologies Inc.Mentium Technologies History Spin-off from UCSB In 2015,we demonstrated the first DNN in-memory computing in an integrated array.Mentium,an AI Co-processors Startup,was founded in 20172 2024 Mentium Technologies Inc.Cloud-level AI on EdEric StotzerSr.Group Director NeuroWeaveTMSDKThe Challenges of Creating Software Development Tools for Diverse AI Hardware Architectures 2024 Cadence Design Systems,Inc.All rights reserved.2AI at the Edge The Software Development Tools Challenge 2024 Cadence Design Systems,Inc.All rights reserved.3AI at the Edge The Software Development Tools ChallengeApplications 2024 Cadence Design Systems,Inc.All rights reserved.4AI at the Edge The Software Development Tools ChallengeNetworksHardwareApplicatiAI Hardware&SystemsaiandsystemsAI for All:Pushing Infra BoundariesManoj WadekarAI System Technologist,MetaAI Hardware&SystemsaiandsystemsAI-enabled creation toolsText-to-image generationsurrealist paintingLarge language models(LLMs)+173%Source:Meta for Business.Culture Rising:2023 Trends Report.2023.Conversation topic growth on InstagramMeta AI is used for diverse casesSource:Meta for Business.Culture Rising:2023 Trends Report.2023.AI Hardware&SystemsaiandsystemsGenAI runs on Large LFor Reg.AC certification and important disclosures,see the last page(s)of this report.AI Hardware:The Second Wave(2025-2027)11 September 2024Brett Simpson,AnalystArete Research Services LLP+44(0)20 7959 1320Janco Venter,AnalystArete Research Services LLPJ+27 71216 3220Nam Hyung Kim,AnalystArete Research,LLC +1 424 228 79142Key Summary TakeawaysWe see a second wave of AI growth coming in the 2025-2027 period.Training clusters are scaling up,while we see AI inference demand layering in as new archAI Hardware&SystemsaiandsystemsMicro-Prompting LLMs and SLMsFrom Copilots to Agentic WorkloadsDonald ThompsonDistinguished EngineerMicrosoft/LinkedInAI Hardware&SystemsaiandsystemsAgenda Macro-prompting vs micro-prompting Micro-prompting example Automatic prompt optimization Automatic fine-tuning SLMs vs LLMsAI Hardware&SystemsaiandsystemsMacro-PromptingThe Current Paradigm in GenAI Dominant approach since late 2022 Crafting extensive,detailed prompts Provides comprehensive instructiAgents,Inference and Hardware Andrew Ng LandingAI All rights reserved.Technical Trends In AIOn-device AI.Instead of running an LLM in the cloud,run it on your own laptop,phone or industrial PC.Image/Video analysis.LLMs brought us the text processing revolution.The visual processing revolution is coming not just generation,but analysis.This will affect,manufacturing,life sciences,self-driving,retail,etc.AI Agentic Workflows.Given an instruction(“research topic X for me”)software that can carry AI Hardware&SystemsAI Hardware&SystemsaiandsystemsaiandsystemsThe Gigawatt GambleThe Economics,Impact,and an Alternative Path for Global AI Inference DeploymentThomas SohmersCEO,Positron AIthomaspositron.aiAI Hardware&SystemsAI Hardware&SystemsaiandsystemsaiandsystemsIntroductionIntroductionForbes 30 under 302013 Thiel FellowMIT ResearcherCEO&Co-FounderDirector of Technology StrategyPrincipal Hardware ArchitectCEO&Co-FounderStudent&EntrepreneurThomas SohmersCEO,PositrINNOVATIVE TECHNOLOGIES SUDDEN SERVICE GLOBAL REACHOptimizing Data Routing via Copper&Optical Interconnects in Scalable AI Hardware ArchitecturesMatthew BurnsGlobal Director,Technical MarketingSeptember 10,2024Samtec ConfidentialOptimized Data Routing-How?1.21 GW!224 Gbps/SIFabricsDisaggregationThermal ReliefScalabilityMemoryAccessOpticsSamtec ConfidentialKey AI Hardware ApplicationsCHIPSETSSoMs/CoMsACCELERATORSDSAsSamtec ConfidentialAI Chipsets/Characterization PlatformsA number of emergingAI Hardware&SystemsaiandsystemsInformation-BasedSoftware Optimization of AI ModelsGerald Friedland,Principal Scientist AWS|Adjunct Faculty,UC BerkeleyDisclaimer:The views and opinions expressed in this presentation are my own and do not necessarily reflect the official policy or position of Amazon or its affiliates.https:/ Hardware&SystemsaiandsystemsBook on this Topic 200 downloads per day(e-book)Spanish version soon.Jupyter Notebooks with algorithms as code:https:/ Hardware&Systems 2023 Celestial AI.All Rights Reserved.Celestial AI ConfidentialPhotonic FabricTMbased Scale-Up Network for Chip-to-Chip&Chip-to-Memory ConnectivityPreet VirkCo-Founder&COO,Celestial AI 2024 Celestial AI Inc.,All Rights Reserved.Celestial AI,the C logo,and Photonic Fabric are trademarks or registered trademarks of Celestial AI Inc.in the United States and other countriesAI Hardware Summit,San Jose11th Sep 2024AI The Largest Technological Wave We Have Ever Seen2Notes:1.Source:The economicAI Hardware&SystemsaiandsystemsEfficiency Challenges in GenomicsTom Sheffler AI Hardware&SystemsaiandsystemsPreface Goal is to give insights into the characteristics of genomics computations Explain AI/ML on the Edge for DNA processing Challenges in AI/ML for genomics(from the real world)AI Hardware&SystemsaiandsystemsGenomics Applications why does it matter?Cancer Screening identify DNA changes that increase a persons risk guide selection of therapies Whole Genome Sequencing for newDan RabinovitsjVP,Engineering MetaResilient AIBuilding Fault-Tolerant AI SystemsArtificial intelligence(AI)is having quite a momentAI-enabled creation toolsText-to-image generationA hedgehog playing chessLarge Language Models(LLMs)Llama 3.1Source:Meta for Business.Culture Rising:2023 Trends Report.2023.pushed our model training tonew heights,leveraging a significantly optimized full training stack16K H100 GPUsused to train Llama 3.1 405B15T tokensTRAINED AT UNPRECEDENTED SCALEThe Challenge of ScAI Hardware&SystemsaiandsystemsInside Microsoft AI hardware innovationMark RussinovichCTO,Deputy CISO and Technical Fellow,Microsoft AzuremarkrussinovichMicrosoft AI supercomputer10,000 V100 GPUs#5 supercomputerMay 202014,400 H100 GPUs#3 in TOP500Nov 202330 x supercomputersMay 2024AcceleratorsCoolingNetworkingPowerConfidential AICoolingNetworkingAcceleratorsPowerConfidential AIDiverse accelerators on AzureA100,H100(available today)H200,GB200(coming soon)MI300 x(available today)Azure Maia 100AI Hardware&SystemsaiandsystemsMastering AI Cluster ManagementPhil Pokorny Phil Pokorny CTO Penguin Solutions AI Hardware&SystemsaiandsystemsConsider thisFor an organization to make effective use of an AI cluster,it is important to take into consideration the entire process of designing,building,deploying and managing the resource.At each step,a cluster for AI presents new and different challenges that even experienced IT team members may not have encountered before.AI Hardware&SysteConfidential Information Nscale Group B.V.2024Maximising GenAI Performance:The GPU&BeyondConfidential Information Nscale Group B.V.2024Karl HavardCOO NscaleSeptember 11,2024Maximising GenAI PerformanceConfidential Information Nscale Group B.V.202401 FactorsThe key elements of GenAI performance,its not just the GPU.02 MaturityAs the market matures,values,priority and focus will change03 CorrelationGenAI performance will directly correlate to business performanceContextMaximising GenAI PerformAll Rights Reserved Arun Nandi-2024Architecting Intelligence:Next-Generation Databases for RAG-Powered AIArun NandiHead of Data and Analytics at UnileverBoard Advisor Top 100 Leaders in AI Keynote SpeakerAll Rights Reserved Arun Nandi-20242What does RAG do?All Rights Reserved Arun Nandi-2024Retrieval Augmented Generation Schema1All Rights Reserved Arun Nandi-20243All Rights Reserved Arun Nandi-2024Query Routing and RAGImproved Accuracy and RelevanceEnhanced compute efficiencyModel agnosticismScaAI Hardware&SystemsaiandsystemsWhats Next In Generative AI?Future Looking Trends In LLM DesignBaskar SridharanVice President,AI/ML Services&InfrastructureAI Hardware&SystemsaiandsystemsGenerative AIAI Hardware&SystemsaiandsystemsThe year of POCsWhat does this mean for my business?What is a foundation model?What is a large language model?What is generative AI?Do I need to become a prompt engineer?Is this secure?How do I choose a model?Where do I get started?Which models should we Broadcom Proprietary and Confidential.Copyright 2024 Broadcom.All Rights Reserved.The term“Broadcom”refers to Broadcom Inc.and/or its subsidiaries.|Ethernet for AI Scale Hasan SirajHead of Software and AI Infrastructure Products,BroadcomBroadcom Proprietary and Confidential.Copyright 2024 Broadcom.All Rights Reserved.The term“Broadcom”refers to Broadcom Inc.and/or its subsidiaries.|Exponential Acceleration of Compute for AIOptimized forSerial TasksGPUOptimized forParallel TasksCPUMultiple CoAnton McGonnellVP of ProductSept 10,2024Sept 10,2024Beyond GPUs:Powering the Next Wave of AIv 1.0Copyright 2024 SambaNova Systems Inc.|Confidential&Proprietary|Internal Use Only2 2The Need for SpeedSpeed and Latency are important Speed and Latency are important criteria for Gen AI Developers criteria for Gen AI Developers Artificial AnalysisArtificial Analysis65%Building Agents Requires Many Building Agents Requires Many Models and Faster RealModels and Faster Real-Time Time InferenceInferenFuriosaAI Inc.AI Hardware Summit 2024Hyunsik Choi,Head of SW Platform,Jihoon Yoon,Product Marketing ManagerBeyond Just Hardware Full-stack Optimization Towards Efficient AI InferenceFuriosaAI Inc.AI Hardware Summit 2024FuriosaAI founded&Launch Gen 1 vision NPU RNGD raw silicon sample arrivalFirst LLM demo 2017-2021 2024 May2024 JulyGPT3 inspired RNGD 2021 RNGD DevelopmentKick off 2022 FuriosaAI Inc.AI Hardware Summit 202401Mass AI adoption is bottlenecked02 Energy efficient AI inference03 FuAI Hardware&SystemsaiandsystemsBeyond the Edge:Revolutionizing AI Workloads with Axelera AIs Digital In-Memory Computing and RISC-V technology AI Hardware&Systemsaiandsystems Co-founded in July 2021,by Fabrizio Del Maffeo and Evangelos Eleftheriou,with 16 founding team members from IBM,ETH Zurich,IMEC,Bitfury AI,Google and Qualcomm.Our team has grown to 180+people,including 60+PhDs and are present in 16 countries.Our first product,Metis,is the most powerful AI processing unit for computeTim MamtoraChief of Innovation&Engineering,ImaginationSpeaker:A software-first approach to Huge societal opportunity for AI to“do good”Source:IDC,#US50888824,May 2024Worldwide Edge Endpoint AI Processor and Accelerator Forecast,20242028Challenges at the edge are all too familiarThe biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective,and by a large margin”Two key lessons:Scalable and OpenDirectly cAI HARDWARE&EDGE AI SUMMITSoftware-Defined Memory:Tackling AIs Challenges Now with Memory FlexibilityTodays PresentersJohn OvertonCEO,KoveBill WrightEdge AI Technology Evangelist,Red HatWith the rise of AI/ML combined with an increase in edge computing,containerization and other memory-hungry considerations enterprises have been stretching the limits of their technologists,their infrastructure and their memory.2Memory demand is exploding but traditional servers arent built to keep pace.With Agnostic Software for AI WorkloadsDecoupling Software from Hardware2Change is ComingAdapt or3Workloads are Evolving at an Accelerated PaceModels are increasing in size exponentially,with no foreseeable slow downNetwork architectures are changing rapidlyTransformers are not the only solution that will existThe balance between training and inference is 50/50,even at scaleThe progression from ML to AI to Agents is not slowing downInsight:Observations:4Machine Learning is the Gateway to Autonomous AThis presentation is intended solely for the audience of this session only.Unauthorized sharing or editing is strictly prohibited.This presentation is intended solely for the audience of this session only.Unauthorized sharing or editing is strictly prohibited.2This presentation is intended solely for the audience of this session only.Unauthorized sharing or editing is strictly prohibited.3This presentation is intended solely for the audience of this session only.Unauthorized sharing or editing iEnabling IntelligenceThrough Sustainable ComputingUnrestricted|AI Hardware Summit|Siemens 2024|Siemens Digital Industries SoftwaresustainabilitynounThe ability to be maintained at a certain rate or level:the sustainability of economic growth the long-term sustainability of the project“Avoidance of the depletion of natural resources to maintain an ecological balance:the pursuit of global environmental sustainability the ecological sustainability of the planetUnrestricted|AI Hardware Summit|Sieme1Alphawave SEMI All Rights Reserved.Accelerating the Connected WorldRedefining Connectivity:Charting Next-Gen Pathways in Chiplet InterconnectsTony Chan Carusone,CTOAI Hardware and Edge AI SummitSeptember 11,20242Alphawave SEMI All Rights Reserved.Delivering Custom Silicon in the Data CentreCompute CPUCustom xPUsNetworkingCustomised compute and connectivity reliant on chiplets3Alphawave SEMI All Rights Reserved.Outline Scaling AI with Connectivity The Role of Chiplets for AI Compute Enabling a CAI Hardware&SystemsaiandsystemsSteven Brightfield Efficient Models on Efficient ArchitecturesChief Marketing Officer2024 BrainChip Inc.1AI Hardware&Systemsaiandsystems4 Elementsof AIDataSoftwareHardwareElectricitySoftwareDataEnergyHardwareDatasets growing exponentially and the resulting model parametersPower costs rising in cloud,hard limits on power on the edge4 Elements of AI2024 BrainChip Inc.2AI Hardware&SystemsaiandsystemsModel Execution PowerNeural Model Complexity(operations/m