徐梦炜-终端大模型操作系统的架构、优化与展望.pdf

1、端侧大模型操作系统的架构、优化与展望Mengwei Xu(徐梦炜)目录010203040605Mobile intelligence before LLMOn-device LLMThe changes LLM brings:AppThe changes LLM brings:OSThe changes LLM brings:H/WTakeawaysMobile intelligence before LLMMobile intelligence before LLMDNN-embedded mobile apps Increased by almost 10 x(2018 to 2021)1

2、,2 Downloaded billions of times in one year Include almost every high-popularity app Up to 200+DNNs in a single app31 Mengwei Xu,et al.“A First Look at Deep Learning Apps on Smartphones”.In WWW 20192 Mario Almeida,et al.“Smart at what cost?Characterising Mobile Deep Neural Networks in the wild”.In I

3、MC 2021.3 Through offline communication with application developers.DNNs already run on devices at large scaleWhat we expect as mobile intelligenceWhat we expect as mobile intelligenceComprehendhuman languageReasoning&PlanningZero-shot&in-context learningMultimodalalignmentInstruct.followingA device

4、 that canThe opportunity:LLMThe opportunity:LLM To bring mobile devices the“next-level”intelligenceComprehendhuman languageReasoning&PlanningZero-shot&in-context learningMultimodalalignmentInstruct.followingOnOn-device LLMdevice LLM On-device LLMs handle language tasks in a way that is.cost-efficien

5、t(important,obviously)more available(even w/o network)faster(not always)privacy-preserving(very important,LLMs can leverage almost every bits of local data)LLMs on devices does not obviate mega-scale LLMs on clouds!-Creating music/poetry,solving math problems,etc.1 Jiajun Xu,et al.“On-Device Languag

6、e Models:A Comprehensive Review”.In preprint24.OnOn-device LLMdevice LLM We already have a mobile device that can function with high intelligence!A mobile device that can comprehend,reason,and plan without a cloud!Call for fullCall for full-stack designstack design Our response:agent-model-runtime-O