openyurt-dragonfly-enhancing-efficient-distribution-of-llms-in-cloud-edge-collaborative-scenarios-openyurtredragonflydaepyi-pencellmzha-pan-linbo-he-alibaba-cloud-jim-ma-ant-group.pdf

《openyurt-dragonfly-enhancing-efficient-distribution-of-llms-in-cloud-edge-collaborative-scenarios-openyurtredragonflydaepyi-pencellmzha-pan-linbo-he-alibaba-cloud-jim-ma-ant-group.pdf》由会员分享，可在线阅读，更多相关《openyurt-dragonfly-enhancing-efficient-distribution-of-llms-in-cloud-edge-collaborative-scenarios-openyurtredragonflydaepyi-pencellmzha-pan-linbo-he-alibaba-cloud-jim-ma-ant-group.pdf（28页珍藏版）》请在三个皮匠报告上搜索。

1、OpenYurt&Dragonfly:Enhancing Efficient Distribution of LLMs in Cloud-Edge CollaborativeLinbo He,Alibaba cloudJim Ma,Ant GroupAgendaEdge Computing and AIOpenYurtDragonflyPracticeEdge Computing and AIThe evolution of large models is rapid,and edge AI will be the next frontier.Cloud-edge collaborative

2、AI computing power deployment to address diversified challenges.The efficiency issue on the edge side is prominent,and agiledevelopment is key point.Refer:https:/arxiv.org/pdf/1907.08349.pdfOpenYurt and DragonflyOpenYurt,as the industrys first non-intrusive open-source project for edge computing clo

3、ud-native platforms,can address challenges in various aspects such as edge autonomy,edge networking,and edge storage.At the same time,in terms of model distribution,Dragonfly currently supports accelerated distribution of models for various applications.The collaboration between OpenYurt and Dragonf

4、ly can provide efficient and lightweight deployment of AI applications.ChallengesDistribute LLM to edge nodes in multiple regions,The challenges are faced by OpenYurt and Dragonfly as following:OpenYurtdistribute workloads to multiple regions with different configurations.how to expose LLM service i

5、n multiple regions?how to reduce control-plane bandwidth in large scale LLM application management?Dragonflyhow to load LLM images in multiple regions from cloud efficiently?how to integrate deployment of OpenYurt and Dragonfly?Whats OpenYurt?OpenYurt is the industrys first edge computing platform t

6、hat requires no modifications to the Kubernetes system.Through the control-plane located in the cloud,it centrally manages massive edge resources(such as CDN sites)across various locations.OpenYurt helps users to easily complete large-scale application deployment,operation,maintenance.Key FeaturesKe