Michael Yuan-用 Rust 实现 AI LLM 应用.pdf

1、使用 Rust 开发 LLM Agent主讲人：Michael Yuan演讲嘉宾介绍Michael Yuan CNCF WasmEdge 项目维护者 Second State 的创始人 他撰写过5本软件工程书籍，由 Addison-Wesley、Prentice-Hall 和 OReilly 出版 Michael 是一位长期的开源开发者和贡献者 他之前曾在许多行业会议上发表过演讲，包括 OpenSourceSummit、The Linux Foundation Member Summit 和 KubeCon此处添加标题Workloads for the agents Memories Conv

2、ersation histories Creating and searching embeddings Planning Managing a chain or a tree of prompts Parsing LLM responses and matching the next step Interacting with external systems Inference using other models Pre-processing Inference on Tensorflow/PyTorch Post-processingTech stack-How it started

3、Python Optimized C/C+/CUDA libraries for computational tasks But a lot of slow GIL Python code for the application and networking logic Complex dependencies Linux OS Containers or VMsMassive image sizes(over 1GB)and low concurrency for agentsChristopher Arthur Lattner(born 1978)is the co-founder of

4、LLVM,Clang compiler,MLIR compiler infrastructure and the Swift programming language.As of 2023,he is the co-founder and CEO of Modular AI,an artificial intelligence platform for developers.Tech stack how it is going High performance and lightweight Also works with C/C+/CUDA native libraries OpenCV,f

5、fmpeg,PyTorch,Tensorflow,OpenVINO Language agnostic Rust,C/C+,Go,Moonbit,Grain Python,JavaScript,TypeScript,PHP,Kotlin OS/hardware agnostic ARM,x86,Apple,GPU,TPU Secure sandbox for plugins and serverless functionsA lightweight,secure,high-performance and extensibleWebAssembly Runtimehttps:/ for AI i

6、nferneceTech stack-How it started Pre-processing in Rust Model inference in native C/C+Post-processing in Rusthttps:/ for LLM agentshttps:/work/Senior developers are very busy and very expensive.Yet,the development process cannot move forwa