微调代理的开源模型.pdf

1、2024 Databricks Inc.All rights reservedFINE-TUNING OPEN SOURCE MODELS FOR AGENTSTristan Zajonc-Continual.AIJune 11,20241Were entering the era of AI agentsConversational agentsGithub Copilot:https:/ agentsHex Magic:https:/hex.tech/product/magic-ai/Event-driven agentsCleric:https:/cleric.ioAutonomous

2、agentsGithub Copilot Workspace:https:/ do we build these types of agents?A G E N T S A R E S Y S T E M S N O T M O D E L SExternal ServicesAPIsData SourcesAgent SystemOrchestrationModelsToolsKnowledge BasesEvaluation,Monitoring,Feedback,and Fine-TuningIn-context agentsConversational agentsEvent-driv

3、en agentsAgent ExperienceAutonomous agentsLets talk about modelsOpen source models are laggingAgentBench:https:/llmbench.ai/agentSWE-Bench(Lite):https:/ UseReasoningKnowledgeTool UseReasoningKnowledgeIs there hope?TinyAgent:https:/bair.berkeley.edu/blog/2024/05/29/tiny-agent/How do we fine-tune a mo

4、del for agents?TinyAgent/LLMCompilerChoose how you want to call functionsOpenAIGenerate synthetic data using self-instruct or agent gymAgentGymOpenAI CookbookDont forgot to cover the full scope of user behaviorAgent-FLAN:https:/arxiv.org/abs/2403.12881Choose chat template suitable for LLM trainingRe

5、Act chat templateAgent-FLAN chat templateAgent-FLAN:https:/arxiv.org/abs/2403.12881Fine-tune your base modelAgentTuning:https:/thudm.github.io/AgentTuning/Consider embedding within a multi-agent systemMixture of Agents:https:/arxiv.org/abs/2406.04692Should you fine-tune open source models for AI age

6、nts?C O NProprietary models are still significantly ahead of open source models for agent use cases.Collecting agent trajectories for complex tasks like coding is non-trivial.Fine-tuning can easily degrade general performance and become a g