赞助方：Galileo Technologies Inc.通过可观察性驱动的评估驯服恶意人工智能代理.pdf

1、2025|12025|2Confidential|2025 Galileo Technologies,Inc.The AI Reliability PlaybookAtin SanyalCo-founder and CTOGalileoRogue Agents2025|3The ironyof Gen AI progressConfidenceLLM capabilitiesTime2025|4Weve gone from elementary RAG apps to multi agent systems.with almost no ability to evaluate and obse

2、rve.AgentAgentVectorstoreAPI callsDatabaseLLMsAgentDatabaseMulti agent systemsRAG Q&APromptLLMVector storeWhy?2025|5User query/inputMemory/Contextmanagement(stores/updates conversation state,previous retrievals)RetrieveCite(references)Final verifiedresponse to userMCP Connectors(standardized APIs)E.

3、g.web,vector DB,knowledge graphs,enterprise DB,WikipediaVerify(cross-check claims)RAG 2025RAG 2023-24User queryRetrieve(knowledge base)LLM answers(using retrieved data)Is RAG dead?Not really2025|6A system that can plan and take actionsbased on instructions and dataTaki ng a s t ep backWhat are agent

4、s?2025|7Reasoning/BrainHandles reasoning and planningMemoryRemember past interactions as context(for better decisioning)ToolsDo things in the world using traditional software paradigmsData retrievals(web search,db lookups)Taking an actionOrchestration(agentic networks-calling other agents etc)Compon

5、ents of an agentThe Fundamentals2025|8Undesirable behavior of an agentHow can I help you today?Do you have corporate cards with good interest rate?infoGreat,Ive started your home loan app.Give me your info,Ill get you started.um2025|9The compounding effect of small errorsUnpredictable interactions(t

6、ool&memory errors)Prompt sensitivity:one word makes a huge differenceRetrieval quality issues3 obstacles to reliable agents1 12 23 32025|10How can we tame rogue agents?2025|11Integrated Observability2025|12The 8-step playbook for agent reliability2025|13StepAccuracyTool Over l apRat eSt epLi mi t Co