GenAI 在客户服务中的可观察性.pdf

《GenAI 在客户服务中的可观察性.pdf》由会员分享，可在线阅读，更多相关《GenAI 在客户服务中的可观察性.pdf（32页珍藏版）》请在三个皮匠报告上搜索。

1、GenAI Observability in Customer Care EarnInWillem DHaeseleerMatteo CiccozziAgenda3Customer Support At EarnIn5 minutesChatbot Journey5 minutesMetrics5 minutesPlatform10 minutesLooking Forward2 minutesHallucinations8 minutesEarnIn4Make Any Day PayDayMake Any Day PayDayNo interest,no credit checkOver 1

2、9 million downloads and countingCustomer Care5At EarnInTaxonomy of 100+customer support issues24/7 real time support across USHighly regulated by CFPBCustomer Care6At EarnInTaxonomy of 100+customer support issues24/7 real time support across USHighly regulated by CFPB7March 2024FAQ RAGJune 2024Tool

3、callingAugust 2024Datadog-DatabricksOctober 2024Ai/Bi Dashboard February 2025 Agentic CapabilitiesApril 2025Multimodal2025+ReasoningChatbot JourneyAt EarnInMetricsImmediate feedback during experimentationMetricsHuman reviewEvolving definition of metricsHigh cardinality metrics8Closing the feedback l

4、oopMetricsSingle use case evaluationDid we help the customer?Did we contain the conversation?Intent precision Multi use case evaluationMeasured over all chat trafficBottom Line/North starIntent recallContainmentCoverage9Measuring in productionMetrics10Containment/CoveragePlatform12PlatformArchitectu

5、re13PlatformArchitecture14PlatformArchitecture15PlatformArchitecture16PlatformArchitecturePlatform17Publishing Closeup*https:/microservices.io/patterns/data/transactional-outbox.html*Platform18Materialization CloseupPlatform19Materialization Self-ServePlatform20Materialization Self-ServePlatformClou

6、deventsTransactional OutboxDefine contract for eventsPublish events to kafkaDelta Live TablesAccess control w/UnityNear real time optionSelf-Serve Materialization21Developer ExperienceProtobuf or JSON SchemaSchema Registry&Lifecycle Integrated data environmentNotebook explorationTrack experimentsInc