WGU 的 MLOps 平台：使用 Databricks 实现生产 ML 的解决方案.pdf

1、2024 Databricks Inc.All rights reserved1MLOpsMLOps at WGU:at WGU:Solutions to Solutions to Production ML with Production ML with DatabricksDatabricksZach Clement,Jonathan BownZach Clement,Jonathan BownDate:TBDDate:TBDI WGU2024 Databricks Inc.All rights reserved2OVERVIEWOVERVIEWChallengesProject Goal

2、s and FeaturesArchitecture CI/CDUser and Developer ComponentsDemoDiscussion2024 Databricks Inc.All rights reserved3KEY CHALLENGESKEY CHALLENGESGovernanceAutomationTraceabilityRepeatabilityMonitoringModel POCSDeployment to ProdLack OfLack OfA Gap BetweenA Gap Between2024 Databricks Inc.All rights res

3、erved4GOALS AND FEATURESGOALS AND FEATURESGoalsGoalsFeaturesFeaturesSelf-governed data science environmentAutomated Project level resources and permissions accessVersion ControlEverything as code(ETL,workflows,compute,permissions)AuditableLineage tracking of data,workflows,experiments,models,code,pe

4、rmissionsSimplify productionalization using repeatable and standardized processesAutomation via CI/CD-Dev/stage/prod environmentsOrchestration of pipelinesMaintain model performance.Monitor batch inference modelsCompare candidate models to current modelsData validation and profiling toolsBalance MLO

5、ps needs with Data Scientists skillsAccommodate notebooks,widgets in workflowsMake it as usable as possible2024 Databricks Inc.All rights reserved The MARVIN platform is built to put models into productionoCreating and maintaining project infrastructureoProviding tools for data scientists to streaml

6、ine their developmentoMonitoring for workflow failures and communicating to stakeholdersoIntegrating Databricks featuresoCompatibility with wide variety of model types and frameworksoModularity to allow rapid integration of new or changing feature requirements5MARVIN MARVIN ML AND DATA OPS PLATFORMM