利用 Unity Catalog 提供的数据智能大规模监控质量和合规性.pdf

1、Monitor Quality+Compliance at ScaleJacqueline Li and Danny ChiaoEnrich Unity Catalogs Data Intelligence1Common challengesWhere teams startProduct announcementsDemoRoadmapAgendaHas anyone experienced a problem with data quality or maintaining compliance?Question:How do you ensure quality and complian

2、ce across all your data,at scale?Challenge:The average enterprise manages 15,000 tables in Unity Catalog.How do you ensure quality and compliance across all your data,at scale?Challenge:The average number of tables grows 4x every year.How do you ensure quality and compliance across all your data,at

3、scale?Challenge:Why is quality and compliance is needed7Enable business decisions to be made with compliant and reliable dataData PlatformCatalog 1Catalog 2Catalog NSchema 1Schema 2Schema N1Schema N2Schema 1Table 1Table 2Table 3Table N1Table N2Notebook 1Revenue reportNotebook NDashboard 1Dashboard N

4、!CEOWhy is quality and compliance is needed8Enable business decisions to be made with compliant and reliable dataData PlatformCatalog 1Catalog 2Catalog NSchema 1Schema 2Schema N1Schema N2Schema 1Table1Table 2Table 3Table N1Table N2Notebook 1Revenue reportNotebook NDashboard 1Dashboard NCEOProblem9En

5、sure quality and compliance across all your tablesRetail company Product dataCustomer dataTransactionsProblem10Ensure quality and compliance across all your tablesWhere teams startTables with sensitive data are segregated11What works for smaller environments cant scale upWhere teams startTables with

6、 sensitive data are segregated12What works for smaller environments cant scale upQuality only understood by central team?Where teams startTables with sensitive data are segregated13What works for smaller environments cant scale upBespoke rules are written per tableQuality only understood by central