联邦数据分析平台.pdf

1、Federated Data Analytics PlatformFrom Data Pipelines to a Data PlatformRohit MathewsNeo NiLet The Data DecideDelta Live TablesJobs ServiceSQL GatewayServerlessFile ScanNotebook ServiceBI ToolsBillingFeature StoreReposDatabricks LightClassicPhotonDeltaUnity CatalogMaterialized Viewsretry_policymonito

2、ring_enabledgit_integrationnotebook_versionbudget_alertstableau_connectorlineage_trackingauto_recoverywarehouse_sizesql_access_controlfeature_registrysparse_checkoutcredential_typecost_optimizationlegacy_aclsingle_tenantcode_gendelta_optimizationvectorized_engineauto_scalingschema_inferencepartition

3、_discoverydata_skippingtime_travellineagerefresh_schedulingview_cachingDelta Live TablesScheduled JobDashboardsServerless 30 GBNotebook ServiceBI ToolsBillingFeature StoreReposDatabricks LightClassicPhotonDeltaUnity CatalogMaterialized ViewsSales CompensationFinance ReportsProduct KPIRevenue Forecas

4、tingAbuse DetectionDollarsDaily Active UsersWeekly Active UsersNum New WorkspacesDBUsA GOLD TABLEIt wasnt always this rosyInsights for a given day generated 48 hours after end of the daySlowest job went from 5 hours to 15 hoursNo surprises here given the extremely long pipeline run timeCross referen

5、cing log files from 100s of micro-servicesExtremely Slow50%UptimeMetric Accuracy8It wasnt always this rosyInsights for a given day generated 48 hours after end of the daySlowest job went from 5 hours to 15 hoursNo surprises here given the extremely long pipeline run timeCross referencing log files f

6、rom 100s of micro-servicesExtremely Slow50%UptimeMetric Accuracy9It wasnt always this rosyInsights for a given day generated 48 hours after end of the daySlowest job went from 5 hours to 15 hoursNo surprises here given the extremely long pipeline run timeCross referencing log files from 100s of micr