HMS Federation 如何助力 Nationwide 实现无缝高效的 Unity 目录迁移.pdf

1、How HMS Federation Powered Nationwides Seamless and Efficient Unity Catalog MigrationNationwidePraveena Edward Panneer Selvam2025Migration Purpose01Opportunities02Roadblocks&Rethinks03Solution HMS Federation04UC Migration Design05Agenda Purpose WHYWHATHOWHarness New CapabilitiesUnified,Secure,Simpli

2、fied&Scalable SystemFail Fast&ForwardPurpose5OpportunitiesStandardize Table NamesAWS S3 Path ChangesMore MetadataRoadblocks&Rethinks SyncOnly Creates External TableShallow CloneFailed Due to S3 directory path changesDeep CloneFailed Due to Table Name&schema changesUnity CatalogThe Challenges Solutio

3、ns HMS FedSchemasSchemasSchemasHMS Federation CatalogADGroupsAD GroupsSolution:HMS FederationOur CatalogUnity CatalogHive MetastoreEnable Hiveto HMS FederationCreate Views in UC pointing to HMS using HMS fedCustomer OnboardingBuild New TablesCustomer NotificationRelease and SwitchHMS Clean upPhase 1

4、:Federation+Customer OnboardingPhase 2:Data Migration+Customer MigrationMigration DesignPhase 1-HMS Federation&Customer Onboarding Enable HMS FederationHMS Federation Identify tables Establish new naming standards Create views using the new namesCreate Views in UC Create AD group Provision access in

5、 both Catalogs in UCCustomer OnboardingPhase 2 Data Migration&Customer Migration Build tables to replace viewsBuild New Tables Notify about the field structure changesCustomer Notification Promote the code Schedule the jobsRelease and Switch Replay all historical data Decommission HMS jobsHMS Clean

6、up1324Leveraging UC features before fully migratedBalancing short-term needs with Long-Term GoalsDatabricks collaborationRunning jobs in a single locationBehind the Build:Lessons from our JourneyKnow your purpose.Embrace your challenges.Lov