构建弹性数据团队和可扩展解决方案：来自现场的经验教训.pdf

1、Hosted by:Welcome!The presentation will begin shortly If your badge was not scanned at the door,please use the self check-in feature in the session details in the Schedule.Building Resilient Data Teams And Scalable SolutionsLessons From The Field Of Data And Analytics EngineeringGreat Lakes Data,AI&

2、Analytics SummitApr 10,2025PresentersBrittany SpencerDirector,Analytics EngineerAAA LifeHarini RajagopalManager,Data EngineeringAAA LifeAgendaEngineering is done when Data is UseableGathering Business Requirements into Technical SpecsScalable Solutions as Opposed to One and DoneCoordination for Prod

3、 Support on the Data PipelineSelf-Serve Analytics&Single Unified Source of TruthInvestment in Data Tech InfrastructureData and Analytics Engineering Not all Data Teams are the SameTruly empowered data driven organizations require support across every stage of the data pipelineEngineering is done whe

4、n Data is UseableDelivering raw data isnt the finish line of a projectTimelines should reflect the work it takes to make it consumableRobust project plans include dependencies for data deliveryBusiness Requirements!=Technical SpecsData is a natural bridge builder across business units Bridging the g

5、ap between technical specs for implementation and business requirements is much harderFocus on simple questions to view the issue from a stakeholder perspective:What questions do you need to be able to answer at the end of this project?How are you measuring the success of this launch?How people need

6、 to see the data is just as important as how to get it to themScalable Solutions vs Build for Distant FutureDesignFor smaller incrementsBuild modular pipelines with reusable componentsTerraform,Airflow,dbt and AWSStart somewhereFocus onOn Automation and MonitoringCI/CDFalse Positives monitoringPrior