AnalyticDB-快数据时代的实时数据仓库技术内幕（15页）.pdf

1、AnalyticDB 快数据时代的实时数据仓库技术内幕 林亮 阿里云智能 研究员 Realtime Datawarehouse In the Fast Data Era 从 Big Data 到 Fast Data 41%41%?寻求买家?Fast Fast+OnlineOnline Full Data?Realtime Data?Cloud-Native?Realtime Computing?AnalyticDBAnalyticDB FastDataFastData的最佳代表的最佳代表?Big Data to Fast Data实时数仓的设计挑战 灵活 Arbitrarily Join Ar

2、bitrarily Filter 高并发 100K QPS 10K Clients 低延时 95%50ms 高可用 99.999%实时 Read Committed 10M Records/s Insert 准确 100%AnalyticDBAnalyticDB 755M755M+Active Users 5+PB 5+PB Max instance Design Challenges for Realtime DatawarehouseAgilityConcurrencyLow LatencyHigh AvailabilityRealtimeAccuracy阿里巴巴OLAP系统演进 Orac

3、le RAC 2008 Greenplum AnalyticDB 1.0 2012 HBase MySQL Sharding Hadoop AnalyticDB 3.0 2018 p High concurrency p Volume p High concurrency p High availability (leadernode)p Realtime Write p Consistency(offline/online)p Agility(Cube)p ACID p Consistency(offline/online)p Realtime Write p ACID Realtime C

4、onsistency Agility Accuracy Volume(PB)Agility Accuracy Volume(PB)High concurrency Low Latency Accuracy Volume(PB)High concurrency Agility Low Latency Accuracy High availability Volume(100PB)High concurrency Agility Low Latency High availability Accuracy Realtime RW p ACID High concurrencyHigh concur

5、rency：1000 QPS(Complex Query)VolumeVolume:10PB+Realtime WriteRealtime Write :10M Records/s 2009 2011 Evolution of Alibaba OLAP systemsAnalyticDB：阿里唯一经过大规模验证的分析类数据库 以下是生产环境的真实数据：?阿里巴巴集团某营销应用单DB表数超过20000张?某客户单DB数据量近3PB，单日分析查询次数超过1亿?阿里巴巴集团内某单个ADB集群超过2000台节点规模?云上某业务实时写入压力高达1000w TPS?菜鸟网络某数据业务极度复杂分析场景，查询

6、QPS 100+?支撑阿里集团双十一业务支撑阿里集团大部分OLAP业务阿里集团内部超过300+业务单日查询次数1亿+Alibabas only large-scale validated OLAP databaseAnalyticDB-PB级实时数仓 云原生云原生 实时按需极致弹性实时按需极致弹性 存储从GB至100PB 计算节点从3台到2000台 混合负载 完备的企业级特性完备的企业级特性 备份/恢复/回收站 审计/白名单/自建账号/VPC 跨AZ/跨Region(On-going)兼容兼容&超越超越 MySQL/PostgreSQL MySQL/Po