1-1 Flink Table Store：流批一体存储.pdf

1、Flink Table StoreFlink Table Store流批一体存储Jingsong LeeAgendaAgenda 业务需求 使用 Flink Table Store 理解 Flink Table Store Project and Roadmap A stream is the changelog of a dynamic table As change messages are ingested from a stream,a table evolves As a table evolves,change messages are emitted to a stream A

2、batch is the snapshot of a dynamic table A batch write,or a transaction commit,generates a snapshot A batch query,a snapshot of the current timeDynamic Table in Streaming WarehouseDynamic Table in Streaming WarehouseStorage for Streaming and BatchStorage for Streaming and BatchWe need:A unified stor

3、age,A unified abstraction！需求 流批统一存储，打造流式数仓 流写，流读 OLAP 可查询 毫秒级实时流读 批写，INSERT OVERWRITE 最完善的 Flink Connector 支持 Flink SQL 所有概念 支持任意 Flink Job 的输出 支持所有类型 最好用的 Flink Connector Managed，结合 Flink SQL 提供 DB 级体验 干净，简单，少依赖 支持大规模更新 业务需求 使用Flink Table StoreFlink Table Store 理解 Flink Table Store Project and Road

4、mapAgendaAgenda-Set root path for tableSET table-store.file.path=/tmp/store;-Set default bucket number for tableSET table-store.bucket=4;-Optional,set kafkaSET table-store.log.system=kafka;SET table-store.log.kafka.bootstrap.servers=.;CREATE TABLE clicks(userVARCHAR,url VARCHAR,cTime TIMESTAMP(3),PR

5、IMARY KEY(rowkey)NOT ENFORCED);DROP TABLE clicks;-Filtering queries based on primary keys is also FastSELECT price+tax FROM Orders WHERE id=10;Create Physical StoreCreate Physical StoreDrop Physical StoreDrop Physical StoreManaged:Act like Table in DatabaseManaged:Act like Table in DatabaseA real ta

6、ble and not just a mappingCREATE TABLE clicks(userVARCHAR,url VARCHAR,cTime TIMESTAMP(3);-Batch QuerySET execution.runtime-mode=batch;SELECT*FROM clicks;-Streaming QuerySET execution.runtime-mode=streaming;SELECT*FROM clicks;Unify Streaming and Batch:Based on execution runtime mode Supports all chan