4-4Flink-1.11 Hive集成与批流一体(3).pdf

《4-4Flink-1.11 Hive集成与批流一体(3).pdf》由会员分享，可在线阅读，更多相关《4-4Flink-1.11 Hive集成与批流一体(3).pdf（31页珍藏版）》请在三个皮匠报告上搜索。

1、0 1000 2000 3000 4000 5000 6000 7000 8000 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97 100 Flink vs Hive TPC-DS 10 TB FlinkHive on MR SQL Client TableEnvironment CatalogManagerCatalog GenericInMemoryCatalogHiveCatalog Hive Metastore HiveShim referen

2、ceinheritance Streaming Data StreamingFileWriter Finished Files Pending Files In-progress Files Finished Files Pending Files In-progress Files Bucket Finished Files Pending Files In-progress Files Finished Files Pending Files In-progress Files Bucket Subtask Subtask StreamingFileCommitter Commit Mes

3、sage PartitionCommitTrigger PartitionCommitPolicy Partitions ready to commit Metastore Success File ContinuousFileMonitoringFunction ContinuousFileReaderOperator Splits FileSystem HiveContinuousMonitoringFunction ContinuousFileReaderOperator Splits FileSystem Metastore Streaming Data LookupJoinRunner FileSystemLookupFunction Hive Load & Cache Join Key Batch Analytics Batch Analytics Online Streams