【陆家靖】SkyWalking BanyanDB：时序数据库的查询引擎和流式计算.pdf

1、S Sk ky yWWa al lk ki in ng g B Ba an ny ya an nD DB B时序数据库的查询引擎和流式计算陆陆家家靖靖收钱吧框架工具负责人复旦大学核物理博士收钱吧框架工具团队负责人从事可观测性平台、API网关和服务治理平台研发APACHE SKYWALKING PMC MEMBER陆陆家家靖靖0 01 1可可观观测测性性与与时时序序数数据据库库可可观观测测性性三三大大支支柱柱指标、链路、日志*The three pillar of the Observability.Image source:Metrics,tracing,and logging,P.Bourg

2、on.https:/peter.bourgon.org/blog/2017/02/21/metrics-tracing-and-logging.htmlLowvolumeHighvolumeRequest-scoped eventsRequest-scoped metricsT Tr ra ac ci in ng gRequestscopedMMe et tr ri ic cs sAggregatableL Lo og gg gi in ng gEventsRequest-scoped,aggregatable eventsAggregatable eventse.g.rollupsTraci

3、ng&LoggingWorkflow-centric,distributedCausalMetricsStatistic/Aggregatable(rollups)Temporal:fixed interval,compression 时时序序数数据据的的数数据据结结构构Tag&Fields*How ClickHouse inspired us to build a high performance time series database.Aliaksandr Valialkin(valyala).https:/ IndexSeriesID(UInt64)*Frame of Referenc

4、e and Roaring Bitmaps,A.Grand.https:/www.elastic.co/blog/frame-of-reference-and-roaring-bitmapshash(all Tags):InfluxDB,VictoriaMetrics,etc.hash(partial TagValues):BanyanDB时时序序数数据据的的数数据据结结构构高基数问题H Ho oww Q Qu ui ic ck kl ly y D Do oe es s C Ca ar rd di in na al li it ty y G Gr ro oww?*What is High Ca

5、rdinality,R.Skillington.https:/chronosphere.io/learn/what-is-high-cardinality/时时序序数数据据的的数数据据结结构构高基数问题1 https:/ https:/ https:/ https:/ https:/ via Prometheus Recording Rules1TimescaleDB:Tiered B-Tree,Chunks2VictoriaMetrics/VictoriaLogs:MergeSet3,High-cardinality TSDB benchmarks4InfluxDB IOx:columnar

6、 built on Apache Arrow and Parquet 5BanyanDB:(tailored for SkyWalking)Partial tags for seriesIDCompact seriesID(xxhash)时时序序数数据据的的数数据据结结构构读写模式Vertical writeHorizontal readInsertions lookupsOld data is less likely to be 时时序序数数据据的的存存储储RUM Conjecture“We cannot design an access method for a storage syste