幕后：智能工作负载管理.pdf

1、Intelligent Workload Management in Databricks SQLUnder the H2023 Databricks Inc.All rights reservedConfidential and ProprietaryOverview Background Workload Management in Databricks SQL Load based workload management Query Costing Whats next2023 Databricks Inc.All rights reservedConfidential and Prop

2、rietaryBackground32023 Databricks Inc.All rights reservedConfidential and ProprietaryWhat is Workload Management Workload Management-efficient compute utilization in Databricks SQLWhen and where to run a queryWhen to Scale up or down2023 Databricks Inc.All rights reservedConfidential and Proprietary

3、Databricks SQL Logical Architecture52023 Databricks Inc.All rights reservedConfidential and ProprietaryWhen and where to run a query Whether to run the query or to put it in a queue Which compute resource to run the query on2023 Databricks Inc.All rights reservedConfidential and ProprietaryWhen to S

4、cale up/down Upscale when we see queueing we see high utilization Downscale when we see idle compute we see low utilization2023 Databricks Inc.All rights reservedConfidential and ProprietaryHow to do this rightOptimize For Latency?Keep the latency same even if we increase the cost Throughput?Process

5、 as many queries as possible Cost?Use as few resources as possible2023 Databricks Inc.All rights reservedConfidential and ProprietaryHow to do this rightPrinciples Latency is important for short queries Throughput is important for longer queries Both of the above should be optimized against cost2023

6、 Databricks Inc.All rights reservedConfidential and ProprietaryDatabricks SQL:Workload Management102023 Databricks Inc.All rights reservedConfidential and ProprietaryWorkload Management TodayQuery Concurrency based Allows a static concurrency Autoscaling based on query throughput,rate of incoming qu