how-to-increase-the-throughput-of-kubernetes-scheduler-by-tens-of-times-kuberneteshoy-jzha-hao-pan-yuquan-ren-bing-li-bytedance.pdf

1、How to Increase the Throughput of Kubernetes Scheduler by Tens of TimesYuquan Ren,Bing LiByteDanceIntroductionContent-Background-Scheduling performance optimizations-Future workBackground-Kubernetes ScalabilityKubernetes Scalability Thresholds#Nodes5000#Pods 150000No more than 100 Pods/s at ultra-la

2、rge-scale cluster-Cannot support offline(stream and batch)workloads whose QPS usually exceed 1K+Pods/s-Cannot further improve resource utilization by colocation in large-scale clusterBackground-Kubernetes SchedulerArchitecture:monolithicDetailed implementation:-queue-cache-snapshot(data syncing)-sch

3、eduling-preemptionGdel Scheduling SystemHow to improve scheduling throughput?Gdela unified scheduling system for bothonline(micro service)and offline(ml,stream,batch)workloadsGdel Scheduling System-ArchitectureGdel Scheduling System-OptimizationsOptimize single-shard performance to the utmostthrough

4、data structures&algorithmsGdel Optimizations-Data SynchronizationGdel Optimizations-Data SynchronizationAbstract the underlying storage GenerationStorethat supports incremental updatesMigrate all existing storage to GenerationStoreGdel Optimizations-Data SynchronizationGdel Optimizations-Data Synchr

5、onizationGdel Optimizations-Data SynchronizationUltra-large-scale clusterwith 20K+Nodes and 1M+Pods,while 1K+Incoming Pods/sE2E latency distribution shiftfrom minutes to millisecondsGdel Optimizations-SchedulingHigh Water LevelGdel Optimizations-SchedulingAlgorithm Latency of almost all Pods isreduc

6、ed by more than 50%Reducing evaluated nodes while NOTsignificantly decrease feasible nodesGdel Optimizations-SchedulingUnschedulableGdel Optimizations-SchedulingMany nodes remained unchangedduring multiple scheduling attemptsFilter durationfrom 27ms to 7msGdel