12-Sharing GPUs among multiple containers- 李孟轩.pdf

1、Is sharing GPU to multiple containers feasible?李孟轩目录Background01Scheduling attempts03Device Layer attempts02ContentSummary04Background:Part 01Device cant be fully utilizedA typical GPU utilization in production environmentTwo factors lead to low utilization of GPU devices in k8s clusters：GPU resourc

2、es can only be applied by container in an exclusive mannerIn order to match the trend of computing power growth,GPU manufacturers have released new GPUs rapidly,with more powerful computing power,and higher price.A typical GPU utilization in GPU task in kubernetes：Core utilization can be 0 for a lon

3、g period of timeIn order to match the trend of computing power growth,GPU manufacturers have released new GPUs rapidly,with more powerful computing power,and higher price.Issue#52757 点击左上角开始 新建幻灯片旁的下拉箭头，选择 Title and Content 添加 默认一级段落内容字号为 18Device layer Attempts:Part 02How to construct a GPU-resourc

4、e sandox?Nvidia TimeSliceapiVersion:v1kind:ConfigMapmetadata:name:time-slicing-config-alldata:any:|-version:v1flags:migStrategy:nonesharing:timeSlicing:renameByDefault:falsefailRequestsGreaterThanOne:falseresources:-name: Nvidia Time-slice is like put multiple containers directly into that GPU:No re

5、source controlNo OverheadNvidia MIGNvidia MIG splits a GPU into serveral MIG-instances:Resource Isolation guranteeLow OverheadOnly apply for ampere or later GPUs Device memory and compute-core are cut simultaneouslyHave to follow certain templateHard to configure dynamically in kubernetesNvidia MPSN

6、vidia MPS smashes tasks from multiple containers into a single context,which brings high-performace,but high-risks:Resource Isolation guranteeHigh performanceHigh risk for task failureHard to configure inside kubernetesversion:v1sharing:mps:renameByDefault:trueresources:-name: is a third-party resou