刘涛-Adlik 对深度学习模型推理优化的实践（GOTC上海会场）（27页）.pdf

1、AI、大数据与数字经济论坛专场刘涛 2021年07月10日本期议题：Adlik对深度学习模型推理优化的实践Background:Three Big Stages in Machine Learning PipelineRaw DataThere are still many technical difficulties(both deployment and inferernce)in AI application stage that Adlik can help to solve.Pre-ProcessingCollectionStoreFeature ExtractSampling Fe

2、ature SelectionModelEvaluateModelOptimizeModel TuringTraining SamplesBusiness RequirementInference OutputModel DeployBackground:InferenceGenerate commercial value,！Whats Adlik Adlik dlik,a toolkit for accelerating deep learning inference on specific hardware.Support several kinds of hardwares.Collab

3、orate with existing inference solutions with unified entrance.An open source project of LFAI and code hosted on GitHub.https:/ dlik,a toolkit for accelerating deep learning inference on specific hardware.Support several kinds of hardwares.Collaborate with existing inference solutions with unified en

4、trance.An open source project of LFAI and code hosted on GitHub.https:/ AdlikWhy Using AdlikEfficientDirectly using training framework to do inference will be inefficient.Meet performance requirements(latency、throughput).ConvenientConvenient to use in different deployment scenario and specific hardw

5、are.Easy for user to choose correct inference params to get ideal performance in specific hardware.PortableAdaptive for different hardwares.Uniform interface for model compiler and optimizer.Unified inference interface and model management.Adlik Architecture7Model Optimizer&Compiler：boost computing

6、efficiency,reduce power consumption and latencyModel TrainingGraph OptimizerModel CompilerAdlik Inference EngineQuantizationPruningbig model fp32small model int8Graph ConversionLayer FusionKubernetesDockerStructural CompressionImage-based Engine+ModelBinary-file EngineGPU ClusterStorage Cluster Mgt