使用 DeGirum DeLight 云平台使机器学习模型实现边缘计算.pdf

1、Bring Your ML Models to the Edge with the DeGirumDeLight Cloud PlatformShashi ChilappagariCo-Founder and Chief ArchitectDeGirum Corp.Not all ML models are compatible with all hardware Runtime software can be hardwarespecific Different types of ML models need different softwareDeveloping AI Applicati

2、ons for the Edge 2023 DeGirum Corp.2SoftwareHardwareModelsEvaluating AI Hardware Accelerators is HardBuy hardwareInstall software toolchainPort/design hardware compatible ML modelDevelop application softwareBenchmark performanceRepeat for all hardware optionsResult:Separate application software&mode

3、ls for each hardware optionWasted time,money,and effort 2023 DeGirum Corp.3Problems with the Hardware Evaluation ProcessImpossible to judge a hardware option based on datasheetNo unified application software that makes evaluation easy 2023 DeGirum Corp.41.Cloud access to AI hardware2.Software that s

4、upports multiple hardware options3.Tools to choose/port/design ML modelsDeGirum Cloud Platform&Model ZooDeGirum Device FarmAccess through DeGirum Cloud ServerDirect Access through Peer-To-Peer VPNDeGirum Solution:DeLight Cloud PlatformHighlights 2023 DeGirum Corp.5ORCAEdgeTPUEdgeTPUEdgeTPUGPUGPUGPUC

5、PUCPUCPUOtherOtherOtherORCAORCA ORCA-NNX(DeGirum):Available now CPU(Intel,AMD,ARM):Available now Edge TPU(Google):Coming soon GPU(Nvidia):Coming soon(Jetson and Orin series)Others can be added on demandSupported Hardware Options6 2023 DeGirum Corp.DeGirum PySDK:Sophisticated Applications with Simple

6、 APIs7 2023 DeGirum Corp.DeGirum cloud farmAI server IP addressLocal hardwareimport degirum as dgzoo=dg.connect(ai_server,model_zoo,token)model=zoo.load_model(model_name)res=model(image)res.image_overlayCloud model zooLocal model zooSingle model fileEach line in the above code solves a different pro