HC2022.KAIST.DongseokIm.v03.pdf

1、1 of 25HOTCHIPS 2022DSPU:A 281.6mW Real-Time Deep Learning-Based Dense RGB-D Data Acquisition with Sensor Fusion and 3D Perception System-on-Chip DSPU:A 281.6mW Real-Time Deep Learning-Based Dense RGB-D Data Acquisition with Sensor Fusion and 3D Perception System-on-Chip Dongseok Im,Gwangtae Park,Zh

2、iyong Li,Junha Ryu,Sanghoon Kang,Donghyeon Han,Jinsu Lee,Wonhoon Park,Hankyul Kown,and Hoi-Jun YooSemiconductor System Lab.School of EE,KAIST2 of 25HOTCHIPS 2022DSPU:A 281.6mW Real-Time Deep Learning-Based Dense RGB-D Data Acquisition with Sensor Fusion and 3D Perception System-on-Chip 3D Data in Mo

3、bile Platforms RGB-D data More Accurate and Versatile Applications CNN recognizes only 2D pictures,but real world consists of 3D objects RGB-D(3D)data enables the exact 3D object recognitionsTimeAccuracy(mAP)CVPR20406560555045CVPR16CVPR17CVPR18CVPR17ICCV193D-based2D-basedFace RecognitionAR/VR3D Geom

4、etryHigh AccuracyCVPR21ICCV213 of 25HOTCHIPS 2022DSPU:A 281.6mW Real-Time Deep Learning-Based Dense RGB-D Data Acquisition with Sensor Fusion and 3D Perception System-on-Chip DSPU:End-to-end 3D Perception SoC A 281 mW and 31.9 fps 3D Object Recognition Processor For Low-Power RGB-D Data Acquisition

5、CNN-based MDE&Sensor Fusion SW/HW Architecture For Real-time 3D Perception(e.g.3D Bounding Box)Window-based Search&Point Feature Reuse SW/HW ArchitectureUMPU Core#1UMPU Core#2UMPU Core#4UMPU Core#3UMPU Core#0UMPUCore#6UMPU Core#7UMPU Core#5DMU Core#0DMU Core#1DMU Core#2DMU Core#3RISC-V CoreInterconn

6、ect NetworkUPPU Core#0UPPU Core#1Interconnect Network3.6 mm3.6 mm1)MDE:Monocular Depth EstimationAligned Dense RGB-D3D Bounding BoxRGB DataRaw Depth Datat1t2t0Final ResultMonocular Depth EstimationSensor Fusion 3D PerceptionLow-Power and Real-Time DSPU SoCRGB Cam.Low-power ToFt2t04 of 25HOTCHIPS 202