HC2022.THU.WenqiJi.v02.pdf

《HC2022.THU.WenqiJi.v02.pdf》由会员分享，可在线阅读，更多相关《HC2022.THU.WenqiJi.v02.pdf（13页珍藏版）》请在三个皮匠报告上搜索。

1、1Vision Perception Unit:Next-Generation Smart CMOS Image SensorWenqi Ji,Yuxing Han,Jiangtao Wen,Yubin Hu,FutangWang,Yuze He,Xi Li and Jun ZhangDepartment of Computer Science and Technology,Tsinghua UniversityAbstractAs we reach the end of Moores Law and Dennard Scaling,it has become highly desirable

2、 to design a highly integrated and optimized pipeline specifically for computer vision.A new generation of integrated“smart”visual processors that streamline an end-to-end optimized visual information acquisition and processing pipeline(VIAPP)becomes necessary to lower the cost,power consumption,and

3、 latency.We describe a new paradigm for VIAPP as Vision Perception Unit(VPU),wherein electric signals generated by photons are amplified before converting to the digital signals to emulate an initial layer of a convolutional neural network(CNN).The outputs from these layers are then converted to dig

4、ital signals and processed by following layers of a deep CNN.2Abstract3Conventional CMOS Image Sensor4 An image signal processor(ISP)for color processing,denoising,correction,etc.Further pre-processing in digital signal processor(DSP)e.g.,image enhancement and compression All capture image frames ar

5、e processed with original high resolutionVPU:Next-Generation CIS5 Sensing and processing are integrated DSP takes raw images as input No delay and power consumption from ISP In-pixel filters driven by DSPVPU:Process and Domain Specific Architecture6Pixels for Dynamic Architecturea.Dynamic Resolution

6、Tb.ROITPartially read-out from CMOS Image SensorDynamically lower frequency of ADCs&Clocks,reduce the bandwidth of data transmissionVision TasksVideo SegmentationEdge ExtractionFeedbackc.Dynamicread-out precisionXY2Bit4Bit8Bit8BitVPU:Sensing-Processing-Integrated Hardware7Edge Extraction and Video S