AI-ISP：为图像信号处理添加实时 AI 功能同时减少内存占用和处理延迟.pdf

1、AI-ISP:Adding RealTime AI Functionality to Image Signal Processing with Reduced Memory Footprint and Processing LatencyMankit LoChief Architect,NPU IP DevelopmentVeriSilicon Inc.Image Signal Processor is responsible for processing the raw image data captured by a cameras sensor and turning it into a

2、 usable imageWhat Is an ISP(Image Signal Processor)?2 2023 VerisiliconISP Traditional Hardware Pipeline3 2023 VerisiliconDemosaicLensShadingCorrectionWhite BalanceDefectPixelCorrectionDenoiseColor CorrectGamma CorrectAF,AE,AWBStatisticColor Space ConvertPost ProcessBlack Level CorrectionThe Growing

3、Body of Research on AI for ISP4 2023 Verisilicon“Learning to See in the Dark”:low-light image enhancementThe Growing Body of Research on AI for ISP(cont 2/3)5 2023 Verisilicon“Deep Joint Demosaicing and Denoising”“Learning Deep Priors for Image Dehazing”The Growing Body of Research on AI for ISP(con

4、t 3/3)6 2023 Verisilicon“HDR image reconstruction from a single exposure using deep CNNs”“Merging-ISP:Multi-Exposure High Dynamic Range Image Signal Processing”6Noise Reduction with AI-ISP7 2023 VerisiliconLow Light Image Enhancement for Automotive with AI-ISP 8 2023 VerisiliconNot to Fall Behind9 2

5、023 VerisiliconHow to Implement an AI ISP?10 2023 VerisiliconNPUDemosaicLensShadingCorrectionWhite BalanceDefectPixelCorrectionDenoiseColor CorrectGamma CorrectAF,AE,AWBStatisticColor Space ConvertPost ProcessBlack Level Correction Programmable:Ever improving algorithms and changing scenarios demand

6、 a programmable solution.Computation:High resolution sensors and powerful algorithm require high computation powerReal-time AI-ISP Challenges11 2023 Verisilicon Memory usage:Conventional image signal processing techniques often require the whole image frame to be stored in memory before processing c