LightCL：面向边缘设备的低内存占用紧凑型持续学习.pdf

1、30th Asia and South Pacific Design Automation ConferenceASP-DAC 2025Date:January 20-23,TokyoLightCL:Compact Continual Learning with Low Memory Footprint For Edge DeviceZeqing Wang,Fei Cheng*,Kangye Ji,Bohu Huang2025/1/21School of Computer Science and TechnologyXidian UniversityOutlineBackgroundChall

2、engeMethodExperimental ResultsConclusionBackgroundEdge devices are everywhereAI empowers edge devices more intelligence 1Continual Learning!smartphonedronerobotBackground1Edge DevicesDataPrivacy ConcernRequire InternetCloud platform Catastrophic Forgetting Model training on a new task tends to forge

3、t the knowledge learned from previous tasks Key idea is to make trade-off between learning plasticity and memory stability for gaining generalizabilityChallengeA Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning Wang et al.,TPAMI 20242 Limited Resource in Edge Device Limi

4、ted computational power and memory Training consumes more than inference Memory has become the primary bottleneck in AI applicationsChallenge2AI and Memory Wall GHOLAMI et al.,Micro 2024020406080100120140160180Memory Footprint(MB)FLOPs(1015)TrainingInference3x3x Motivation CL process has the potenti

5、al to leverage previous knowledge when training on new tasks Redundancy in training Method3EvaluateCompressMemorizegeneralizabilitylayers CL SettingMethod3Task 1Task tTask T1tTModel1(1)(t)(T)AccuracyTime Analysis of Generalizability Memory stability(MS)and Learning plasticity(LP)are two different ch

6、aracteristics of generalizability MS denotes the loss of previous knowledge LP denotes the adaptation to new knowledgeMethod3A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning Wang et al.,TPAMI 2024 Analysis of Generalizability During CL,lower and middle layers have stro