JC_Belfiore_New_AI_Theory_Workshop_watermark.pdf

《JC_Belfiore_New_AI_Theory_Workshop_watermark.pdf》由会员分享，可在线阅读，更多相关《JC_Belfiore_New_AI_Theory_Workshop_watermark.pdf（48页珍藏版）》请在三个皮匠报告上搜索。

1、SecurityLevel:Semantic Informationand the difficulty of Learning:Paving the Future of AIAdvanced Wireless Technology Lab.Huawei Paris Research CentreAuthor:J.-C.Belfiore with D.Bennequin,X.Giraud,M.Hamad and A.SagnierPlace and Date:New AI Theory Workshop-Online-Nov.24th,2023HUAWEI TECHNOLOGIES CO.,L

2、TD.Part ISemantic spaces-Learning by conceptsOutlineLearning,imagining and inventing like humansHUAWEI TECHNOLOGIESHuawei proprietaryNo spread without permissionPage 2/33Learning-Imagining-InventingUp to now,neural networks have been unable to learn like humans,to invent,to imagineWe propose a new m

3、ethodology to design neural networks that1.Can learn like humans(by concepts).2.Can imagine(by internal activities and memories implemented in the neural network itself.3.Can invent(thanks to reasoning and generalization).This methodology will strongly rely on Grothendieck toposes.HUAWEI TECHNOLOGIE

4、SHuawei proprietaryNo spread without permissionPage 3/33Statistical learning vs Concept learningStatistical Learning:pure data vs human-like way:invariant-based.Figure:Pure data-based learningFigure:Learning through invariantsGeneralization,invariants,semantics are strongly related notions.Action of

5、 a structure(inductive bias)may help.HUAWEI TECHNOLOGIESHuawei proprietaryNo spread without permissionPage 4/33The Shepard et al.experimentLearning of a conceptTest Result for Simple MLP Architecture(5 hidden layers:3 non-equivariant vsequivariant+2 FCL)Parameters:Training dataset:Only 9 representat

6、ives(1 for each Type+3 for the generators)out of 70 number of epochs:10,batch size:256,number of batches perepoch:64,learning rate:0.0015,Non-linear activation function:ReLU,Bias=TrueHUAWEI TECHNOLOGIESHuawei proprietaryNo spread without permissionPage 5/33Part IIWhat is semantic information?Outline