4-4 基于环境虚拟化的强化学习应用实践.pdf

1、基于环境虚拟化的强化学习应用实践基于环境虚拟化的强化学习应用实践俞扬南京大学/南栖仙策奖励行动观测强化学习通过与环境反复交互试错，找到最优策略强化学习是机器学习中关于如何学习决策的分支人工智能机器学习监督学习人脸识别，图像识别，统计预测强化学习AI围棋，AI游戏无监督学习数据降维，数据压缩，数据可视化Reinforcement Learning:About the intelligence of actionsAbout Reinforcement LearningJ()=Zxp(x)loss(x)dxSupervised learning objectiveJ()=ZTrap()R()dp(

2、)=p(s0)TYi=1p(si|ai,si?1)(ai|si?1)Reinforcement learning objectiveAgentEnvironmentaction/decisionrewardstateWhy SL has wide applicationsSL is much more data-drivenLess artificial,more applications“the actual contents of minds are tremendously,irredeemably complex;we should stop trying to find simple

3、 ways to think about the contents of minds We want AI agents that can discover like we can,not which contain what we have discovered.Building in our discoveries only makes it harder to see how the discovering process can be done.”Human-level Records of RL1992TD-Gammon2016AlphaGoDeep Q-Network2014Alp

4、haZero20182019AlphaStarMuZero20202020Agent57Industrial problem exampleHybrid Mode ControlData from a bad policyGlobal constraintDemands in industrial applications1.Trial-and-success3.Fully offline evaluation No errors Adaptive Performance expectation Confidence for going online4.Other challenges Cha

5、nging reward functions Mostly have no knowledge about RL for their decision-making tasks2.Very few data Decision data is always smallJ Degrave,et al.Magnetic control of tokamak plasmas through deep reinforcement learning,Nature 602:414419,2022.Recent application by DeepMindRecent application by Deep

6、Mind“We use a simulator that has enough physical fidelity to describe the evolution of plasma shape and current,while remaining sufficiently computationally cheap for learning”“This achievement required overcoming gaps in capability and infrastructure through scientific and engineering advances:1.an