张希_RLChina talk1126_watermark.pdf

1、融合大语言模型的智能体学习与决策融合大语言模型的智能体学习与决策 构建强化学习世界模型构建强化学习世界模型张张 希希 Xi Sheryl Zhang Nov.26,20232Why Reinforcement Learning?Active Learning vs Passive Learning Paradigms The main distinction with supervised learning Active learner interacts with the environment at training time,say,by posing queries or perfor

2、ming experiments Passive learner only observes the information provided by the environment(or the teacher)without influencing or directing it 3RL Nomenclature*Partially Observed MDP(POMDP)model is usually advocated when the agent has no access to the exact system state but only an observationof the

3、state.known/unknownstationary/non-stationarySergey Levin and Chelsea Finn,Deep Reinforcement Learning,Decision Making,and Control,ICML 2017 Tutorial4RL Algorithm:Policy Evaluation&ImprovementMDP ControlGeneralized Policy Iteration(GPI)Richard S.Sutton and Andrew G.Barto,Reinforcement Learning:An Int

4、roduction,Second Edition,The MIT Press 2018FQ2:How can we learn a good policy?FQ1:How good is a specific policy?5Computational RL Anatomy Deep RLDeep Neural Networks wSergey Levin and Chelsea Finn,Deep Reinforcement Learning,Decision Making,and Control,ICML 2017 Tutorial6DQNAlphaGoMuZeroAlphaStarAta

5、riGoChessStarCraftSuccesses obtained via DRLBeyond the Imitation Game:Quantifying and extrapolating the capabilities of language models,2023Checkmate-in-one taskLarger models are better at finding legal chess moves,but struggle to find checkmating moves.How about using LM?None of the BIG-G models te

6、sted can solve checkmate-in-one taskNot so goodNatural Natural Language inLanguage inRLRL7How might intelligent agents ground language understanding in their own embodied perception?Natural language inNatural language in RLRL89Natural language inNatural language in RLRLtask-agnostic controlRelabeled