黄世宇-OpenRL支持大模型训练的强化学习框架与大模型时代的PluginStore.pdf

1、OpenRL:A Unified Reinforcement Learning Framework黄世宇 第四范式演讲嘉宾黄世宇第四范式强化学习科学家，开源强化学习OpenRL Lab负责人本科与博士均毕业于清华大学计算机系，导师是朱军和陈挺教授，本科期间在CMU交换，导师为Deva Ramanan教授。主要研究方向为强化学习，多智能体强化学习，分布式强化学习。曾在ICLR、CVPR、AAAI、NeurIPS,Nature Machine Intelligence,ICML,AAMAS,Pattern Recognition等会议和期刊发表多篇学术论文。其领导开发的TiZero谷歌足球游戏智能

2、体曾在及第平台上取得排名第一的成绩。黄世宇也曾在腾讯AI Lab、华为诺亚、商汤、瑞莱智慧等工作。目 录CONTENTS1.强化学习背景2.OpenRL介绍3.OpenRL未来发展4.OpenPlugin介绍Introduction&MotivationPART 01What is Reinforcement Learning?Goal of RL:Artificial General Intelligence(AGI)Reinforcement learning in dog training.What else?Robotics Autonomous DrivingOpenAI 2019C

3、ARLA 2017What else?Industrial Design Quantitative TradingPrefixRL 2022FinRL 2020What else?Chat BotWhat else?Multi-agent RL Competitive RLTiZero 2023Honor of Kings Arena 2022Do RL in a Unified FrameworkVarious RL AlgorithmsVarious EnvironmentsMulti-agent&Self-playOffline RLOpenRL:An Open-Souce RL Fra

4、meworkPART 02Main Features of OpenRL Friendly to beginnerspip install openrlordocker pull openrllab/openrlMain Features of OpenRL Friendly to beginnersopenrl-mode train-env CartPole-v1Main Features of OpenRL Friendly to beginnersMain Features of OpenRL Friendly to beginnersDocumentation/中文文档Tutorial

5、Main Features of OpenRL Customizable capabilities for professionalsConfigure everything via YAMLUse yaml python train_ppo.py-config mpe_ppo.yamlUse yaml python train_ppo.py-config mpe_ppo.yaml python train_ppo.py-seed 1-lr 5e-4Main Features of OpenRL Customizable capabilities for professionalsTrack

6、your experiments via WandbMain Features of OpenRL Customizable capabilities for professionalsTrack your experiments via TensorboardCustomize Wandb Outputhttps:/ Wandb OutputMain Features of OpenRL Customizable capabilities for professionalsAbstract&Modularized DesignReward ModulePolicy ModuleValue M