大模型智能体能力对齐.pdf

1、Aligning Agents to Follow Human ValuesTao GuiFudan University2024年6月YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP20242Fudan NLP LabWhere Is the Attention?Zhao,Wayne Xin,et al.A survey of large language models.arXiv preprint arXiv:2303.18223(2023).YSSNLP2024YS

2、SNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP20243Fudan NLP LabWhere Is the Attention?Achiam,Josh,et al.Gpt-4 technical report.arXiv preprint arXiv:2303.08774(2023).Ganguli,Deep,et al.Predictability and surprise in large generative models.Proceedings of the 2022 ACM Conf

3、erence on Fairness,Accountability,and Transparency.2022.YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP20244Fudan NLP LabExciting and Perilous Journey toward AGISelf-preservationSpecific versus General Principles for Constitutional AI AnthropicThe exciting,peri

4、lous journey toward AGI Ilya SutskeverSelf-identityYSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP20245Fudan NLP Lab Responsible Scaling Policy(RSP)AnthropicNo meaningful catastrophic riskEarly signs of dangerous capabilitiesSubstantially increase the risk of c

5、atastrophic misuseNot yet defined as it is too far from present systemsYSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP20246Fudan NLP LabTraining language models to follow instructions with human feedbackWhat is Alignment?H He el lp pf fu ul lFollow instructions

6、Ask relevant follow-up questions and obtain necessary detailsRe-direct ill-informed requestsH Ho on ne es st tKnow who it is,and what can/cannot it do/knowH Ha ar rmml le es ss sRefuse inappropriate requestsYSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP2024YSSNLP20247Fu