2020年终大会-对话智能：14-5.pdf

1、2020年12月19日 刘知远 清 华 大 学 Knowledgeable Pretrained Language ModelsKnowledgeable Pretrained Language Models Deep Learning: Data-Driven NLP 4 Advances in Natural Language Processing. Science 2015. Language & Knowledge Knowledge enables people to understand language from superficial meanings to implicati

2、ve meanings 这个夏天就像烤箱一样 This summer is hot We need cooling devices We need to cure heatstroke Domain Human Knowledge World Commonsense Linguistics The summer is like an oven 5 Challenges of DL for NLU & NLP 6 we feel confident that more data and computation, in addition to recent advances in ML and d

3、eep learning, will lead to further substantial progress in NLP.However,thetrulydifficult problems of semantics, context, and knowledge will probably require new discoveries in linguistics and inference. Advances in Natural Language Processing. Science 2015. Pretrained Language Model as a Breakthrough in 2018 Impressive progress of deep learning on unsupervised text corpora 7 Sebastian Ruder http:/