LLM 时代的金融知识图谱实践.pdf

1、演讲人：王俊吾道科技首席科学家 2023 ChatGPT的冲击和启示 LLM:自然语言处理新范式 LLM在金融垂直领域的落地 LLM辅助金融知识图谱构建的具体案例ChatGPT的冲击和启示 技术冲击ChatGPT发布时，我正在参加NeurlPS-2022“2022年10月，Jasper获得1.25亿美元A轮融资，估值达到15亿美元，而去年的收入达7500万美元，从诞生到成为独角兽，仅用了18个月。”Ref:The rise of the Jasper Empire启发从技术和业务两方面对LLM的思考和探索在NeurlPS会场切身感受到对AI社区的震动完全颠覆了以往Chatbot“人工智障”的体

2、验遥不可及通用人工智能(AGI)似乎有了可能性 商业启示从会场展台上了解到Jasper.ai业务模式成功直观认识到大语言模型(LLM)的商业潜力LLM:自然语言处理新范式Building Blocks:Transformers Attention Is All You Need,https:/arxiv.org/abs/1706.03762,Google,2017/06 BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding,https:/arxiv.org/abs/1810.04805,G

3、oogle,2018/10Transformer EncoderEncoderTransformer DecoderDecoderEncoderDecoderMask LMAutoregressive LM主流模式:Pre-Trained+FinetuneBERT(Mask LM)BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding,https:/arxiv.org/abs/1810.04805,Google,2018/10 Exploring the Limits of Transfer

4、 Learning with a Unified Text-to-Text Transformer,https:/arxiv.org/abs/1910.10683,Google,2019/10T5(Encoder-Decoder)参数:110M，340M参数:60M,220M,770M,3B,11BGPT代表的Autoregressive LM成为新范式GPT-1:Improving Language Understanding by Generative Pre-Training,https:/ Models are Unsupervised Multitask Learners,https

5、:/ Models are Few-Shot Learners,https:/arxiv.org/abs/2005.14165,OpenAI,2020/05GPT-4:https:/ Models for Dialog Applications,https:/arxiv.org/abs/2201.08239,Google,2022/01PaLM:Scaling Language Modeling with Pathways,https:/arxiv.org/abs/2204.02311,Google,2022/04OPT:Open Pre-trained Transformer Languag

6、e Models,https:/arxiv.org/abs/2205.01068,Meta,2022/05BLOOM:A 176B-Parameter Open-Access Multilingual Language Model,https:/arxiv.org/abs/2211.05100,BigScience,2022/11GPT-2 参数:1.5B 数据：40G本质就是本质就是AutoAuto-complete-complete 或者文字接龙或者文字接龙GPT-1参数:117M数据:5G(Ref:https:/jalammar.github.io/illustrated-gpt2/)G