任旭滨LLMs Enhanced Rec.pdf

《任旭滨LLMs Enhanced Rec.pdf》由会员分享，可在线阅读，更多相关《任旭滨LLMs Enhanced Rec.pdf（22页珍藏版）》请在三个皮匠报告上搜索。

1、Enhancing Recommender Systems with Large Language ModelsXubin RenThe University of Hong Kong2LLMs+Graph in Recommendation1 Li,Yuhan,et al.A survey of graph meets large language model:Progress and future directions.arXiv preprint推荐算法推荐算法LLM as EnhancerGNN-LLMAlignment强化强化推荐系统3LLMRec:Large Language Mo

2、dels with Graph Augmentation for Recommendation1 Wei,Wei,et al.LLMRec:Large Language Models with Graph Augmentation for Recommendation.WSDM 2024.4BackgroundBackground原始数据原始数据用户用户-商品交互图商品交互图图神经网络图神经网络Bayesian Personalized Ranking(BPR)从交互图中采样训练数据从交互图中采样训练数据（正负样本对）（正负样本对）1 Wang,Xiang,et al.Neural graph

3、 collaborative filtering.SIGIR.2019.5存在的问题存在的问题用户用户-商品交互图商品交互图节点特征节点特征缺乏交互边交互边存在False positive基于基于大模型大模型进行进行训练数据增广训练数据增广引入引入多模态特征多模态特征基于基于大模型大模型进行进行节点节点特征增强特征增强图数据图数据增强增强 Graph Augmentation 1 Wei,Wei,et al.LLMRec:Large Language Models with Graph Augmentation for Recommendation.WSDM 2024.6基于大模型的节点特征增

4、强基于大模型的节点特征增强对用户画像的对用户画像的AugmentationPromptsGenerate user profile based on the history of user,that each movie with title,year,genre.History:332 Heart and Souls(1993),Comedy|Fantasy 364 Men with Brooms(2002),Comedy|Drama|Romance Please output the following infomation of user,output format:age:,gende

5、r:,liked genre:,disliked genre:,liked directors:,country:,language:LLMs Outputage:50,gender:female,liked genre:Comedy|Fantasy,Comedy|Drama|Romance,disliked genre:Thriller,Horror,liked directors:Ron Underwood,country:Canada,United States,language:English对商品属性的对商品属性的AugmentationPromptsProvide the inqu

6、ired information of the given movie.332 Heart and Souls(1993),Comedy|Fantasy The inquired information is:director,country,language.And please output them in form of:director,country,languageLLMs OutputRon Underwood,USA,English1 Wei,Wei,et al.LLMRec:Large Language Models with Graph Augmentation for R

7、ecommendation.WSDM 2024.7基于大模型的节点特征增强基于大模型的节点特征增强1 Wei,Wei,et al.LLMRec:Large Language Models with Graph Augmentation for Recommendation.WSDM 2024.增强文本增强文本Sentence-BERTwith MLP额外特征向量额外特征向量用户特征用户特征商品特征商品特征节点特征融合节点特征融合多模态特征融合多模态特征融合（包含大模型增强的特征向量）（包含大模型增强的特征向量）特征重构训练（特征重构训练（Masked Auto-encoding）对部分节点进行

8、掩膜重构对部分节点进行掩膜重构（增强对多模态特征的健壮性）（增强对多模态特征的健壮性）8基于大模型的训练数据增广基于大模型的训练数据增广PromptsRecommend user with movies based on user history that each movie with title,year,genre.History:332 Heart and Souls(1993),Comedy|Fantasy 364 Men with Brooms(2002),Comedy|Drama|Romance Candidate:121The Vampire Lovers(1970),Horr

9、or 155 Billabong Odyssey(2003),Documentary 248The Invisible Guest 2016,Crime,Drama,Mystery Output index of users favorite and dislike movie from candidate.Please just give the index in.LLMs Output248 121对对每一个用户每一个用户进行数据增广进行数据增广利用CF算法获取Candidate items基于大模型从中挑选正负样本正负样本扩增BPR训练样本对噪音过滤剪枝噪音过滤剪枝1 Wei,Wei,e

10、t al.LLMRec:Large Language Models with Graph Augmentation for Recommendation.WSDM 2024.9ExperimentsExperiments在在多模态数据集（多模态数据集（Netflix和和MovieLens）上进行测试，获得最优的性能上进行测试，获得最优的性能1 Wei,Wei,et al.LLMRec:Large Language Models with Graph Augmentation for Recommendation.WSDM 2024.10ExperimentsExperiments从消融实验得知

11、，从消融实验得知，图增强的范式对性能提升有很大帮助图增强的范式对性能提升有很大帮助去除训练数据增广去除图节点数据增强模块去除训练数据剪枝模块去除训练数据剪枝模块和MAE模块1 Wei,Wei,et al.LLMRec:Large Language Models with Graph Augmentation for Recommendation.WSDM 2024.11ExperimentsExperiments算法框架具有算法框架具有扩展性扩展性，并且能，并且能以低成本实现高性能提升以低成本实现高性能提升1 Wei,Wei,et al.LLMRec:Large Language Models

12、 with Graph Augmentation for Recommendation.WSDM 2024.12Representation Learning withLarge Language Models for Recommendation1 Ren,Xubin,et al.Representation Learning with Large Language Models for Recommendation.WWW 202413BackgroundBackground用户用户-商品交互图商品交互图推荐算法用户用户/商品商品协同过滤协同过滤特征表示特征表示噪音有噪的表征学习用户用户/

13、商品商品协同过滤协同过滤特征表示特征表示用户用户/商品商品文本模态文本模态特征表示特征表示促进融合表征中对推荐有益的部表征中对推荐有益的部分分1 Ren,Xubin,et al.Representation Learning with Large Language Models for Recommendation.WWW 202414理论方法理论方法最大化(;)critic function1.如何有效地获得如何有效地获得高质量的文本模态的特征表示高质量的文本模态的特征表示2.如何有效地建模如何有效地建模critic function1 Ren,Xubin,et al.Representat

14、ion Learning with Large Language Models for Recommendation.arXiv preprint2 Poole,Ben,et al.On variational bounds of mutual information.ICML 20193 Oord,Aaron van den,et al.Representation learning with contrastive predictive coding.arXiv preprint15 基于大模型的文本特征获取基于大模型的文本特征获取首先要有首先要有高质量的文本内容高质量的文本内容商品画像：

15、商品画像：描述其会吸引哪一类的用户群体用户画像：用户画像：描述其会喜欢什么类别的商品Item-to-User 生成范式生成范式商品画像生成商品画像生成,Prompts 构建构建基于基于 商品描述商品描述 生成生成基于基于 商品属性商品属性&用户反馈用户反馈 生成生成用户画像生成用户画像生成,Prompts 构建构建基于基于 商品画像商品画像&用户反馈用户反馈 生成生成核心是要描述出用户核心是要描述出用户/商品的商品的True preference1 Ren,Xubin,et al.Representation Learning with Large Language Models for Re

16、commendation.WWW 202416 基于大模型的文本特征获取基于大模型的文本特征获取其次要有其次要有高质量的特征表示高质量的特征表示文本描述Embedder用户用户/商品商品文本模态文本模态特征表示特征表示ContrieverInstructorText-embedding-ada-0021 Ren,Xubin,et al.Representation Learning with Large Language Models for Recommendation.arXiv preprint2 Izacard,Gautier,et al.Unsupervised dense info

17、rmation retrieval with contrastive learning.TMLR 20223 Su,Hongjin,et al.One embedder,any task:Instruction-finetuned text embeddings.ACL 202317建模建模critic functioncritic function对比式对齐（对比式对齐（Contrastive Alignment，RLMRec-Con）生成式对齐（生成式对齐（Generative Alignment，RLMRec-Gen）可以可以无缝嵌入无缝嵌入到任意以表征学习以基到任意以表征学习以基础的推

18、荐算法中础的推荐算法中1 Ren,Xubin,et al.Representation Learning with Large Language Models for Recommendation.WWW 202418ExperimentsExperiments在在协同过滤数据集（协同过滤数据集（Yelp,Amazon-book,steam）上进行测试，获得性能提升上进行测试，获得性能提升1 Ren,Xubin,et al.Representation Learning with Large Language Models for Recommendation.WWW 202419Experi

19、mentsExperiments控制变量调整文本表征的质量，控制变量调整文本表征的质量，越好的文本表征对性能提升越大越好的文本表征对性能提升越大1 Ren,Xubin,et al.Representation Learning with Large Language Models for Recommendation.WWW 202420ExperimentsExperiments1 Ren,Xubin,et al.Representation Learning with Large Language Models for Recommendation.WWW 2024对比式对齐（对比式对齐（

20、RLMRec-Con）抵御噪声抵御噪声对性能下降的影响对性能下降的影响生成式对齐（生成式对齐（RLMRec-Gen）提升预训练提升预训练对性能的增益对性能的增益2024-1-3021SummarizationSummarizationLLMRec:Large Language Models with Graph Augmentation for RecommendationRepresentation Learning with Large Language Models for Recommendationhttps:/ to follow us on the Social Media!Thanks!