没有环境标签的不变性学习的若干问题探讨.pdf

1、ZIN:When and How to Learn Invariance withoutDomain PartitionYong LinOctober 18,2023(Yong Lin)ZIN:When and How to Learn Invariance without Domain PartitionOctober 18,20231/24Contents1Learning Invariance without Environment Indexes2References(Yong Lin)ZIN:When and How to Learn Invariance without Domai

2、n PartitionOctober 18,20232/24IntroductionThe common i.i.d(independent and identically distributed)assumption does not always hold.In many real world applications,we may encounter novel testingdistribution different from the training one.Known as the out-of-distribution generalization(OOD)problem.(Y

3、ong Lin)ZIN:When and How to Learn Invariance without Domain PartitionOctober 18,20233/24A Motivating Example(Yong Lin)ZIN:When and How to Learn Invariance without Domain PartitionOctober 18,20234/24Invariance in CausalityLemma(Invariance Property)The conditional distribution of Y given the the direc

4、t causes will notchange when we intervene on any other node except for Y.Figure:Images taken from Peters et al.,2016.The conditional EY|X2,X4remains invariant under each possible intervention on nodes except for Y.Invariant Causal Prediction(ICP)Peters et al.,2016 first proposes toutilize the invari

5、ance property to identify Ys parent.(Yong Lin)ZIN:When and How to Learn Invariance without Domain PartitionOctober 18,20235/24Invariant Risk Minimization(IRM)Definitions:Invariant Features Xinv:Direct cause of Y,i.e.,X2,X4in the formerexample.Spurious Features Xs:Features other than direct cause.IRM

6、 seeks to learn an representation(X)to exclusively rely on invariantfeatures.(Yong Lin)ZIN:When and How to Learn Invariance without Domain PartitionOctober 18,20236/24MotivationIRM requires sufficient environments to learn invariance.However,given acollected dataset(that may contains a mixture of en