A Bayesian perspective on negative transfer

Music City Center 

Transfer learning is a framework for specifying and refining knowledge about the effects that transfer between a set of source (training) and of target (prediction) data. An open problem is addressing the empirical phenomenon of negative transfer, whereby the transfer learner performs worse on the target data after taking the source data into account than before. In this talk, I will introduce a Bayesian perspective on negative transfer and a method to address it. The key insight is that negative transfer can stem from misspecified prior information about non-transferable causes of the source data. Our proposed method does not require prior knowledge of the source data, and is thus applicable in the presence of latent confounders. Moreover, the learner need not have access to observations in the target task (be able to "fine-tune"), and instead makes use of proxy (indirect) information. Our theoretical results show that the threat of negative transfer does not depend on the informativeness of the proxy information, and our method applies even when only noisy indirect information is available. This talk is based on the paper available at https://arxiv.org/abs/2411.03263.

Bayesian methods

machine learning

transfer learning

robust inference

latent confounders

nuisance variables 

Business and Economic Statistics Section