Adaptive Deep Transfer Learning in high dimensional regression

Music City Center 

In high-dimensional regression, transfer learning provides an effective framework for leveraging knowledge from a related source domain to improve learning in a target domain. We propose an adaptive deep transfer learning approach for nonparametric regression, where the target function is approximated by leveraging knowledge from a pre-trained neural network on the source dataset. Specifically, we first train a deep neural network to learn the functional relationship in the source domain. Then, instead of fine-tuning the source model directly, we model the function difference between the source and target domains using a second neural network trained on the target dataset. The final target function approximation is obtained as the summation of these two networks, enabling adaptive knowledge transfer while preserving flexibility in high-dimensional settings. This framework enhances generalization by efficiently capturing both shared structures and domain-specific variations. We demonstrate the effectiveness of our method through theoretical analysis and empirical evaluations on synthetic and real-world datasets, showing improved predictive performance.

Neural networks

Nonparametric regression

Transfer learning 

Section on Statistical Learning and Data Science