Multi-Task Active Learning with Efficient Resource Allocation

Music City Center 

A sufficient amount of high-quality labeled data is essential for training supervised machine learning models; however, labeling all data can be costly, particularly in domains like healthcare and image classification. While Active learning (AL) has emerged as a promising approach to improve labeling efficiency, existing frameworks overlook scenarios where supplementary tasks with varying resource costs can assist the labeling process, such as out-of-network diagnostic tests. In this work, we introduce Active Learning with Cost-Adaptive Task Resource Allocations (ALCATRAs), a novel framework designed to optimize task resource allocation and improve model predictive performance under limited budgets. ALCATRAs consists of two main components: a task-selection policy which strategically selects a sequence of cost-effective tasks for unlabeled data to perform, and a surrogate learning procedure which transfers knowledge from completed tasks to enhance model predictions. Extensive experiments and applications to UC electronic health records (EHR) and the FashionMNIST benchmark dataset demonstrate the superior sample efficiency of our proposed ALCATRAs framework.

Active learning

Sequential decision-making

Surrogate learning

Sample efficiency

Feature acquisition

Resource allocation 

Uncertainty Quantification in Complex Systems Interest Group