A swarm of regressions from regression data with one and two predictors

Music City Center 

In a simple linear regression model, one has a numeric response variable Y and a predictor X. The model has an intercept β_0 and slope β_1, which are unknown. We assume X is numeric. We will have data (Y1, X1), (Y2, X2), …, (Yn, Xn). The observation Yi is drawn from the conditional distribution of Y|X=Xi. Using the data, one can estimate the intercept and slope of the model using the least squared method. The estimators are linear in Yi's, and are unbiased with minimum variance. Assume all Xi's are distinct. A unique line passes through any two points (Yi, Xi) and (Yj, Xj) with i≠j. We will have a swarm of lines, each of which provides unbiased estimators of β_0 and β_1. The swarm is used to develop a nonparametric regression model of Y on X. We show that a small subset of the swarm combined reproduces the least squared estimators. We extended this result to the case of two predictors.

simple linear regression

linear regression with two predictors

nonparametric simple linear regression

swarm of regressions 

Section on Nonparametric Statistics