Uncertainty-Aware Embeddings for Occupational Similarity: Evidence from German Labor Market Data

Thomas M. Menino Convention & Exhibition Center 

Neural language model-based text embeddings are increasingly used in labor economics to measure occupational similarity and skill distance, yet standard practice treats them as fixed-point estimates. We show that downstream analyses, such as wage regressions or automation risk scores, can be sensitive to embedding uncertainty, leading to overconfident and biased inference.

We present an uncertainty-aware framework. First, we construct embedding-based similarity measures from BERUFENET, a database of expert-written German occupational descriptions, enabling finer comparisons than standard taxonomies such as ISCO or the German KldB 2010. Second, we quantify embedding uncertainty via Monte Carlo dropout and ensemble methods, and propagate it through downstream estimators via simulation to obtain valid confidence intervals.

We demonstrate our approach to occupational mobility analysis, showing that ignoring embedding uncertainty can substantially underestimate standard errors. Our methods apply wherever embeddings serve as inputs to statistical models.

Uncertainty Quantification

LLM

Econometrics

Labor Economics

Text as Data 

Business and Economic Statistics Section