Cross-fitting model evaluation for small area estimation using complex survey data.
Abstract Number:
3360
Submission Type:
Contributed Abstract
Contributed Abstract Type:
Speed
Participants:
Qianyu Dong (1), Zehang Richard Li (1)
Institutions:
(1) University of California, Santa Cruz, CA
Co-Author:
First Author:
Presenting Author:
Abstract Text:
Model checking, evaluation, or comparison in Small Area Estimation (SAE) with limited data is difficult. A generic problem is that given a survey dataset D, what is a good metric to score a model M? Considering cluster sampling for the national surveys, we would like to achieve two goals: 1) to score models based on their ability to estimate subpopulation prevalence at different administrative levels. 2) to decide if a given model M can be accepted (or not rejected under a hypothesis testing framework). Focusing on a scenario where there is one level of spatial unit, we want to score models based on their ability to produce national estimates. We evaluate models using score rules such as mean square error (MSE), continuous ranked probability score (CRPS), and distribution-free score from conformal prediction, based on leave-one-region out, leave-one-cluster-out, or other splitting methods, and we use design-based estimates as a reference.
Keywords:
Cross validation|Small Area Estimation|Complex survey data| | |
Sponsors:
Survey Research Methods Section
Tracks:
Data Analysis/Modeling
Can this be considered for alternate subtype?
Yes
Are you interested in volunteering to serve as a session chair?
Yes
I have read and understand that JSM participants must abide by the Participant Guidelines.
Yes
I understand that JSM participants must register and pay the appropriate registration fee by June 1, 2024. The registration fee is non-refundable.
I understand
You have unsaved changes.