Thursday, Aug 8: 8:30 AM - 10:20 AM
5179
Contributed Papers
Oregon Convention Center
Room: CC-E147
Main Sponsor
IMS
Presentations
The model-X framework provides provable non-asymptotical error control on variable selection and conditional independence testing. It has no restrictions or assumptions on the dimensionality of the data or the conditional distribution of the response given the covariates. To relax the requirement of the model-X framework that the distribution of the covariate samples is precisely known, we proposed to construct knockoffs by conditioning on sufficient statistics when the distribution is known up to a parametric model with as many as Ω(np) parameters, where p is the dimension and n is the number of covariate samples (including unlabeled samples if available). We demonstrate how this idea can be implemented in Gaussian graphical models and show the new approach remains powerful under the weaker assumption. We will discuss how such conditioning can be extended to constructing a conditional randomization test for testing conditional independence between the response and a subset of the covariates.
Keywords
variable selection
knockoff
model-X
Gaussian graphical model
randomization test
goodness-of-fit test
In nonparametric regression settings we propose a novel concept of "individual variable importance'', referring to the relevance of some covariates with respect to an outcome variable among individuals with certain features. This concept holds practical importance for risk assessment and association identification. It can represent usefulness of expensive biomarkers in disease prediction for individuals at certain baseline risk, or age-specific associations between physiological indicators. We quantify the individual variable importance by a ratio parameter between two conditional mean squared errors, for which we develop nonparametric estimators. We demonstrate our approaches through a real data application, showing a scientifically interesting result: the association between body shape and systolic blood pressure decays with increasing age. While aligning with the existing medical literature based on parametric regression, our finding is more reliable since its validity is not affected by model misspecification. The fully nonparametric nature equips the individual variable importance framework with broader applicability in contexts that go beyond traditional parametric modeling.
Keywords
Confidence interval
Convergence rate
Individual variable importance
Kernel smoothing
Nonparametric regression
We show a dependence condition, positive regression dependence on subsets, for multivariate X² random variables of a generalized Wishart type. This class of random variables occurs asymptotically on collections of test statistics arising from analysis of variance models. The dependence condition holds when the test statistics are generated under an arbitrary combination of null and alternative hypotheses, where the subset on which positive regression dependence holds is those test statistics realized under the null hypothesis. The condition is therefore sufficient for application of several multiple testing adjustment procedures including false discovery rate adjustment, and useful for high-dimensional settings when differences between multiple strata is of inferential interest.
Keywords
multiple testing
positive regression dependence
false discovery rate
positive regression dependence on subsets
First Author
David Swanson, University of Texas MD Anderson Cancer Center
Presenting Author
David Swanson, University of Texas MD Anderson Cancer Center
An unfortunate feature of traditional hypothesis testing is the necessity to pre-specify a significance level α to bound the 'size' of the test: its probability to falsely reject the hypothesis. Indeed, a data-dependent selection of α would generally distort the size, possibly making it larger than the selected level α. We develop post-hoc hypothesis testing, which guarantees that there is no such size distortion in expectation, even if the level α is arbitrarily selected based on the data. Unlike regular p-values, resulting 'post-hoc p-values' allow us to 'reject at level p' and still provide this guarantee. Moreover, they can easily be combined since the product of independent post-hoc p-values is a post-hoc p-value. Interestingly, we find that p is a post-hoc p-value if and only if 1/p is an e-value, a recently introduced measure of evidence. Post-hoc hypothesis testing eliminates the need for standardized levels such as α = 0.05. We believe this may take away incentives for p-hacking and contribute to solving the file-drawer problem, as both these issues arise from using a pre-specified significance level.
Keywords
p-value
e-value
post-hoc inference
anytime valid inference
p-hacking
Scientific discourse
Abstracts
Recently, there has been interest in testing hypotheses under misspecification. In tolerant testing, the practitioner states a simple null hypothesis and indicates how much deviation from it should be tolerated when testing it. In this work, we study the tolerant testing problem in the Gaussian sequence model. Specifically, given an observation from a high-dimensional Gaussian distribution, is the p-norm of its mean less than δ (null hypothesis) or greater than ε (alternative hypothesis)? When δ = 0, the problem reduces to simple hypothesis testing, while δ > 0 indicates how much imprecision in the null hypothesis should be tolerated. Via the minimax hypothesis testing framework, we characterise the smallest separation between null and alternative hypotheses such that it is possible to consistently distinguish them. Extending the results of Ingster 2001, we find that as δ is increased, the hardness of the problem interpolates between simple hypothesis testing and functional estimation. Furthermore, our results show a strong connection to tolerant testing with multinomial data (Canonne et al. 2022).
Keywords
minimax
hypothesis testing
tolerant testing
imprecise hypothesis
Gaussian sequence model
misspecification
Decision-making pipelines involve trading-off risks with rewards.
It is often desirable to determine how much risk can be tolerated based on measured quantities, using collected data.
In this work, we address this problem, and allow decision-makers to control risks at a data-dependent level.
We demonstrate that, when applied without modification, state-of-the art uncertainty quantification methods can lead to gross violations on real problem instances when the levels are data-dependent.
As a remedy, we propose methods that permit the data analyst to claim valid control.
Our methodology, which is based on uniform tail bounds, supports monotone and nearly-monotone risks, but otherwise makes no distributional assumptions.
To illustrate the benefits of our approach, we carry out numerical experiments on synthetic data and the large-scale vision dataset MS-COCO.
Keywords
Conformal prediction, uncertainty quantification, distribution-free, bootstrap, empirical process theory, simultaneous inference
Traditional statistical inference methods often face limitations due to their reliance on strict assumptions. Moreover, these methods are typically tailored to specific assumptions, restricting their adaptability to any alternative set of assumptions. In this work, we present a unified framework for deriving confidence intervals for various functionals (e.g., mean or median) under a broad class of user-specified assumptions (e.g., finite variance or tail behavior). Leveraging confidence sets for cumulative distribution functions (CDFs), this framework offers a principled and flexible inference strategy, reducing dependence on stringent assumptions and providing applicability in diverse contexts.
Keywords
Confidence intervals
Assumption-lean inference
Confidence sets for CDF
Semi-infinite programming
Non-parametric methods