The Strengthening Analytical Thinking in Observational Studies (STRATOS) Initiative

The Strengthening Analytical Thinking in Observational Studies (STRATOS) initiative is an international collective of statisticians and experts founded over 10 years ago. The mission of STRATOS is to elevate the current practice of statistics in observational studies by providing accessible and accurate guidance for the design and analysis of observational studies. In this talk, the organizational structure of STRATOS, its topic areas of focus, and outreach activities over the last ten years will be covered- highlighting the major successes and discussing challenges encountered. Current priorities and future areas of focus will also be discussed. One important focus is in connecting with analysts and investigators at all levels in the field to provide a series of tools and educational materials regarding how to avoid common pitfalls in the design and analysis of observational studies. We will present some of the educational materials currently available, where to access them, and how to get involved in STRATOS. 

STRATOS Topic Group 3 (TG3) deals with all assessment steps performed on the data of a study between the end of the data collection/entry and start of those statistical analyses that address research questions —referred to as Initial Data Analysis (IDA). Deficiencies in these preliminary steps may lead to the application of inappropriate statistical methods or incorrect conclusions. Consequently, TG3 develops guidance for systematically planning and conducting IDA1.
This presentation will first discuss the rationale for incorporating IDA into statistical analysis plans and outline how to effectively integrate it. Second, it will provide an overview of best practices for conducting IDA in the context of regression-type analyses2,3. Third, at the intersection of IDA and data quality assessment (DQA), approaches to data handling will be introduced to facilitate systematic data evaluation and checking1 4. The talk will highlight the critical role of effective metadata management—specifically, the structured annotation of knowledge about the data—in supporting both IDA and DQA.

1. Huebner M, le Cessie S, Schmidt CO, Vach W. A Contemporary Conceptual Framework for Initial Data Analysis. Observational Studies 2018;4:171-92.
2. Heinze G, Baillie M, Lusa L, et al. Regression without regrets -initial data analysis is a prerequisite for multivariable regression. BMC Med Res Methodol 2024;24:178.
3. Lusa L, Proust-Lima C, Schmidt CO, et al. Initial data analysis for longitudinal studies to build a solid foundation for reproducible analysis. PLoS ONE 2024;19:e0295726.
4. Struckmann S, Marino J, Kasbohm E, Salogni E, Schmidt CO. dataquieR 2: An updated R package for FAIR data quality assessments in observational studies and electronic health record data. . Journal of Open Source Software 2024;9:6581.
 

Despite advances in observational research methods used to estimate the effects of long-term drug exposures on safety outcomes, deficiencies in study design persist, potentially compromising the validity of reported evidence. STRATOS Topic Group 5 (TG5) focuses on developing guidance for designing observational studies, and is currently developing much-needed guidance in this area.

Informed by results of a scoping review on methods used in observational studies examining antidiabetic medications and fracture risk, this talk will review key study design aspects to consider when assessing the effects of long-term drug exposures on safety outcomes. Focus will be on establishing purposeful connections between etiologic and clinical pharmacology foundations and principles of pharmacoepidemiology. A case example will be presented to demonstrate how to appropriately consider key study design aspects when designing an observational study to estimate the effects of antidiabetic medications on hip fracture risk.
More specifically, three key study design aspects will be emphasized: (1) pathophysiology of the condition that indicated the drug exposure and how it may be implicated in the pathogenesis of the drug exposure-outcome relationship under study; (2) clinical pharmacology of the drug exposure; and (3) time trends in the drug exposure as well as in the outcome and other study variables at the population and individual levels.
In the context of the case example, aspects (1) and (2) are considered particularly important when defining a biologically plausible exposure risk window, exposure measurement, and a set of potential confounders. Aspect (3) is important to understand the time trends in the use of antidiabetic medications that may be due to changes in drug markets and policies at the population level, and in healthcare needs at the individual level, which may result in significantly biased effect estimates.
Careful consideration and transparent reporting of the three key study design aspects emphasized, and of study design and analytic approaches in general, will improve the validity of reported evidence from observational studies.
 

STRATOS Topic Group 4 focuses on providing guidance for appropriate statistical methods for data with measurement error or misclassification, where observed measurements differ from those we wish to observe. In this talk, we will provide an overview of recent contributions from the group. Two projects have focused on the categorization of continuous variables that are measured with error. This can result in misclassification - where measurements are categorized into the wrong category - with varied and complex implications for analysis. In one project, we explore various misconceptions about categorized error-prone variables. These include that categorization can assist with finding the shape of the exposure-outcome relationship, and that categorization can mitigate bias due to measurement error. We have also explored how to account for measurement error in such analyses via a relatively easy-to-implement method that combines MacMahon's method and regression calibration. We will also summarize separate work on the impact of measurement error on prediction. First, we discuss how predicted continuous variables have Berkson error, describe the impact of this error on inference, and propose a simple method to correct for bias in this context. We then also consider post-prediction inference in the context of regression calibration, including the need to account for the uncertainty in the regression calibration estimates of error-prone variables. For this, we discuss a stacked estimating equation approach, and an associated R package. Lastly, we briefly mention new areas of focus and will discuss outreach work in the form of short videos introducing the key themes, and challenges, of measurement error to a general audience. 

STRATOS Topic Group 2 (TG2) deals with "Selection of variables and functional forms in multivariable analysis". In multivariable regression, researchers build models relating an outcome variable to a set of predictor variables with the aim to either accurately predict the outcome or to investigate how the outcome is associated with the predictors. There is currently no known state-of-the-art method for variable and functional form selection in multivariable analysis [1]. Therefore, TG2 works on identifying and evaluating methods currently used in practice and conducting neutral comparison studies with the aim to develop consensus-based guidance.

As part of these efforts, we have re-evaluated prognostic models included in a systematic review [2] of diagnostic and prognostic models that were published during the Covid-19 pandemic. Our primary aim was to assess what model building strategies practitioners relied on when prediction models for Covid-19 were urgently needed. In our talk, we will describe which methods are commonly used, illustrate their weaknesses, identify misunderstandings related to their application and highlight good practices that should receive more attention.

[1] Sauerbrei W, Perperoglou A, Schmid M, et al. State of the art in selection of variables and functional forms in multivariable analysis—outstanding issues. Diagn Progn Res. 2020; 4(1). doi:10.1186/s41512-020-00074-3
[2] Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020; m1328. doi:10.1136/bmj.m1328