Contributed Poster Presentations: Section on Medical Devices and Diagnostics

Medical device studies frequently present study design challenges due to the use of modest sample sizes and/or data distributions that differ from standard mathematical families of distributions. With increasing adoption of novel endpoints (e.g. hierarchical composites), statistical methods (e.g. restricted mean survival time for comparing time-to-event data), and complex adaptive designs, asymptotic formulas for sample size calculation and evaluation of operating characteristics are unavailable or unreliable. Simulation affords a practical and flexible method to optimize study design, through evaluation of the robustness of proposed statistical methods and sample size estimates. Similarly, in cases where pilot or feasibility study data is available, resampling methods may be employed to help evaluate the performance of planned statistical analysis methods and adequacy of sample size. This poster present examples of effective use of simulation and re-sampling based on recent experience with designing and reporting on medical device studies. 

Intro: Technological advances have bolstered biomarker discovery in personalized medicine. Evaluating biomarkers' ability to distinguish disease states is critical. DeLong's method, commonly used for comparing correlated AUCs (Area Under the Receiver Operating Curve) in diagnostic tests on the same subjects, has been reported to underestimate confidence interval coverage in small samples with high correlations between tests, as indicated by recent studies[1]. Methods: We compared DeLong's method with a Bootstrap normal approximation via simulations using logistic regression models under various conditions. Variations included sample sizes (20-200), case-control ratios (1:1 to 1:5), AUC levels (0.5-0.8), and test correlations (0-0.75). Results: Though results suggest poor coverage probability for both DeLong and Bootstrap normal approaches at small sample sizes, the Bootstrap approach consistently outperformed DeLong's method, especially at higher correlations. This pronounced improvement at higher correlations advocates the Bootstrap method as a superior alternative for AUC comparison in small samples with correlated biomarkers. 

Test sensitivity in cancer screening, defined as the likelihood that a screening test will correctly identify the presence of pre-clinical disease, is a key driver of its potential benefit. However, studies on new biomarkers often reports sensitivity at the point of clinical diagnosis, which is generally higher than what is observed in pre-clinical stages. Our study investigates the relationship between diagnostic and pre-clinical sensitivities in cancer screening and explore the factors contributing to their discrepancies. We model the true sensitivity increasing over time after pre-clinical onset, within a natural history model of disease progression. This increase continues until the point of clinical diagnosis and sensitivity at this point is pre-specified. The pre-clinical sensitivity is determined by averaging the true sensitivity observed at the time of screening. Further, we introduce a multi-state model of disease progression that accounts for varying sensitivity levels at different stages of clinical diagnosis. We find the overall pre-clinical sensitivity would generally underestimate the diagnostic sensitivity, whereas the sensitivity measured at the early stages of pre-clinical disease may be optimistic. The model using clinical sensitivity to project the benefits of novel biomarkers in cancer screening maybe overly optimistic. Models using stage-specific sensitivity maybe more appropriate. 

Gait cycle variability during steady walking, characterized by the stride interval time series, has been
used as a measure of gait stability. Positive persistency (long-range positive
correlation), representing a 1/f-like fluctuation of stride interval, in healthy young adults and
reduction of persistency due to aging and/or neurological diseases, respectively, are well-
documented metrics for stability and instability of gait. We examined effects of a dual task on
gait cycle variability in healthy young adults, based on the mean and std dev statistics as
well as the positive persistency of the treadmill stride interval time series during walking .
Three gait conditions were examined: control condition, non-cognitive task holding
a smartphone in front of chest using dominant hand and looking fixedly at a blank screen of
smartphone, and cognitive task with holding smartphone as in non-cognitive task and
playing a puzzle game displayed on the smartphone by one-thumb operation. Only
positive persistency was affected by the cognitive and motor load of smartphone usage.Positive persistency in the
control & non-cognitive conditions was significantly reduced in cogniive case