Innovations in AI-Enhanced Peer Review and Automated Data Analysis Agents

Music City Center 

Room: CC-Dean Grand Ballroom A1 

This session highlights two advances at the nexus of machine learning and statistical methodology.

The first paper, "LAMBDA: A Large Model Based Data Agent," introduces a code‐free, multi‐agent framework for statistical data analysis powered by large language models. A "programmer" agent generates domain‐aware code, while an "inspector" agent debugs, and a Knowledge Integration Mechanism incorporates external algorithms. Demonstrated on real‐world datasets, LAMBDA blends human‐in‐the‐loop statistical reasoning with AI automation to democratize complex analyses. Together, these contributions illustrate how ML‐driven tools can enhance the rigor and efficiency of both peer review and statistical data workflows.

The second paper, "The ICML 2023 Ranking Experiment," presents a large‐scale study in which 1,342 authors ranked 2,592 submissions by perceived quality. Using an Isotonic Mechanism to recalibrate reviewer scores with these self‐assessments, the authors achieve significant reductions in squared and absolute error. They propose low‐risk applications—supporting senior area chairs, paper awards, and emergency reviewer recruitment—to seamlessly integrate statistical calibration into peer review.

Yes

JASA Applications and Case Studies

Association for the Advancement of Artificial Intelligence

Co-Author(s)

Jian Huang
Weijie Su, University of Pennsylvania

Speaker

Jian Huang