Abstract Number:
1681
Submission Type:
Topic-Contributed Paper Session
Participants:
Anna Fergusson (1), Allison Theobold (3), Laura Le (2), Matthew Beckman (4), Jason Fleischer (5), Ayse Bilgin (6), Anna Fergusson (1)
Institutions:
(1) University of Auckland, N/A, (2) University of Minnesota, N/A, (3) Cal Poly, N/A, (4) Penn State University, N/A, (5) University of California (San Diego), United States, (6) Macquarie University, N/A
Chair:
Discussant:
Session Organizer:
Speaker(s):
Session Description:
Various data technologies and automated approaches can assist with assessment, but care is needed to develop tools and practices that value and support the human learning experience, at the same time as optimising for efficiency and accuracy. Tools are also needed that support teachers to make consistent and valid judgments across very large quantities of responses. This session will present talks related to research and practice within the emerging area of large scale automated-assisted assessment.
The focus for this session is timely as there is an urgent need to re-consider education and assessment policies in light of automation and generative models such as LLMs. Primarily using the teaching context of introductory statistics and data science courses, each speaker will discuss their development of different innovative assessment practices and the opportunities, challenges, and rewards of using automation with respect to teaching and research. The following talks are proposed for this session.
Navigating the Sea of Students: Innovations in Assessments for Massified Higher Education (Ayse Bilgin). Establishing individual connections with students in large lectures is challenging, so how can we ensure efficient marking of a large number of student answers within a week to provide valuable feedback for them to improve their learning? In this talk, I'll share our experiments and experiences in designing assessments that preserve student voices, utilise automation, maintain marking consistency, and uphold academic integrity to the best possible extent.
Developing tools for "real time" formative assessment of writing within large introductory statistics lectures (Anna Fergusson). Scalable methods of support are needed to help students not only produce high quality writing, but also understand the statistical and computational concepts that necessitate the careful use of language. This talk will present our pedagogical and technological explorations for supporting "real time" formative assessment of writing within large introductory statistics lectures.
Progress toward NLP-assisted formative assessment feedback (Matt Beckman). This talk seeks to articulate the benefit of free-response tasks and timely formative assessment feedback, a roadmap for developing human-in-the-loop natural language processing (NLP) assisted feedback, and results from a pilot study establishing proof of principle. This talk discusses the results and examines a preliminary cluster analysis of response text as a mechanism for scalable formative assessment.
Tools for automating project-based data science classes with hundreds of students (Jason Fleischer). We have developed an introductory data science course that serves between 400 and 700 upper division students every quarter. This talk will discuss decisions around the types of assessment used and whether to use automation or not. If there is time I will talk about using our automated tooling to support a pedagogical experiment investigating how group gender composition affects student satisfaction and group dynamics.
Sponsors:
No Additional Sponsor 3
No Additional Sponsor 2
Section on Statistics and Data Science Education 1
Theme:
Statistics and Data Science: Informing Policy and Countering Misinformation
Yes
Applied
Yes
Estimated Audience Size
Medium (80-150)
I have read and understand that JSM participants must abide by the Participant Guidelines.
Yes
I understand and have communicated to my proposed speakers that JSM participants must register and pay the appropriate registration fee by June 1, 2024. The registration fee is nonrefundable.
I understand