Detecting and Mitigating Algorithmic Bias in Online Misinformation

Oregon Convention Center 

There is perhaps no bigger issue facing our field right now than that of misinformation – and the advent of tools like ChatGPT has increased this risk. A central reason for this is bias from large language models and how that can lead to misleading and/or incorrect information disproportionately impacting certain communities. NORC is developing a model of online information to better understand how to detect and mitigate bias in such models. The data are focused specifically on the topic of COVID vaccine misinformation, which the study team chose because of the strong historical record of misinformation across social media platforms and issues related to health equity. NORC collected more than 10 terabytes of data from across Twitter and Instagram from 2020 to 2023. The study team hand-coded a training sample, building upon several different open-source misinformation indexes, and then trained and deployed the model. This presentation will share learnings from this process; share the model developed; and finally, describe the learnings gleaned from the process related to bias in LLM development.