Evaluating the Impact of In-App Notifications

Music City Center 

This paper explores the impact of Search in-app notifications on user behavior, focusing on mitigating the potential costs associated with their use. Due to the inherent challenges in measuring the diverse benefits of notifications, we primarily concentrate on cost measurement. By establishing a budget system, we aim to prevent excessive notifications and protect user experience. Our methodology involves a three-period experimental setup—pre-period, treatment period, and post-period—to rigorously evaluate the effects of notification frequency on user behavior. To account for variations in user engagement with Google surfaces and, consequently, notification exposure, we utilize a matching method to conduct post-experimental analysis and identify heterogeneous treatment effects. Through this study, we seek to develop a more mature and consistent evaluation framework for in-app notifications, ultimately identifying the optimal frequency of notifications to balance short-term interruption and long-term user satisfaction.

Online experiments