Student programming behavior with and without phone notification suppression

Abstract

Background and Context. Computer programming often involves extended periods of sustained activity and mobile phone notifications introduce frequent opportunities for interruption. Prior work demonstrates that suppressing phone notifications may reduce these disruptions. Objectives. Our primary research question is: How does suppressing phone notifications affect students' task engagement and productivity while programming? Method. We report on a replication and methodological extension study conducted in a CS1 course involving 22 students. Using a within-subject design, selected programming assignments were randomly designated for enabling notification suppression. Phone state logs were synchronized with millisecond-resolution IDE keystroke data to measure student attention and focus when in the control and notification-suppression conditions. Findings. Assignments completed with notification suppression enabled significantly lower break rates and longer intervals of focus compared to assignments completed in the control condition for many, but not all, students. This study provides evidence that notification suppression is associated with measurable differences in programming engagement and behavior. We also find a remarkable bimodality in the effect across students -- many students are positively affected, a small number are negatively affected, and very few experience little or no effect. This finding is consistent with other studies in diverse disciplines. Implications. Our results show that, for many students, phone notification suppression tools, such as Do Not Disturb, can improve attention and focus. Implications apply to educational settings (do-not-disturb as an intervention) and scholarship (understanding the effects of phone distraction).