arXiv subjects

Connor Nelson

Publications and source records attributed to Connor Nelson.

3 recordsLinked to original sources

arXiv2026

Do Hackers Dream of Electric Teachers?: A Large-Scale, In-Situ Evaluation of Cybersecurity Student Behaviors and Performance with AI Tutors

To meet the ever-increasing demands of the cybersecurity workforce, AI tutors have been proposed for personalized, scalable education. But, while AI tutors have shown promise in introductory programming courses, no work has evaluated their use in hands-on exploration and exploitation of systems (e.g., ``capture-the-flag'') commonly used to teach cybersecurity. Thus, despite growing interest and need, no work has evaluated how students use AI tutors or whether they benefit from their presence in real, large-scale cybersecurity courses. To answer this, we conducted a semester-long observational study on the use of an embedded AI tutor with 309 students in an upper-division introductory cybersecurity course. By analyzing 142,526 student queries sent to the AI tutor across 396 cybersecurity challenges spanning 9 core cybersecurity topics and an accompanying set of post-semester surveys, we find (1) what queries and conversational strategies students use with AI tutors, (2) how these strategies correlate with challenge completion, and (3) students' perceptions of AI tutors in cybersecurity education. In particular, we identify three broad AI tutor conversational styles among users: Short (bounded, few-turn exchanges), Reactive (repeatedly submitting code and errors), and Proactive (driving problem-solving through targeted inquiry). We also find that the use of these styles significantly predicts challenge completion, and that this effect increases as materials become more advanced. Furthermore, students valued the tutor's availability but reported that it became less useful for harder material. Based on this, we provide suggestions for security educators and developers on practical AI tutor use.

cs.HC↗

arXiv2025

CTF Archive: Capture, Curate, Learn Forever

Capture the Flag (CTF) competitions represent a powerful experiential learning approach within cybersecurity education, blending diverse concepts into interactive challenges. However, the short duration (typically 24-48 hours) and ephemeral infrastructure of these events often impede sustained educational benefit. Learners face substantial barriers in revisiting unsolved challenges, primarily due to the cumbersome process of manually reconstructing and rehosting the challenges without comprehensive documentation or guidance. To address this critical gap, we introduce CTF Archive, a platform designed to preserve the educational value of CTF competitions by centralizing and archiving hundreds of challenges spanning over a decade in fully configured, ready-to-use environments. By removing the complexity of environment setup, CTF Archive allows learners to focus directly on conceptual understanding rather than technical troubleshooting. The availability of these preserved challenges encourages in-depth research and exploration at the learner's pace, significantly enhancing conceptual comprehension without the pressures of live competition. Additionally, public accessibility lowers entry barriers, promoting an inclusive educational experience. Overall, CTF Archive provides a scalable solution to integrate persistent, practical cybersecurity learning into academic curricula.

cs.CR↗

arXiv2021

Technical Report -- Expected Exploitability: Predicting the Development of Functional Vulnerability Exploits

Assessing the exploitability of software vulnerabilities at the time of disclosure is difficult and error-prone, as features extracted via technical analysis by existing metrics are poor predictors for exploit development. Moreover, exploitability assessments suffer from a class bias because "not exploitable" labels could be inaccurate. To overcome these challenges, we propose a new metric, called Expected Exploitability (EE), which reflects, over time, the likelihood that functional exploits will be developed. Key to our solution is a time-varying view of exploitability, a departure from existing metrics. This allows us to learn EE using data-driven techniques from artifacts published after disclosure, such as technical write-ups and proof-of-concept exploits, for which we design novel feature sets. This view also allows us to investigate the effect of the label biases on the classifiers. We characterize the noise-generating process for exploit prediction, showing that our problem is subject to the most challenging type of label noise, and propose techniques to learn EE in the presence of noise. On a dataset of 103,137 vulnerabilities, we show that EE increases precision from 49% to 86% over existing metrics, including two state-of-the-art exploit classifiers, while its precision substantially improves over time. We also highlight the practical utility of EE for predicting imminent exploits and prioritizing critical vulnerabilities. We develop EE into an online platform which is publicly available at https://exploitability.app/.

cs.CR↗