ISCAP Proceedings: Abstract Presentation
When AI Plays The CTF: Rethinking Cybersecurity Assessment In The Age Of Generative AI
Angela Orebaugh
University of Virginia
Alex Chen
University of Virginia
Abstract
Capture-the-Flag (CTF) competitions are widely used in cybersecurity education to assess students’ technical knowledge, problem-solving abilities, and practical skills. As generative artificial intelligence (AI) tools have become increasingly capable, their role in cybersecurity learning and assessment has changed dramatically. Since Spring 2023, students enrolled in an undergraduate Introduction to Cybersecurity course have been permitted to use AI tools while completing an individual CTF-based final examination. This policy has provided a unique opportunity to observe how AI capabilities have evolved and how those changes affect the educational value of CTF assessments.
Early observations from Spring 2023 suggested that AI tools provided limited assistance. Models frequently hallucinated solutions, generated incorrect commands, and struggled with common cybersecurity tasks such as cryptographic analysis and forensic investigation. By 2024 and 2025, AI systems became more effective as coaching tools, offering useful hints, identifying relevant techniques, and helping students navigate unfamiliar tools. Recent observations from Spring 2026 indicate a substantial shift: modern frontier models can independently solve many traditional CTF challenges with little or no human intervention.
A student-led exploratory study conducted on the CTF examined AI performance across challenges spanning reconnaissance, cryptography, forensics, network security, web security, and binary analysis. Results suggest that AI can now fully solve many challenge types, including metadata analysis, file recovery, string extraction, and basic cryptography. However, limitations remain. AI performance decreases when challenges require complex multi-step reasoning, exploitation workflows, or the use of external tools and environments. Certain tasks, such as advanced binary exploitation and some forms of packet-capture analysis, still require meaningful human guidance. The study also documented persistent issues with hallucinations and overconfidence.
These observations raise important questions for cybersecurity educators. If AI can independently solve a growing percentage of traditional CTF challenges, what skills are actually being assessed? Should CTFs be redesigned to emphasize higher-order reasoning, validation of AI-generated outputs, and human-AI collaboration? How should instructors distinguish between productive AI assistance and the outsourcing of learning?
This work-in-progress presentation will share longitudinal classroom observations, examples of AI successes and failures across multiple years, and preliminary findings from the student-led exploratory study. The presentation will also outline a research agenda focused on developing an AI-resistance taxonomy for CTF challenges, measuring challenge solvability across AI model generations, and identifying assessment strategies that preserve learning outcomes in an era of increasingly capable generative AI. Feedback from ISCAP attendees will be sought regarding research design, evaluation metrics, and future collaborative opportunities.