Applied AI Security: Attacking and Defending LLMs
Description
Name of Training: Applied AI Security: Attacking and Defending LLMs
Trainer(s): Drinor Selmanaj, Iva Amos
Dates: November 08-09, 2026
Time: 9:00 am - 5:00 pm
Venue: Exhibition World Bahrain
Cost: 800 BHD
1. Short Summary
Applied AI Security: Breaking and Defending LLMs training focuses on how AI systems fail in practice, how those failures can be abused, and how to stop them. Learners exploit LLMs the way attackers do, then flip sides and build defenses that work. All exercises run on a professional cyber range purpose-built for hands-on skill development.
Applied AI Security: Breaking and Defending LLMs is an in-depth, hands-on training that examines the security of systems built on large language models (LLMs) as they are deployed in real-world environments. As organizations increasingly integrate LLMs into applications, services, and operational workflows, these systems introduce new security risks that extend beyond traditional application and infrastructure security models.
2. Course Description
This course focuses on how LLM-based systems fail in practice, how those failures can be abused, and how they can be mitigated through sound security engineering and operational controls.
Participants will explore common design patterns used in LLM-enabled applications, including prompt-based interfaces, retrieval-augmented systems, tool-using agents, and automated workflows. Through guided exercises, attendees will analyze how trust boundaries shift in these systems and how misuse, manipulation, or unsafe behavior can lead to data exposure, unauthorized actions, or downstream system impact.
The training takes a balanced approach between offense and defense. On the offensive side, participants will examine techniques used to manipulate model behavior, influence decision-making, and abuse integrations between LLMs and external systems.
On the defensive side, the course emphasizes secure design principles, input and output controls, monitoring strategies, and response mechanisms tailored to AI-enabled systems. Rather than focusing on isolated model weaknesses, the course treats AI security as a system-level problem involving architecture, configuration, and operational context.
All concepts are reinforced through hands-on labs conducted in realistic environments designed to reflect how LLMs are used in production. Participants will practice assessing risk, testing AI-enabled features, and applying mitigations that are practical and maintainable.
The course also addresses operational considerations, including how to evaluate AI-related security incidents, how to integrate AI security into existing security workflows, and how to reason about risk when deploying or managing LLM-based systems.
By the end of the training, participants will have a practical understanding of how to evaluate the security posture of LLM-powered systems, how to identify and test meaningful failure modes, and how to design and operate defenses that reduce risk while allowing AI systems to remain useful and effective.
3. Course Outline
Day 1
3.1 AI Systems Security Foundations
Topics
3.1.1 How LLM-based systems are built and deployed in practice
3.1.2 Trust boundaries in AI-enabled workflows
3.1.3 Why traditional AppSec assumptions fail for LLMs
Labs
3.1.4 Exploring an LLM-enabled application
3.1.5 Identifying components, data flows, and trust boundaries
3.2 LLM Architecture and Integration Patterns
Topics
3.2.1 Prompt-driven applications and system prompts
3.2.2 Retrieval-augmented generation (RAG)
3.2.3 Tool use, plugins, and agent-based workflows
3.2.4 Where attackers gain leverage in real deployments
Labs
3.2.5 Architecture mapping of an LLM-based system
3.2.6 Identifying attacker-controlled inputs and integration risks
3.3 AI Red Teaming Concepts and Methodology
Topics
3.3.1 What AI red teaming is (and what it is not)
3.3.2 Red teaming LLM systems vs traditional applications
3.3.3 Adversarial thinking for language-driven systems
3.3.4 Using structured frameworks to guide testing
Frameworks discussed
3.3.5 OWASP guidance for AI and LLM risk categorization
3.3.6 MITRE ATLAS for adversary behavior modeling
Labs
3.3.7 Mapping AI system behaviors to structured red-team categories
3.3.8 Identifying realistic abuse goals and success criteria
3.4 Threat Modeling for AI Systems
Topics
3.4.1 Threat modeling LLM systems end-to-end
3.4.2 Assets, actors, and abuse paths
3.4.3 Differentiating misuse, abuse, and unintended behavior
3.4.4 Prioritizing attacks based on real-world impact
Labs
3.4.5 Threat modeling an AI-enabled workflow
3.4.6 Selecting high-impact red-team test scenarios
3.5 Breaking LLM-Based Systems (Red Team Execution)
Topics
3.5.1 Prompt manipulation and behavioral influence
3.5.2 Indirect input abuse through documents, data sources, or tools
3.5.3 Unsafe agent behavior and excessive autonomy
3.5.4 Chaining weaknesses across system components
Labs
3.5.5 Executing controlled red-team scenarios against an LLM system
3.5.6 Demonstrating unintended actions or data exposure
Day 2
3.6 Defensive Engineering, and Operations Defensive Design for LLM-Based Systems
Topics
3.6.1 Secure design principles for AI-enabled applications
3.6.2 Reducing attack surface in prompts, tools, and agents
3.6.3 Managing permissions, autonomy, and scope
Labs
3.6.4 Hardening an LLM system against identified red-team techniques
3.6.5 Applying guardrails and constraints
3.7 Monitoring and Detection for AI Abuse
Topics
3.7.1 Observability challenges in AI workflows
3.7.2 Logging, tracing, and behavior monitoring
3.7.3 Detecting prompt manipulation, misuse, and abnormal behavior
3.7.4 Turning red-team findings into detection logic
Labs
3.7.5 Instrumenting an LLM system for visibility
3.7.6 Detecting simulated red-team activity
3.8 AI Blue Teaming and Incident Response
Topics
3.8.1 Responding to AI-specific security incidents
3.8.2 Containment and remediation for AI misuse
3.8.3 Assessing downstream and organizational impact
3.8.4 Coordinating technical and non-technical response
Labs
3.8.5 Handling a simulated AI security incident
3.8.6 Analysis, containment, and response decisions
3.9 Operationalizing AI Security and Red Team Feedback
Topics
3.9.1 Feeding red-team findings into secure development
3.9.2 Continuous testing of AI-enabled features
3.9.3 Aligning AI security with organizational risk management
3.9.4 Using red teaming to improve system design over time
Labs
3.9.5 Creating an AI red-team test plan for a real deployment
3.9.6 Defining testing scope, goals, and reporting outputs
3.10 Capstone: End-to-End AI Security Assessment
Topics
3.10.1 Applying red and blue team concepts together
3.10.2 Evaluating risk, controls, and residual exposure
3.10.3 Communicating findings clearly to stakeholders
Labs (Capstone / Certification)
3.10.4 Full assessment of an LLM-enabled system
3.10.5 Identify risks, execute red-team tests, and propose defenses
4. Difficulty Level
Intermediate - The student has education and some experience in the field and familiarity with the topic being presented. The student has foundational knowledge that the course will leverage to provide practical skills on the topic.
5. Suggested Prerequisites
Students should have:
- Basic familiarity with using popular AI or chat-based tools
- Ability to read and modify basic Python code
- Comfort using command-line tools and working in a lab environment
- Basic understanding of how APIs and modern web applications are structured
6. What Students Should Bring
- Laptop with Wi-Fi
- Modern web browser
- Ability to follow guided, hands-on lab instructions
7. What the Trainer Will Provide
During the training, students will be provided with:
- Pre-configured adversary and target infrastructure
- Custom tooling and scripts
- Digital course workbook and reference materials
8. Trainer Bio
Drinor Selmanaj
Drinor Selmanaj is a cybersecurity practitioner, researcher, and instructor specializing in adversary emulation, threat-informed defense, and offensive security operations. He is the author of the O’Reilly book Adversary Emulation with MITRE ATT&CK, which provides a structured methodology for translating real-world threat intelligence into realistic, measurable adversary campaigns.
With over a decade of experience across penetration testing, red teaming, and national-level cybersecurity consulting, Drinor has worked with multinational enterprises, critical infrastructure operators, and government-aligned stakeholders. He is the founder of Cyber Academy, where he designs and delivers advanced cybersecurity training programs and develops cyber ranges focused on realistic offensive and defensive scenarios.
Iva Amos
Iva Amos is a cybersecurity professional and practitioner researcher specializing in AI security, prompt injection, and agentic AI risk. She leads a multimillion CISA federal contract for upskilling government employees and architects training programs for over 70% of the Fortune 500 at Infosec.
The scope of her work spans a professional cyber range platform with 600+ labs, training 10K+ security professionals annually. She also teaches cybersecurity and AI/ML at two universities. She presents breakout sessions and workshops at Gartner, EDUCAUSE, ISACA, NICE, Infosec World, and Hack Space Con, and brings 19+ years in technology with deep focus on cybersecurity and AI security training.
Iva and Drinor have worked together for over a year, architecting hands-on lab exercises for multiple training programs. Their most recent collaboration produced the labs for Infosec's "Generative and Agentic AI for Cybersecurity Professionals" course.
9. Registration Terms and Conditions
9.1 Trainings are refundable before September 21, 2026, minus a non-refundable processing fee of 100 BHD.
9.2 Between September 21, 2026 and October 21, 2026 partial refunds will be granted, equal to 50% of the course fee minus a processing fee of 100 BHD.
9.3 All trainings are non-refundable after October 21, 2026.
9.4 Training tickets may be transferred to another student. Please email us at training@defcon.org for specifics.
9.5 If a training does not reach the minimum registration requirement, it may be cancelled. In the event the training you choose is cancelled, you will be provided the option of receiving a full refund or transferring to another training (subject to availability).
9.6 Failure to attend the training without prior written notification will be considered a no-show. No refund will be given.
9.7 DEF CON Training may share student contact information, including names and emails, with the course instructor(s) to facilitate sharing of pre-work and course instructions. Instructors are required to safeguard this information and provide appropriate protection so that it is kept private. Instructors may not use student information outside the delivery of this course without the permission of the student.
9.8 By purchasing this ticket you agree to abide by the DEF CON Training Code of Conduct and the registration terms and conditions listed above.
