Návrh Školení
Module 1 — AI Systems for Security Engineers
Lab: Lab 01 — 01-Introduction
Understanding the underlying architecture.
Topics:
- LLMs vs. normal applications
- AI inference pipelines
- Prompt flow
- RAG architecture
- Embeddings/vector databases
- Agentic workflows
- Tool calling
- AI gateways
- Copilots
- MCP and agent protocols
- Where WAF visibility exists
- Where WAF visibility disappears
Key insight: Traditional WAFs often lose visibility once the prompt reaches the model.
Module 2 — OWASP GenAI Top 10
Lab: none — interactive recap/discussion
Core AI attack categories.
Topics:
- Prompt Injection
- Insecure Output Handling
- Training Data Poisoning
- Model DoS
- Supply Chain Vulnerabilities
- Sensitive Information Disclosure
- Excessive Agency
- Vector/Embedding Weaknesses
- Misinformation
- Unbounded Consumption
Include:
- Differences from classic OWASP guidelines
- Mapping to defensive controls (WAF, gateway, app-layer)
- Where each control provides assistance
- Where each control fails
Module 3 — Prompt Injection Detection
Lab: Lab 02 — 02-Prompt-Injection
The “SQL injection moment” for AI systems.
Topics:
- Direct prompt injection
- Indirect prompt injection
- Hidden instructions
- Document-based attacks
- HTML/Markdown injection
- Jailbreak patterns
- Context override attacks
- Role confusion attacks
Detection strategies:
- Keyword heuristics
- Semantic classification
- Prompt linting
- Instruction boundary enforcement
- Allow/deny policies
- AI-aware regex patterns
Hands-on labs:
- Attack a chatbot
- Bypass naive filters
- Build layered detection mechanisms
Module 4 — AI-Aware WAF Rules
Lab: Lab 03 — 03-WAF-Basics
How WAF rules evolve for AI systems.
- Topics:
- Protecting LLM endpoints
- Inference API protection
- Token-aware rate limiting
- Prompt size inspection
- AI-specific signatures
- Conversation anomaly detection
- Multi-turn abuse patterns
- Model enumeration attempts
- Inference scraping
- Denial-of-wallet protection
Examples:
- Protecting /v1/chat/completions
- Defending streaming APIs
- Blocking recursive agent calls
Module 5 — Securing RAG Pipelines
Lab: Lab 04 — 04-RAG-Security
One of the most significant new attack surfaces.
Topics:
- Vector DB threats
- Embedding poisoning
- Malicious PDFs/docs
- Retrieval manipulation
- Semantic poisoning
- Hidden instructions in documents
- Cross-document contamination
- Data exfiltration via retrieval
Defenses:
- Ingestion sanitization
- Trust scoring
- Metadata isolation
- Document provenance
- Retrieval policies
- Segmentation
Case study: “Upload a poisoned PDF and take over the AI assistant.”
Module 6 — Agentic AI Security
Lab: Lab 05 — 05-Agent-Security
Where risks become dangerous.
Topics:
- Excessive agency
- Tool abuse
- API chaining
- Autonomous loops
- Permission escalation
- Memory poisoning
- Indirect tool execution
- Agent impersonation
- Credential leakage
- Multi-agent attacks
Defenses:
- Least privilege for agents
- Approval gates
- Runtime policy engines
- Sandboxing
- Scoped credentials
- Tool whitelisting
- Human-in-the-loop
This section is typically of greatest interest to managers because the risk becomes operational and directly impacts business outcomes.
Module 7 — API Security for AI
Lab: Lab 06 — 06-Denial-of-Wallet
AI systems are heavily reliant on APIs.
Topics:
- API gateways
- GraphQL AI risks
- MCP/API abuse
- JWT protection
- AI plugin security
- Agent authentication
- Delegated authorization
- Secret management
- Signed prompts
- API inventory for AI
Tied into: OWASP API Security Top 10
Module 8 — Detection Engineering & SOC Integration
Lab: Lab 07 — 07-Detection
Operational defense.
Topics:
- AI telemetry
- Prompt logging
- Token analytics
- Anomaly detection
- Semantic SIEM pipelines
- AI attack indicators
- Threat hunting for LLM abuse
- AI runtime observability
Examples:
- Detecting jailbreak campaigns
- Spotting automated agent abuse
- Identifying model scraping
Module 9 — Cloud WAFs and AI Security
Lab: none — interactive recap/discussion
Vendor-specific implementations.
Topics:
- AWS WAF for AI APIs
- Azure WAF
- Cloudflare AI Gateway
- API gateways
- Envoy AI filtering
- Kong AI Gateway
- NGINX AI security patterns
Comparison:
- Traditional WAF vs. AI gateway vs. app-layer guardrail
- Proxy-based vs. semantic filtering
Module 10 — Building a Layered AI Defense
Lab: Lab 08 — 08-Layered-Defense
Important philosophical conclusion:
No single layer can secure AI effectively (a WAF least of all when acting alone).
Students build a layered model:
- WAF
- API gateway
- AI gateway
- Guardrails
- Runtime monitoring
- Identity/authorization
- Sandbox
- Human approval
- Observability
- Incident response
This aligns strongly with the “multi-layer security” model.
Module ↔ Lab map
Labs run in lab order, which follows module order.
The course consists of 10 modules but only 8 labs: Modules 2 and 9 are interactive recaps/discussions and do not include a lab.
Each lab is tagged with its corresponding module throughout this outline.
- Lab 01 (Module 1)
- Folder: 01-Introduction
- Title: Explore an AI system — what's on the wire
- Lab 02 (Module 3)
- Folder: 02-Prompt-Injection
- Title: Attack a chatbot & bypass naive filtering
- Lab 03 (Module 4)
- Folder: 03-WAF-Basics
- Title: Build AI-aware WAF rules
- Lab 04 (Module 5)
- Folder: 04-RAG-Security
- Title: Poison a RAG pipeline
- Lab 05 (Module 6)
- Folder: 05-Agent-Security
- Title: Secure an autonomous agent
- Lab 06 (Module 7)
- Folder: 06-Denial-of-Wallet
- Title: Detect denial-of-wallet attacks
- Lab 07 (Module 8)
- Folder: 07-Detection
- Title: Monitor AI abuse patterns in logs
- Lab 08 (Module 10)
- Folder: 08-Layered-Defense
- Title: Build a layered AI defense architecture
Capstone
Students defend a simulated enterprise AI assistant.
Attackers attempt:
- Prompt injection
- Tool abuse
- Credential theft
- Retrieval poisoning
- Excessive API consumption
- Agent escalation
Teams build:
- WAF rules
- AI gateway policies
- Runtime detection
- Guardrails
- Incident response procedures
Požadavky
- Studenti by již měli rozumět zabezpečení HTTP/API, proxy/reverse proxy, autentizaci, OWASP Top 10, REST API a základnímu cloudovému síťování
Cílová skupina
- Bezpečnostní inženýři & AppSec
- Analytici SOC & inženýři detekce
- Inženýři zabezpečení API
- Zabezpečení cloudu / API / platformy
- DevSecOps inženýři
- Bezpečnostní architekti
- Specialisté na WAF / síťovou bezpečnost
- Inženýři AI platforem
Reference (2)
Měl jsem opravdu velký zájem o poznávání útoků na AI a nástroje dostupné pro zahájení praxe a aktivní používání při bezpečnostním testování. Získal jsem mnoho znalostí, které jsem na začátku neměl, a kurz splnil mé očekávání. Mou nejoblíbenější částí z tréninku byl Comet Browser a byl jsem ohromen tím, co umí. Určitě se tomu chci věnovat více. Celkově byl skvělý kurz a užil jsem si vše, co jsem se naučil o OWASP GenAI Top 10.
Patrick Collins - Optum
Kurz - OWASP GenAI Security
Přeloženo strojem
Profesionální znalosti a způsob, jakým nám je prezentoval
Miroslav Nachev - PUBLIC COURSE
Kurz - Cybersecurity in AI Systems
Přeloženo strojem
