AI Security Combo Course: Cybersecurity AI Foundation | AI Security Architecture

• Phase 1: Cybersecurity AI Foundation Program

  Part 1 – AI Fundamentals

  • • Module 1: Introduction to AI
      • What is Artificial Intelligence: AI, Machine Learning, Deep Learning
      • Components of an AI System
      • Brief history and evolution: from rule-based systems to modern deep neural networks
      • How Machines Learn: Supervised learning, Unsupervised learning, Reinforcement learning
      • Types of AI Models and What They Do: Classification models, Regression models, Clustering models, Language models
      • AI Model development lifecycle
      • AI Tools Landscape: Huggingface, OpenWebUI, GenAI platforms, Open-source model platforms for running models locally: Ollama, LM Studio
      • Prompt Engineering Techniques
    • Module 2: Python Basics for Using AI Frameworks
      • Python Fundamentals for AI
      • Lab Setup: Linux VM, Using Jupyter Notebook
      • Google Colab: free cloud compute for coding and AI/ML experiments
      • Python Libraries for Data Analysis and Visualisation: Pandas, NumPy, Matplotlib
      • Types of Data and using them for AI Model development
      • Secure and Responsible use of Data in AI models and Applications
    • Module 3: Machine Learning for Security
      • Libraries and frameworks for Machine Learning and Deep Learning
      • Data Collection
      • Data Processing (Including Feature Extraction and Normalization)
      • Algorithm selection and Model Training
      • TEVV: Test, Evaluation, Verification, and Validation and Fine Tuning
      • Model Deployment, Model Monitoring and Retraining
      • Practical Lab: Building AI (ML and DL) models for Network Security by using GenAI
    • Module 4: Natural Language Processing (NLP)
      • Libraries and frameworks for NLP
      • Text Processing for NLP: Tokenization, Stop words removal, Embeddings
      • NLP Feature Engineering Types
      • Converting Emails and Logs to Vectors
      • Practical Lab: Building a Phishing Mail Detector
    • Module 5: Generative AI, LLMs and Agentic AI
      • Generative Adversarial Networks
      • Transformer Architecture
      • System Prompts and User Prompts
      • Prompt Engineering Techniques
      • Foundational model and fine-tuned model
      • Retrieval Augmented Generation (RAG)
      • Agentic AI: Autonomous agents, Planning, Reasoning, Action: tool use, and multi-agent orchestration frameworks
      • Agentic AI Frameworks: Langchain, CrewAI
      • Model Context Protocol (MCP)
      • Practical Lab: Building a Gen AI Based Custom Chatbot and Agent

  Part 2 – AI Security and Governance

  • • Module 6: Attacking AI
      • Poisoning: Data and Model Poisoning
      • Backdoor Attacks with Trigger-Based Poisoning
      • Evasion Attacks: Evading AI-Based Detectors
      • Model and Data Theft
      • Inversion and Membership Inference Attacks
      • OWASP Machine Learning Security Top 10
      • Prompt Injection: Direct and Indirect
      • Jailbreaking Techniques
      • OWASP Top 10 for Large Language Model Applications
      • OWASP Top 10 for Agentic Applications
      • Practical Lab: Automated LLM Pentesting using Garak
    • Module 7: Securing AI
      • Threat Modelling: MITRE ATLAS, MAESTRO Framework
      • Defense-in-Depth for AI
      • Secure Infrastructure Architecture for AI Systems
      • Data Security in AI Systems
      • Adversarial Training and Bias Checking
      • Guardrails for LLMs: System Prompt Hardening, Input and Output Guards
      • Open-Source Guardrail Tools: LLM-Guard, Guardrails-AI, LlamaGuard
      • AI Gateway: LiteLLM
      • Monitoring and Observability for AI Systems: MLFlow, Arize Phoenix
      • Rate Limiting & Cost Budgeting
      • Data and Model Versioning
      • Access control for AI
      • Data and Model Integrity
      • AI Supply Chain Security
      • Practical Lab: Integrating guardrails with AI models
    • Module 8: AI Governance
      • AI Ethics & Properties of a Responsible AI System
      • Data Governance & Model Governance
      • NIST AI RMF (Govern/Map/Measure/Manage) and ISO 42001 Annex A Controls
      • EU AI Act risk tiers
      • Security Roles & Operating Model for AI
      • Practical Lab: Governance control mapping exercise

  Part 3 – Security Operations and Offensive Security using AI

  • Module 9: Offensive Security using AI
    • AI in the Penetration Testing Lifecycle: Reconnaissance, Scanning, Exploitation, Reporting
    • Reconnaissance using AI: OSINT Tools, AI-Generated Recon Scripts
    • Intelligent Scanning and Enumeration: Nmap with AI, AI-Generated Scan Profiles
    • AI-Enhanced Password Attacks: Wordlist Generation, Brute Force Automation
    • Web Application Pentesting with AI
    • Social Engineering with AI: AI-Generated Phishing Emails
    • Covering Tracks and Maintaining Access with AI-Generated Scripts
    • Practical Lab: AI-Assisted Reconnaissance, Network Scan, Exploitation, and Pentest Report Writing
  • Module 10: Security Operations using AI
    • Security Operations: Structure, Roles, and Core Functions
    • Signature based detection principles
    • AI Integration in Security Operations
    • Using ML+Signatures for best results
    • AI for Log Analysis: Windows Event Logs, Network Logs
    • AI for Alert Triage and False Positive Reduction
    • Threat Intelligence: IOCs, MITRE ATT&CK, AI-Assisted IOC Correlation
    • Vulnerability Management with AI: CVE Explanation, Prioritization, Remediation
    • Incident Response Lifecycle with AI-Guided Playbooks
    • User and Entity Behaviour Analytics (UEBA)
    • Malware detection using AI
    • Using AI with SIEM tools (ELK Stack)
    • Practical Lab: Analysis of IDS Logs using Generative AI and Agentic AI

  PART 1: FOUNDATIONS & BUILDING AI SYSTEMS

  • • Module 1: Introduction to AI Security Engineering
      • AI Introduction: Types of AI learning algorithms, components of AI systems and Use Cases (Predictive, Generative and Agentic)
      • Model Development Lifecycle
      • NLP and Generative AI: Understanding Tokens, Context Windows, System Prompts, and Model Responses
      • Limitations of AI: hallucination, bias, explainability gaps, non-determinism, data privacy concerns, overreliance
      • Understanding How AI Applications Are Different from Traditional Applications
      • Overview of the AI Threat Landscape: emerging attack classes unique to AI systems
      • Introduction to AI Security Engineering: what the discipline covers and how it differs from traditional AppSec/InfoSec
      • AI Application Attack Surface Overview: prompt risk, data risk, model risk, API risk, pipeline risk, infrastructure risk
      • Defense-in-Depth for AI: introducing the layered model (data, model, application/pipeline, infrastructure, governance/monitoring)
      • Responsible AI Concepts and the Difference Between AI Safety, AI Security and AI Governance
      • Practical Exercises:
        • Mapping a simple AI application architecture
        • Identifying AI assets, entry points, and data flows in a sample system
        • Identifying where prompt injection and data leakage may occur in that system
    • Module 2: ML System Architecture and Engineering
      • ML system architecture: data pipeline, feature store, training pipeline, model registry, serving/inference layer, monitoring layer
      • ML lifecycle stages: collection→ preprocessing→ training→ validation→ deployment→ monitoring→ retraining
      • Trust boundaries applied to ML systems: data, model, pipeline, infrastructure boundary
      • Python and core ML libraries overview: NumPy, Pandas, Scikit-learn, TensorFlow/Keras, NLTK
      • Common security data formats: network logs, Windows Event Logs, email headers, binary file features
      • Basic feature extraction concepts: flow statistics, text n-grams, byte histograms
      • Practical Exercises:
        • Diagram the architecture and trust boundaries of a sample ML-based security pipeline (e.g., a network intrusion detector)
        • Build a minimal scikit-learn pipeline (load→ preprocess→ train→ evaluate→ save artifact).
    • Module 3: Building ML and DL Security Controls
      • MITRE ATT&CK and the Pyramid of Pain as a guide for detector design
      • Network attack detection: flow-based feature engineering and classification
      • Phishing email detection: TF-IDF vectorisation and text classification
      • Malware detection: static file features and deep learning classification
      • User behaviour anomaly detection: UEBA concept, unsupervised anomaly scoring
      • Serving trained models as REST APIs with FastAPI
      • Experiment tracking with MLflow
      • Practical Exercises:
        • Network Attack Detection: feature engineering→ training→ evaluation→ FastAPI serving→ MLflow registration
        • Phishing Email Detection: TF-IDF classifier, evaluated and served via FastAPI
        • Malware Detection: static-feature DL classifier for malware
    • Module 4: LLM and Agentic AI Architecture and Engineering
      • LLM application components: API layer, model provider, vector database, retrieval layer, logging layer
      • Local and API-based LLM setup: OpenWebUI, Ollama, LM Studio, OpenRouter — cost and data-retention tradeoffs
      • Prompt engineering essentials: zero-shot, few-shot, chain-of-thought, system prompt design
      • RAG architecture: ingestion, chunking, embeddings, vector storage, retrieval, generation
      • Vector stores: ChromaDB, FAISS and retrieval quality as a security concern
      • Parameter-efficient fine-tuning
      • Fine-tuning vs RAG vs prompt engineering: cost, data, and latency tradeoffs
      • Agent architecture: planning, memory, tool use, external API access, action execution, feedback
      • Agentic frameworks and concepts: LangChain/LangGraph, CrewAI/A2A overview, MCP and tool exposure, context-overflow concept, markdown-native agent architecture
      • Trust boundaries for LLM/Agentic systems: model, tool, memory/context, retrieval boundary
      • Practical Exercises:
        • Set up Open WebUI with local Ollama/OpenRouter and a security-analyst system prompt
        • Fine-Tuning Walkthrough: fine-tune a small open-source model on a security dataset, compare before/after responses
        • RAG Pipeline: build a RAG pipeline over a threat-intel knowledge base
        • MCP Exploration: connect to a pre-built MCP server and observe tool exposure

  PART 2: ATTACKING AI SYSTEMS

  • • Module 5: AI Threat Modelling
      • Threat modeling fundamentals: threat actors, assets, abuse cases, entry points, data flows, security controls
      • Applying MITRE ATLAS and STRIDE to AI applications
      • Revisiting trust boundaries in AI Systems with adversarial intent in mind
      • Practical Exercises:
        • Threat-model the ML pipeline built in Module 3 and the RAG/agentic application built in Module 4- produce trust-boundary/data-flow diagrams
    • Module 6: Adversarial ML: Attacking ML and DL Systems
      • OWASP ML Security Top 10
      • Adversarial drift
      • Poisoning, evasion, extraction, model inversion, membership inference
      • Attack tooling: Adversarial Robustness Toolbox (ART), used as a black-box tool
      • Practical Exercises:
        • Data Poisoning Simulation: inject crafted samples into the training pipeline, retrain, observe targeted accuracy impact
        • Evasion Attack: run a pre-built ART evasion attack against the intrusion classifier, interpret accuracy degradation, document using ATLAS
    • Module 7: Adversarial Attacks on LLMs and Agents
      • OWASP LLM Top 10, OWASP Agentic AI Top 10, MAESTRO (threat model for agentic systems)
      • Direct and indirect prompt injection; system prompt leakage; jailbreaking techniques
      • Context-window overflow and vector store poisoning
      • Tool misuse and excessive agency, multi-agent trust exploitation
      • MCP-specific attack vectors: tool output injection, privilege escalation
      • Automated testing tools: Garak, PyRIT, Promptfoo
      • Practical Exercises:
        • Automated vulnerability scan(Garak/Promptfoo) against an AI application and review findings against OWASP LLM Top 10
        • Attack a pre-built, deliberately vulnerable agentic application: direct/indirect injection, memory poisoning; map findings to OWASP Agentic Top 10 and MAESTRO layers

  PART 3: DEFENDING AI SYSTEMS

  • • Module 8: AI Data Security
      • The AI data lifecycle: collection, labeling, preprocessing, training, validation, testing, inference, output data
      • Prompt data, retrieval data, vector database data, logs, and telemetry
      • Data classification for AI: public, internal, confidential, restricted, personal, sensitive, credential
      • Data leakage patterns: prompt leakage, file upload leakage, model output leakage, log leakage, vector database leakage
      • RAG over-retrieval risk and copilot-style oversharing risk
      • Source code and secret exposure risk in AI workflows
      • Data sanitization and de-identification: anonymization, pseudonymization, minimization, masking, redaction
      • Data lineage and provenance
      • Privacy preserving learning techniques
      • Practical Exercises:
        • PII detection and masking using Microsoft Presidio
        • Prompt sanitization techniques— build a safe AI input pipeline
        • Secrets detection using Gitleaks/TruffleHog
        • Classify sample AI data and review AI logs for sensitive data exposure
    • Module 9: AI Model Security (ML and DL)
      • Adversarial training: hardening models with adversarial examples
      • MLflow Model Registry: model versioning and model cards
      • Model artifact protection: safe serialisation vs. unsafe formats
      • Model integrity validation: hashing, signing, checksums
      • Registry protection: access control, approval tags, rollback planning
      • Practical Exercises:
        • Retrain the Module 3 classifier with adversarial examples and measure robustness improvement
        • Register the model in MLflow with a version tag, and auto-generate a model card
        • Validate the model artifact’s hash and serialisation format
    • Module 10: AI Model Security (LLMs and Agentic AI)
      • Input guardrails: prompt injection detection, PII redaction, topic filtering, length limits
      • Output guardrails: hallucination detection, sensitive content filtering, format enforcement
      • Guardrail approaches: rule-based, classifier-based, LLM-as-judge
      • Open-source guardrail tools: LLM-Guard, LlamaGuard, GuardrailsAI, NeMo Guardrails
      • LLM gateway: rate limiting, authentication, request logging, cost controls
      • Model and artifact integrity for LLMs
      • Harness engineering overview: tool sandboxing, behavioural constraints, human oversight gates
      • Practical Exercises:
        • Wrap the insecure LLM application with input/output guardrail scanners and harden system prompts
    • Module 11: AI Application & API Security
      • AI AppSec vs traditional AppSec
      • AI APIs and LLM API workflows
      • API authentication and authorization for AI applications, token handling and key protection
      • Runtime protections: Web Application Firewalls (WAFs) and inference-stage rate-limiters to mitigate DoS and malicious query floods
      • RBAC/ABAC, object-level authorization
      • Document-level authorization in RAG systems
      • Tool-level authorization in AI agents
      • Secure design for AI plugins, connectors, and MCP integrations
      • Practical Exercises:
        • Review an AI API for missing authentication/authorization (Postman/Burp Suite)
        • Identify broken access control in a RAG scenario
        • Test for API key exposure and weak token handling

  PART 4: AI SECURITY OPERATIONS

  • Module 12: Secure MLOps: CI/CD Pipeline Security
    • MLOps pipeline stages: data validation, training, evaluation, registration, deployment
    • Secure data pipeline design: Input validation and schema enforcement before training begins, dataset versioning and rollback using DVC
    • Pipeline access control and credential/secrets risk in build environments
    • CI/CD for AI: triggers on code, data, and model changes
    • Automated pass/fail security gates: SAST(Semgrep, CodeQL), secrets scanning, dependency scanning(Snyk, pip-audit)
    • Production promotion gates: performance and fairness thresholds before a model goes live
    • Practical Exercises:
      • Build a GitHub Actions pipeline combining data validation, training, evaluation, and SAST/secrets/dependency gates
      • Configure an MLflow promotion gate so a model only registers as “production” if it clears the performance/fairness threshold
      • Demonstrate a pipeline failure from a deliberately introduced issue and a clean run that promotes the model
  • Module 13: AI Infrastructure, Access Control & Supply Chain Security
    • AI infrastructure components: compute/GPU workloads, containers, model-serving endpoints, vector database storage
    • Identity and Access Management (IAM) for AI infrastructure: RBAC and least privilege for dataset modification, model deployment, and registry/tuning-parameter access
    • Environment hardening: network segmentation, GPU cluster isolation, and lateral-movement prevention
    • Container security for AI workloads
    • Open-source model and public dataset supply chain risk, third-party AI API risk
    • Model provenance verification
    • SBOM and AI-BOM concept: what they capture and why they matter for AI supply chain transparency
    • Practical Exercises:
      • Deploy an LLM gateway(LiteLLM) in front of the application, configured with rate limits, authentication, and request logging
      • Scan a container image for vulnerabilities(Trivy)
      • Using Model Provenance Toolkit
      • Generate an SBOM/AI-BOM for a sample AI app using a guided template (Syft + CycloneDX)
  • Module 14: AI Security Monitoring and Incident Response
    • ML drift monitoring: data drift vs. concept drift, reviewing an Evidently AI drift report
    • AI security monitoring overview: prompt, API usage, model query, data access, tool execution, and agent action monitoring
    • Output validation: monitoring deployed outputs for sensitive data leakage, copyright infringement, or malicious content generation
    • LLM observability tooling: Arize Phoenix and Langfuse/Langsmith for tracing calls, retrieval steps, tool invocations
    • Abuse detection: spotting attacker behaviour over time (high-volume querying, repeated extraction attempts, repeated jailbreak attempts)
    • Vulnerability intake, classification, and severity rating for AI issues
    • AI incident scenarios: prompt injection, sensitive data exposure, model extraction, poisoned dataset, unauthorized model replacement, API abuse, shadow AI
    • Incident response playbooks: quarantine, retrain, and rollback procedures for compromised models
    • Honeypots in AI Applications
    • Practical Exercises:
      • Review a pre-built Evidently AI drift report and configure a basic alert threshold
      • Analyze AI API abuse logs for extraction/jailbreak patterns
      • Building and deploying a honeypot in RAG Pipeline
  • Module 15: AI Governance and Responsible AI Practice
    • NIST AI RMF mapping exercise (Govern, Map, Measure, Manage)
    • Responsible AI properties of controls built in the course
    • Secure AI SDLC: Acquire vs Build governance questions
    • AI-BOM’s role in governance and audit evidence
    • Practical Exercises:
      • Given a sample AI system(a built combination of one ML model and one LLM/RAG application from earlier in the course), learners:
        • Review an AI API for missing authentication/authorization (Postman/Burp Suite)
        • Identify broken access control in a RAG scenario
        • Test for API key exposure and weak token handling

The training is ideal for:

• Security professionals moving into AI:
  • Security Engineers
  • Security Architects
  • Penetration Testers/Red Teamers
  • SOC Analysts
  • AppSec Engineers
  • DevSecOps Engineers
• AI and engineering professionals moving into security:
  • Data Scientists
  • ML Engineers
  • LLM Engineers
  • MLOps/Platform Engineers

Before attending this accelerated course, you should have:

• Fundamental Knowledge of Security Concepts
• No coding or AI Knowledge required

Upon successful completion of the training, participants will be able to:

• Understand AI, ML, Deep Learning, NLP, Generative AI, RAG, and Agentic AI
• Build practical ML/DL and LLM-based security solutions
• Apply AI to SOC, threat analysis, vulnerability management, and incident response
• Use AI across reconnaissance, scanning, exploitation, and pentesting workflows
• Identify and exploit vulnerabilities in ML, LLM, RAG, and agentic systems
• Conduct AI threat modeling using current security frameworks
• Implement guardrails, gateways, adversarial training, and access controls
• Secure AI applications, APIs, data, models, infrastructure, and supply chains
• Build security controls into MLOps and LLMOps pipelines
• Monitor production AI systems and respond to AI-specific incidents
• Apply NIST AI RMF, ISO/IEC 42001, and Responsible AI practices
• Produce practical governance and audit evidence for AI systems