AI in Cybersecurity
Introduction
What Question Does It Ask?
AI in cybersecurity asks two questions together: how can machine learning catch intrusions, malware and phishing, and how can AI systems themselves be fooled? What sets the field apart is the adversarial setting: on the other side is an attacker trying to evade the model.
At a glance
security knowledge plus ML
network traffic, logs, files
alert, class, risk score
detection rate, false alarm rate
medium
imbalanced and constantly changing
Introduction
A Short History
Denning defines intrusion detection as a model that catches abnormal behaviour.
The KDD Cup 1999 intrusion detection dataset is released; it serves as a benchmark for many years.
Paul Graham popularizes the Bayesian spam filter with "A Plan for Spam".
Szegedy et al. show adversarial examples: an invisible change fools a classifier.
DARPA Cyber Grand Challenge: machines find and patch vulnerabilities with no human help.
OWASP publishes its first Top 10 for LLM applications, led by prompt injection.
The Course
Core Concepts
Signature-based and anomaly-based approaches.
Classifying files by static or behavioural features.
Telling fake emails, domains and web pages apart.
Attacks are very rare next to normal traffic.
Precision, recall; analyst fatigue.
Inputs carefully perturbed to fool a model.
Slipping malicious samples into training data.
Hidden instructions that change a language model's goal.
The Course
A Typical Workflow
The Course
What Does the Course Cover?
- Threat models and the limits of ML in security
- Feature extraction from network traffic and logs
- Signature- and anomaly-based intrusion detection
- Imbalanced data, false alarms and suitable metrics
- Malware: static and behavioural analysis
- Phishing, spam and malicious URL detection
- Fraud and account takeover detection
- Concept drift and attacker adaptation
- Adversarial examples and evasion attacks
- Data poisoning and backdoors
- Model stealing, membership inference, privacy
- Language model security: prompt injection
Skills you will gain
A typical semester; the order and coverage vary between universities.
Try It
Catch Suspicious Logins
Password-guessing attacks are mixed in with normal logins in the company's login records. The model gives each login an anomaly score. Move the threshold or click the chart; what happens when the attacker adapts?
Context
Prerequisites
Courses worth knowing before this one, and why they are needed.
| Course | Why it is needed |
|---|---|
| Machine Learning | intrusion detection, malware classification |
| Operating Systems | processes, file systems and permissions: the layer attacks target |
| Computer Networks | TCP/IP, ports and packets: being able to read network traffic |
| Programming with C | memory and pointers: seeing how overflow vulnerabilities arise |
Context
Are You Ready for This Course?
Ask yourself; for every "no", the right-hand column shows where to start.
| Ask yourself | If not, start with |
|---|---|
| Can you interpret precision and recall on an imbalanced dataset? | Machine Learning |
| Can you explain how a TCP connection is set up and what a port is? | TCP/IP from a computer networks course |
| Can you inspect processes, file permissions and system logs on Linux? | An operating systems course and the Linux shell |
| Can you explain how a buffer overflow happens in memory? | Pointers and arrays from a C programming course |
Context
Neighbouring Courses
Overlap
Overlapping Courses
| Relation | Course | Shared ground |
|---|---|---|
| application | This course applies the methods of Machine Learning | intrusion detection, malware classification |
| shares methods | Shares methods with Explainable AI and AI Ethics | robustness, adversarial examples |
| shares methods | Shares methods with Biometric Systems | authentication, spoof detection |
Differences
Where It Differs (1/2)
| Criterion | AI in Cybersecurity | Machine Learning | Explainable AI & Ethics |
|---|---|---|---|
| Core question | How does AI catch attacks, and how is it attacked itself? | How does a program generalize from examples? | When can we trust an AI system's decision, and who is responsible for the outcome? |
| Approach | security knowledge plus ML | statistical learning | human-centered evaluation |
| Input | network traffic, logs, files | labeled or unlabeled examples | model, its data and context of use |
| Output | alert, class, risk score | a model that makes predictions | explanation, audit report, principles |
| Success measure | detection rate, false alarm rate | error on unseen data | interpretability, fairness, accountability |
| Math load | medium | high: linear algebra, probability, optimization | low-medium |
| Data needs | imbalanced and constantly changing | moderate to large | the model's own data |
- Machine Learning: ML assumes the data distribution stays fixed; here the other side deliberately changes the input to evade the model.
- Explainable AI & Ethics: Explainable AI treats trustworthiness in general; this course focuses on robustness against attackers and on catching attacks.
Differences
Where It Differs (2/2)
| Criterion | AI in Cybersecurity | Biometrics | Data Mining |
|---|---|---|---|
| Core question | How does AI catch attacks, and how is it attacked itself? | How is a person recognized from their face, fingerprint or voice? | How are hidden, useful patterns found in large piles of data? |
| Approach | security knowledge plus ML | matching and verification | discovery-driven analysis |
| Input | network traffic, logs, files | face, fingerprint, iris, voice | large tables or transaction data |
| Output | alert, class, risk score | identity decision | rules, clusters, outliers |
| Success measure | detection rate, false alarm rate | FAR, FRR, EER | interestingness, support, confidence |
| Math load | medium | medium | medium |
| Data needs | imbalanced and constantly changing | few samples per person | large |
- Biometrics: Biometrics verifies identity by recognizing a person; this course covers a wide attack surface, from networks and files to email and models.
- Data Mining: Data mining looks for unusual patterns to discover them; this course interprets them as attacks and turns them into alerts.
Practice
Applications
Finding unusual connections in network traffic.
Judging whether a file is malicious without running it.
Filtering out fake emails and web pages.
Detecting suspicious transactions and account takeover.
Prioritizing and triaging thousands of alerts.
Adversarial examples, poisoning, prompt injection.
Practice
Risks and Ethics
Techniques learned for detection can also produce malware that evades it, or convincing phishing emails.
Scanning another party's network without permission is a crime; experiments belong in an isolated lab, with written consent.
Insider threat detection watches employees' email and behaviour; proportionality, GDPR and similar laws set limits.
Cutting a connection or locking an account on the model's word alone stops legitimate users on a false alarm.
In depth: Explainable AI and AI Ethics.
Practice
Tools and Datasets
Tools and libraries
Well-known datasets
Practice
Common Misconceptions
| Common belief | In fact |
|---|---|
| 99% accuracy means good intrusion detection. | If attacks are very rare, a model that calls everything normal is also 99% accurate; look at precision and false alarm rate. |
| Once trained, a model keeps catching attacks. | Attackers change tactics; the model must be updated continuously with new samples. |
| AI replaces the security analyst. | AI filters and prioritizes alerts; decisions and investigation still rest with people. |
| Deep networks are robust to attacks. | Small, invisible changes (adversarial examples) can easily flip a model's decision. |
Next Steps
After This Course
Courses that build on or extend this one
Related careers
Next Steps
Further Reading
- Chio and Freeman, Machine Learning and Security, O'Reilly, 2018.
- Denning, "An Intrusion-Detection Model", IEEE Transactions on Software Engineering 13(2), 1987.
- Sommer and Paxson, "Outside the Closed World: On Using Machine Learning for Network Intrusion Detection", IEEE Symposium on Security and Privacy, 2010.
- Goodfellow, Shlens and Szegedy, "Explaining and Harnessing Adversarial Examples", ICLR, 2015.