Multi-Agent Systems
Introduction
What Question Does It Ask?
Multi-agent systems study how several autonomous agents, each deciding with its own goals and knowledge, cooperate or compete in a shared environment. One agent's best decision is not enough: what the other agents will do must be taken into account, and joint behavior is steered by communication and rules.
At a glance
interacting autonomous agents
agents, goals, protocols
joint behavior, equilibrium
social welfare, stability
medium: game theory
little; simulation
Introduction
A Short History
Nash defines the equilibrium in which each player best-responds to the others.
Smith publishes the Contract Net protocol, which allocates tasks among agents by bidding.
Reynolds introduces boids, a simulation that produces flocking from three simple rules.
Rao and Georgeff formally define the belief-desire-intention (BDI) agent architecture.
Littman proposes Markov games as the framework for multi-agent reinforcement learning.
The first RoboCup robot soccer tournament is held in Nagoya.
The Course
Core Concepts
Decides for its own goal without outside control.
Agent model that reasons with beliefs, desires and intentions.
Messages such as propose, request, inform: FIPA ACL.
Making agents' actions fit together without conflict.
Agents with different interests reaching agreement: bargaining.
Dividing work with Contract Net and auctions.
No agent wants to change its strategy unilaterally.
Global order arising from simple local rules.
The Course
A Typical Workflow
The Course
What Does the Course Cover?
- Intelligent agents and types of environment
- Deductive and reactive agent architectures
- The BDI architecture and practical reasoning
- Agent communication: speech acts, FIPA ACL
- Agent platforms: JADE, Jason
- Task sharing: Contract Net, distributed planning
- Game theory: Nash equilibrium, prisoner's dilemma
- Social choice and voting
- Auctions and mechanism design
- Bargaining and negotiation protocols
- Agent-based simulation and emergent behaviour
- Multi-agent reinforcement learning
Skills you will gain
A typical semester; the order and coverage vary between universities.
Try It
Delivery Robots Bid for Packages
Three delivery robots work on campus with no manager. Each package is auctioned; each robot bids the extra distance it would add to its route, and the cheapest wins. Run auctions step by step or click the map to add packages.
Context
Prerequisites
Courses worth knowing before this one, and why they are needed.
| Course | Why it is needed |
|---|---|
| Artificial Intelligence | the agent concept, collective decisions |
| Algorithmic Game Theory | Nash equilibrium, mechanism design and auctions |
| Object Oriented Programming with Java | objects and messaging: the language of agent platforms such as JADE |
| Algorithms | graph algorithms and complexity of distributed problems |
Context
Are You Ready for This Course?
Ask yourself; for every "no", the right-hand column shows where to start.
| Ask yourself | If not, start with |
|---|---|
| Can you find the Nash equilibrium in a two-player payoff matrix? | The opening chapters of an algorithmic game theory course |
| Can you explain a single agent that searches a state space? | Artificial Intelligence |
| Can you write classes and have objects send messages via method calls? | An intro to object-oriented programming in Java |
| Can you write an algorithm that finds the shortest path in a graph? | Graph algorithms from an algorithms course |
Context
Neighbouring Courses
Overlap
Overlapping Courses
| Relation | Course | Shared ground |
|---|---|---|
| subfield | This course is a subfield of Artificial Intelligence | the agent concept, collective decisions |
| shares methods | Shares methods with Reinforcement Learning | multi-agent reinforcement learning, game theory |
| shares methods | Shares methods with Robotics and Robot Vision | swarm robotics, coordination |
Differences
Where It Differs (1/2)
| Criterion | Multi-Agent Systems | Artificial Intelligence | Reinforcement Learning |
|---|---|---|---|
| Core question | How do several autonomous agents decide together or against each other? | How does a machine act intelligently by searching, reasoning and planning? | How can good decisions be learned by trial and error from a reward signal alone? |
| Approach | interacting autonomous agents | search, logic, probability and learning combined | trial and error with rewards |
| Input | agents, goals, protocols | problem definition: states, actions, goal | environment: state, reward |
| Output | joint behavior, equilibrium | action sequence, plan or decision | policy: state → action |
| Success measure | social welfare, stability | reaching the goal, cost, rationality | total expected reward |
| Math load | medium: game theory | medium: logic, probability, graphs | high: probability, dynamic programming |
| Data needs | little; simulation | little; often a model and knowledge suffice | generated through interaction |
- Artificial Intelligence: AI seeks one agent's best decision against its environment; this course studies joint behavior where other agents also decide.
- Reinforcement Learning: RL studies one agent learning from reward; this course also covers communication, negotiation and coordination, with or without learning.
Differences
Where It Differs (2/2)
| Criterion | Multi-Agent Systems | Robotics | Evolutionary Optimization |
|---|---|---|---|
| Core question | How do several autonomous agents decide together or against each other? | How does a machine sense the physical world, know where it is and move? | How do we find a good solution when there is no gradient or the search space is huge? |
| Approach | interacting autonomous agents | sense-plan-act loop | population-based heuristic search |
| Input | agents, goals, protocols | sensor measurements | objective, solution encoding |
| Output | joint behavior, equilibrium | motor commands, path | good, not proven optimal, solution |
| Success measure | social welfare, stability | task success, safety | quality, evaluation count |
| Math load | medium: game theory | high: transformations, control | low-medium |
| Data needs | little; simulation | physical experiments and simulation | none; evaluations suffice |
- Robotics: Robotics solves sensing and motion for one machine; this course deals with dividing work among many robots or software agents.
- Evolutionary Optimization: Swarm intelligence algorithms use a swarm as a tool to solve a problem; here the agents and their interaction are the subject.
Practice
Applications
Sharing tasks and paths among hundreds of robots.
Energy trading between producers and consumers.
Coordinating intersections and vehicles.
Online bidding and ad auctions.
Computer opponents that play as a team.
Epidemic, market and crowd behavior.
Practice
Risks and Ethics
Agents that each work correctly can together produce surprises: in the 2010 Flash Crash, automated trading sped up the fall.
Pricing agents that learn can, without any agreement, learn to punish each other and settle on prices above the competitive level.
An agent that finds a loophole in an auction or voting rule can bid or vote untruthfully and tilt the outcome its way.
Results of agent-based epidemic or market simulations depend on the assumed rules and should not go straight into policy.
In depth: Explainable AI and AI Ethics.
Practice
Tools and Datasets
Tools and libraries
Classic benchmark problems
Practice
Common Misconceptions
| Common belief | In fact |
|---|---|
| A multi-agent system is a multi-core program. | In a parallel program the parts serve one goal; agents are autonomous and their goals may conflict. |
| If each agent does its best, the whole is best. | As in the prisoner's dilemma, individually rational choices can lead to an outcome bad for everyone. |
| Coordination needs a central manager. | Swarms and auction protocols achieve decentralized coordination through local rules and messages. |
| Agents always cooperate. | Agents can also be rivals; the course studies cooperation, competition and mixtures of both. |
Next Steps
After This Course
Courses that build on or extend this one
Related careers
Next Steps
Further Reading
- Wooldridge, An Introduction to MultiAgent Systems, 2nd ed., Wiley, 2009.
- Shoham and Leyton-Brown, Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations, Cambridge University Press, 2009.
- Weiss (ed.), Multiagent Systems, 2nd ed., MIT Press, 2013.
- Albrecht, Christianos and Schäfer, Multi-Agent Reinforcement Learning: Foundations and Modern Approaches, MIT Press, 2024.