Multi-Agent Systems

Guide to AI and Data Courses · Part II: Branches of AI
Autonomous agentsCommunicationCoordinationNegotiation and auctionsGame theoryNeighboring courses
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

What Question Does It Ask?

How do several autonomous agents decide together or against each other?

Multi-agent systems study how several autonomous agents, each deciding with its own goals and knowledge, cooperate or compete in a shared environment. One agent's best decision is not enough: what the other agents will do must be taken into account, and joint behavior is steered by communication and rules.

At a glance

Approach

interacting autonomous agents

Input

agents, goals, protocols

Output

joint behavior, equilibrium

Success measure

social welfare, stability

Math load

medium: game theory

Data needs

little; simulation

Introduction

A Short History

1950

Nash defines the equilibrium in which each player best-responds to the others.

1980

Smith publishes the Contract Net protocol, which allocates tasks among agents by bidding.

1987

Reynolds introduces boids, a simulation that produces flocking from three simple rules.

1991

Rao and Georgeff formally define the belief-desire-intention (BDI) agent architecture.

1994

Littman proposes Markov games as the framework for multi-agent reinforcement learning.

1997

The first RoboCup robot soccer tournament is held in Nagoya.

The Course

Core Concepts

Autonomous agent

Decides for its own goal without outside control.

BDI architecture

Agent model that reasons with beliefs, desires and intentions.

Communication language

Messages such as propose, request, inform: FIPA ACL.

Coordination

Making agents' actions fit together without conflict.

Negotiation

Agents with different interests reaching agreement: bargaining.

Auctions and task allocation

Dividing work with Contract Net and auctions.

Nash equilibrium

No agent wants to change its strategy unilaterally.

Emergent behavior

Global order arising from simple local rules.

The Course

A Typical Workflow

Perceiveenvironment,other agentsReasonbeliefs,goalsCommunicatemessages,bidsCoordinatenegotiation,auctionsActjoint outcome
Example: A hundred robots pick orders in a warehouse. When a new order arrives, each robot bids based on its position and the best bid wins the job; robots announce their paths to each other and pass through aisles without colliding. There is no all-knowing central planner.

The Course

What Does the Course Cover?

  1. Intelligent agents and types of environment
  2. Deductive and reactive agent architectures
  3. The BDI architecture and practical reasoning
  4. Agent communication: speech acts, FIPA ACL
  5. Agent platforms: JADE, Jason
  6. Task sharing: Contract Net, distributed planning
  7. Game theory: Nash equilibrium, prisoner's dilemma
  8. Social choice and voting
  9. Auctions and mechanism design
  10. Bargaining and negotiation protocols
  11. Agent-based simulation and emergent behaviour
  12. Multi-agent reinforcement learning

Skills you will gain

writing agents and messaging in JADEsimulation with NetLogo or Mesalearning agents in PettingZoofinding Nash equilibria in small gamesdesigning auction and bargaining protocols

A typical semester; the order and coverage vary between universities.

Try It

Delivery Robots Bid for Packages

Three delivery robots work on campus with no manager. Each package is auctioned; each robot bids the extra distance it would add to its route, and the cheapest wins. Run auctions step by step or click the map to add packages.

The problem was solved not by one program but by negotiating agents: in the contract net a manager announces a task and agents bid according to their own cost. Adding a robot does not break a central plan.

Context

Prerequisites

Courses worth knowing before this one, and why they are needed.

CourseWhy it is needed
Artificial Intelligencethe agent concept, collective decisions
Algorithmic Game TheoryNash equilibrium, mechanism design and auctions
Object Oriented Programming with Javaobjects and messaging: the language of agent platforms such as JADE
Algorithmsgraph algorithms and complexity of distributed problems

Context

Are You Ready for This Course?

Ask yourself; for every "no", the right-hand column shows where to start.

Ask yourselfIf not, start with
Can you find the Nash equilibrium in a two-player payoff matrix?The opening chapters of an algorithmic game theory course
Can you explain a single agent that searches a state space?Artificial Intelligence
Can you write classes and have objects send messages via method calls?An intro to object-oriented programming in Java
Can you write an algorithm that finds the shortest path in a graph?Graph algorithms from an algorithms course
If you can say "yes" to most of these, you are ready; one or two gaps can be closed during the semester.

Context

Neighbouring Courses

subfieldbuilds onshares methodsapplication

Overlap

Overlapping Courses

RelationCourseShared ground
subfieldThis course is a subfield of Artificial Intelligencethe agent concept, collective decisions
shares methodsShares methods with Reinforcement Learningmulti-agent reinforcement learning, game theory
shares methodsShares methods with Robotics and Robot Visionswarm robotics, coordination

Differences

Where It Differs (1/2)

CriterionMulti-Agent SystemsArtificial IntelligenceReinforcement Learning
Core questionHow do several autonomous agents decide together or against each other?How does a machine act intelligently by searching, reasoning and planning?How can good decisions be learned by trial and error from a reward signal alone?
Approachinteracting autonomous agentssearch, logic, probability and learning combinedtrial and error with rewards
Inputagents, goals, protocolsproblem definition: states, actions, goalenvironment: state, reward
Outputjoint behavior, equilibriumaction sequence, plan or decisionpolicy: state → action
Success measuresocial welfare, stabilityreaching the goal, cost, rationalitytotal expected reward
Math loadmedium: game theorymedium: logic, probability, graphshigh: probability, dynamic programming
Data needslittle; simulationlittle; often a model and knowledge sufficegenerated through interaction
  • Artificial Intelligence: AI seeks one agent's best decision against its environment; this course studies joint behavior where other agents also decide.
  • Reinforcement Learning: RL studies one agent learning from reward; this course also covers communication, negotiation and coordination, with or without learning.

Differences

Where It Differs (2/2)

CriterionMulti-Agent SystemsRoboticsEvolutionary Optimization
Core questionHow do several autonomous agents decide together or against each other?How does a machine sense the physical world, know where it is and move?How do we find a good solution when there is no gradient or the search space is huge?
Approachinteracting autonomous agentssense-plan-act looppopulation-based heuristic search
Inputagents, goals, protocolssensor measurementsobjective, solution encoding
Outputjoint behavior, equilibriummotor commands, pathgood, not proven optimal, solution
Success measuresocial welfare, stabilitytask success, safetyquality, evaluation count
Math loadmedium: game theoryhigh: transformations, controllow-medium
Data needslittle; simulationphysical experiments and simulationnone; evaluations suffice
  • Robotics: Robotics solves sensing and motion for one machine; this course deals with dividing work among many robots or software agents.
  • Evolutionary Optimization: Swarm intelligence algorithms use a swarm as a tool to solve a problem; here the agents and their interaction are the subject.

Practice

Applications

Warehouse robots

Sharing tasks and paths among hundreds of robots.

Smart grid

Energy trading between producers and consumers.

Traffic management

Coordinating intersections and vehicles.

Automated negotiation

Online bidding and ad auctions.

Games

Computer opponents that play as a team.

Social simulation

Epidemic, market and crowd behavior.

Practice

Risks and Ethics

Unforeseen collective behaviour

Agents that each work correctly can together produce surprises: in the 2010 Flash Crash, automated trading sped up the fall.

Collusion without talking

Pricing agents that learn can, without any agreement, learn to punish each other and settle on prices above the competitive level.

Gaming the mechanism

An agent that finds a loophole in an auction or voting rule can bid or vote untruthfully and tilt the outcome its way.

Mistaking simulation for reality

Results of agent-based epidemic or market simulations depend on the assumed rules and should not go straight into policy.

In depth: Explainable AI and AI Ethics.

Practice

Tools and Datasets

Tools and libraries

JADENetLogoMesaPettingZooJason (AgentSpeak)RepastMASON

Classic benchmark problems

Prisoner's dilemmaPredator-prey problemRoboCup soccerStarCraft Multi-Agent Challenge (SMAC)Schelling segregation modelFlocking (boids) simulation
In this course data is mostly not collected but generated by simulation: agents' rules are written and joint behavior is observed by running them.

Practice

Common Misconceptions

Common beliefIn fact
A multi-agent system is a multi-core program.In a parallel program the parts serve one goal; agents are autonomous and their goals may conflict.
If each agent does its best, the whole is best.As in the prisoner's dilemma, individually rational choices can lead to an outcome bad for everyone.
Coordination needs a central manager.Swarms and auction protocols achieve decentralized coordination through local rules and messages.
Agents always cooperate.Agents can also be rivals; the course studies cooperation, competition and mixtures of both.

Next Steps

After This Course

Courses that build on or extend this one

This course is mostly an end point: it builds on the others.

Related careers

Robotics software engineerSimulation and modeling specialistGame AI developerLogistics systems engineerResearcher
Multi-agent systems extend AI with a social view. After this course you can move to multi-agent reinforcement learning for learning agents, swarm robotics for physical teams, or algorithmic game theory for designing incentives and rules.

Next Steps

Further Reading

  • Wooldridge, An Introduction to MultiAgent Systems, 2nd ed., Wiley, 2009.
  • Shoham and Leyton-Brown, Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations, Cambridge University Press, 2009.
  • Weiss (ed.), Multiagent Systems, 2nd ed., MIT Press, 2013.
  • Albrecht, Christianos and Schäfer, Multi-Agent Reinforcement Learning: Foundations and Modern Approaches, MIT Press, 2024.

The End

Chapter 14: Multi-Agent Systems
← Previous chapter · Map of the field · Next chapter →
1 / 1 Sercan KÜLCÜ. All rights reserved.