Chapter 1: Decision Theory
Expected Utility
Warm-up Game: Bayesian Decision Lab
You know the prior of a hidden state, see a noisy signal and choose one of two actions; then the state is revealed and you receive the payoff in the table.
- A forecast that is right 90% of the time says "rain". What is the probability of rain?
- If a decision leads to a bad outcome, was the decision bad?
Expected Utility
Learning Outcomes
Expected Utility
Decision Making: Three Kinds
Decision making is the process of choosing among options by evaluating their consequences. Three kinds, by what drives the outcome:
| Kind | Outcome driven by | Tools | Example |
|---|---|---|---|
| Individual, under risk | Nature / chance (probabilities known or estimated) | Expected utility, Bayes, decision trees | Investment, insurance, medical tests |
| Strategic | Choices of other rational agents | Best response, Nash equilibrium | Price competition, bargaining, sports |
| Collective | Joint choice and interaction of a group | Voting, common goods | Policy, corporate strategy |
Expected Utility
Rational Choice: Expected Value or Expected Utility?
- Lottery: a set of outcomes with probabilities. A = 900 TL with probability 50%, 0 TL with probability 50%.
- Expected value: E[X] = Σ pᵢ xᵢ. E[A] = ½·900 + ½·0 = 450 TL > a sure B = 400 TL.
- Expected utility (von Neumann–Morgenstern): E[u] = Σ pᵢ u(xᵢ). A rational decision maker maximizes expected utility.
- With u(x) = √x: E[u(A)] = ½·30 + ½·0 = 15; u(B) = √400 = 20 ⇒ B is chosen.
Red: each criterion's choice. Expected value picks A, expected utility (√x) picks B.
St. Petersburg paradox: nobody pays much for a game with infinite expected value; this is why Daniel Bernoulli (1738) proposed utility (a similar idea: Cramer, 1728).
Expected Utility
Foundations of Expected Utility and the Allais Paradox
- von Neumann–Morgenstern: if preferences satisfy four axioms, some u exists and the decision maker maximizes expected utility. A ≻ B: A preferred to B.
- Completeness: any two options are comparable. Transitivity: A ≻ B and B ≻ C imply A ≻ C. Continuity: a middle option equals some mix of the extremes.
- Independence: mixing the same component (same probability) into two lotteries keeps their ranking.
- Allais (1953): most people pick A in choice 1, D in choice 2.
| Option | 1 million | 5 million | 0 | |
|---|---|---|---|---|
| Choice 1 | A | 100% | — | — |
| B | 89% | 10% | 1% | |
| Choice 2 | C | 11% | — | 89% |
| D | — | 10% | 90% |
Swap the common 89% "1 million" in A and B for 89% "0": C and D.
Expected Utility
What Do Utility Numbers Measure? Uniqueness of the Scale
- Utility is like a thermometer: its zero and its unit are arbitrary. Celsius and Fahrenheit rank temperatures the same way.
- Uniqueness (vNM): a transformation v = a·u + b (a > 0) is called an affine transformation: it rescales and shifts, but changes no decision between lotteries. Other (non-affine) transformations can change decisions.
- Link to game theory: multiplying all of a player's payoffs by 2 and adding 5 changes none of its decisions.
| Utility | lottery A | sure B | choice |
|---|---|---|---|
| u = √x | 15 | 20 | B |
| v = 10√x + 5 | 155 | 205 | B |
| w = (√x)² = x | 450 | 400 | A |
A: 900 TL with probability ½, 0 TL with probability ½; B: 400 TL for sure. The affine v preserves the decision; the increasing but non-affine w reverses it.
Risk Attitudes
Certainty Equivalent and Risk Premium, Step by Step
Concave utility u(x) = √x: each extra lira adds less utility than the one before (diminishing marginal utility). Endpoints: u(0) = 0, u(900) = 30.
Chord: the line joining the two ends of the curve. Lottery A's expected utility is the chord's midpoint: E[u] = ½·0 + ½·30 = 15 (at x = 450).
The same 450 TL for sure gives u(450) ≈ 21.2 > 15. On a concave curve the chord lies below the curve, hence E[u(X)] < u(E[X]) (Jensen's inequality).
Certainty equivalent (CE): the sure amount with utility 15. √x = 15 ⇒ CE = 225 TL. This person would sell the lottery for 225 TL.
Risk premium = E[X] − CE = 450 − 225 = 225 TL. A sure 400 TL has utility 20 > 15 ⇒ a risk-averse person chooses B.
Risk Attitudes
Three Risk Attitudes: Same Lottery, Different Decisions
| Attitude | Utility | E[u(A)] | u(400) | Choice | CE (TL) | Risk premium |
|---|---|---|---|---|---|---|
| Risk-averse | u(x) = √x (concave) | 15 | 20 | B | 225 | 225 |
| Risk-neutral | u(x) = x (linear) | 450 | 400 | A | 450 | 0 |
| Risk-seeking | u(x) = x²/900 (convex) | 450 | 177.8 | A | 636.4 | −186.4 |
Lottery A: 900 TL with probability 50%, 0 TL with probability 50% (E[X] = 450). B: 400 TL for sure. CE = u⁻¹(E[u(A)]): u⁻¹ inverts u; for √x, u⁻¹(15) = 15² = 225. Risk premium = E[X] − CE.
Risk Attitudes
Application: Why Does Insurance Exist?
- Wealth 900 (thousand TL); a loss of 800 (fire) with probability 10%. Utility u(x) = √x.
- Uninsured: E[u] = 0.9·√900 + 0.1·√100 = 27 + 1 = 28.
- Certainty equivalent: 28² = 784. The person accepts a sure 784 in place of a risky 900.
- Maximum premium = 900 − 784 = 116; the expected loss is only 0.1·800 = 80.
Insured: paying a premium ≤ 116 to shed the risk raises utility. Insurer: with many independent policies it is nearly risk-neutral; any premium above 80 is profit.
Risk Attitudes
Known Risk or Unknown Probabilities? The Ellsberg Urn
- Under risk the probabilities are known (dice, roulette); under uncertainty they are not. People treat the two differently.
- Urn: 90 balls; 30 are red, the other 60 are black or yellow, but how many are black is unknown. Drawing the winning color pays 100 TL.
- Typical choices: "red" in bet 1, "black or yellow" in bet 2; in both, the option with a known winning probability.
| Bet | Option | winning balls |
|---|---|---|
| 1 | red | 30 (known) |
| black | b (0–60, unknown) | |
| 2 | red or yellow | 30 + (60 − b) |
| black or yellow | 60 (known) |
b: number of black balls in the urn. Orange: the majority choice.
Risk Attitudes
Unknown Probabilities: Decisions under Uncertainty
Probabilities are unknown (uncertainty). Rows are actions, columns are states; each criterion reflects a different degree of "pessimism".
Maximin (Wald, plan for the worst): row minima −120, −20, 0 → the largest is 0: No investment.
Maximax (optimistic): row maxima 200, 90, 0 → Big investment.
Laplace (treat states as equally likely): averages 43.3, 36.7, 0.0 → Big investment.
Best achievable result in each state: best = [200, 50, 0]. Regret = best − achieved.
Largest regret per row: Big (bad state, 0 − (−120)) = 120, Small (good state, 200 − 90) = 110, None (good state, 200) = 200.
Savage (minimax regret): the smallest maximum regret is 110 → Small investment.
return (2, 0, 0, 1). Returned indices (0: Big, 1: Small, 2: None): maximin → None, maximax and Laplace → Big, Savage → Small. Three different decisions from one table.
One more criterion, not in the code: Hurwicz sits between the extremes: α·(row max) + (1 − α)·(row min); with optimism coefficient α = 0.5: 40, 35, 0 → Big. If probabilities can be estimated, return to expected utility; if not, the choice of criterion reflects the decision maker's attitude.
def criteria(U): A, m = range(len(U)), len(U[0]) mm = max(A, key=lambda i: min(U[i])) # maximin mx = max(A, key=lambda i: max(U[i])) # maximax lp = max(A, key=lambda i: sum(U[i]) / m) # Laplace best = [max(r[j] for r in U) for j in range(m)] reg = [max(best[j] - r[j] for j in range(m)) for r in U] # max regret sv = reg.index(min(reg)) # Savage return mm, mx, lp, sv
Bayesian Decisions
Medical Test: What Does a Positive Result Mean?
Picture a population of 10,000. Using natural frequencies (head counts) instead of probabilities makes Bayesian reasoning intuitive.
Prior: prevalence 1% ⇒ 100 ill, 9,900 healthy.
Sensitivity 90%: 90 of the 100 ill people test positive, 10 negative.
False positive rate 9%: 891 of the 9,900 healthy people also test positive! Healthy people are so numerous that this number is large.
Everyone testing positive: 90 + 891 = 981 people. Only 90 of them are actually ill.
Posterior: P(ill | +) = 90 / 981 = 9.2%. Even with 90% sensitivity, a positive result is mostly a false alarm (base-rate neglect).
Bayesian Decisions
Code Trace: bayes(p, sens, fp, tests)
Call: bayes(0.01, 0.9, 0.09, ["+", "+", "-"]). p starts at the prior (1%); the three test results are processed in turn.
Test 1 result: +. The current belief p = 0.0100 is this test's prior.
res == "+" ? → True: use the likelihoods of a positive result. Likelihood: the probability of this result given the disease status, e.g. P(+ | ill) = 0.9.
num = P(+|ill)·p = 0.9 × 0.0100 = 0.0090; den = num + P(+|healthy)·(1 − p) = 0.0981 (total probability of a positive).
p = num / den = 0.0917, printed. Despite the positive test, the probability of disease is still below 10%.
Test 2 result: +. The current belief p = 0.0917 is this test's prior.
res == "+" ? → True: use the likelihoods of a positive result.
num = P(+|ill)·p = 0.9 × 0.0917 = 0.0826; den = num + P(+|healthy)·(1 − p) = 0.1643 (total probability of a positive).
p = num / den = 0.5025, printed. Two positives, independent given the disease status, raised the probability to ~50%.
Test 3 result: -. The current belief p = 0.5025 is this test's prior.
res == "+" ? → False: use the likelihoods of a negative result.
num = P(−|ill)·p = 0.1 × 0.5025 = 0.0503; den = num + P(−|healthy)·(1 − p) = 0.91 × 0.4975 + num = 0.5030.
p = num / den = 0.0999, printed. The negative result brought the belief back down to 10%.
The loop ends; the function returns p = 0.0999. Each test uses the previous posterior as its prior (sequential updating).
def bayes(p, sens, fp, tests): for res in tests: if res == "+": num = sens * p den = num + fp * (1 - p) elif res == "-": num = (1 - sens) * p den = num + (1 - fp) * (1 - p) else: raise ValueError("result must be + or -") if den == 0: raise ValueError("impossible result") p = num / den print(res, round(p, 4)) return p
Bayesian Decisions
Bayesian Decision Rule: Treat or Wait?
- Update the prior with the evidence; then compute each action's expected loss under the posterior. Choose the action with the lowest expected loss.
- Losses: needless treatment of a healthy patient 30, not treating an ill one 200.
- E[loss | treat] = (1 − p)·30; E[loss | wait] = p·200. Treat if it loses less: (1 − p)·30 < p·200 ⇒ 30 < 230p ⇒ p > 30/230 = 3/23 ≈ 13.0%.
| State | p | treat | wait | decision |
|---|---|---|---|---|
| 1 positive | 0.0917 | 27.2 | 18.3 | wait, retest |
| 2 positives | 0.5025 | 14.9 | 100.5 | treat |
Expected losses (orange = smallest)
Bayesian Decisions
Back to the Game: Bayesian Decision Lab
P(rain | "rain") = 0.9·0.35 / (0.9·0.35 + 0.1·0.65) = 0.315 / 0.38 ≈ 0.829.
| Information | P(rain) | EU(out) | EU(in) | Decision |
|---|---|---|---|---|
| prior only | 0.35 | 3.10 | 3.35 | indoors |
| "rain" | 0.829 | −3.61 | 3.83 | indoors |
| "dry" | 0.056 | 7.21 | 3.06 | outdoors |
EU: expected utility. Outdoors is chosen only when P(rain) < 1/3. At Level 3 (78% accuracy) the signal's value of information is 1.765; it exceeds the signal's price (1), so buying the signal pays off.
In a classic study, physicians and medical students were asked about a test with prevalence 1/1000 and a 5% false positive rate: only 11 of 60 reached the right answer (about 2%); the most common answer was 95% (Casscells, Schoenberger and Graboys, 1978).
The game's "EU-optimal" counter counts not lucky outcomes but decisions that maximize expected utility given the information at hand: a good decision can lead to a bad outcome.
Bayesian Decisions
Bridge to Bayesian Games: Updating the Opponent's Type
- With incomplete information the unknown "state" is the opponent's type: the belief is the prior, the move is the evidence.
- Example (poker): the opponent is a bluffer with probability 20%, honest with 80%. A bluffer raises 90% of the time whatever the hand; an honest player raises only with a strong hand (30%).
- The opponent raised: P(bluffer | raise) = 0.2·0.9 / (0.2·0.9 + 0.8·0.3) = 0.18 / 0.42 ≈ 0.429.
- Calling earns +30 against a bluffer, −30 against an honest player: E[call] = 0.429·30 − 0.571·30 ≈ −4.3 < 0 ⇒ fold. After a second raise the posterior is 0.692.
| Observation | P(bluffer) | E[call] | decision |
|---|---|---|---|
| prior only | 0.200 | −18.0 | fold |
| 1 raise | 0.429 | −4.3 | fold |
| 2 raises | 0.692 | +11.5 | call |
Calling pays only if P(bluffer) > 0.5 (equal gain and loss).
Decision Trees and the Value of Information
Decision Tree: Product Launch and Market Research
- A firm must decide whether to launch a new product. If the market is good (probability 2/5) it earns +500, if bad (3/5) −200 thousand TL; no launch gives 0.
- It can first buy market research for 20 thousand TL. The report is imperfect: P(positive | good) = 0.8, P(positive | bad) = 0.1.
- By Bayes: P(positive) = 0.4·0.8 + 0.6·0.1 = 0.38; P(good | positive) = 0.32/0.38 = 16/19; P(good | negative) = 0.08/0.62 = 4/31.
Decision Trees and the Value of Information
Backward Induction in a Decision Tree
Square = decision node, circle = chance node. Leaves show net payoffs (20 deducted if research is bought). The two root branches: Research and Direct (decide without research). Backward induction: expected value at chance nodes, maximum at decision nodes.
Launching without research: 2/5·500 + 3/5·(−200) = 200 − 120 = 80.
Decision: max(Launch 80, Skip 0) = 80 ⇒ Launch. The Skip branch is pruned.
After a positive report, P(Good | positive) = 0.32/0.38 = 16/19 (Bayes). Expected: 16/19·480 + 3/19·(−220) ≈ 369.5.
Decision: max(369.5, −20) ⇒ Launch after a positive report.
After a negative report, P(Good | negative) = 0.08/0.62 = 4/31. Expected: 4/31·480 + 27/31·(−220) ≈ −129.7.
Decision: max(−129.7, −20) = −20 ⇒ Skip after a negative report (only the research cost is lost).
Report chance node: 0.38·369.5 + 0.62·(−20) = 128.
Root: max(Research 128, Direct 80) ⇒ Research. Strategy: do research; launch if positive, skip if negative.
Decision Trees and the Value of Information
Code Trace: Recursive Backward Induction
value(root) is called. The tree is nested tuples: root = ("D", [("Research", ("C", [(0.38, …), (0.62, …)])), ("Direct", ("D", […]))]); "D" is a decision node, "C" a chance node, a leaf is a number.
value(Research) is called: a chance node; its children are evaluated in turn.
value(positive) is called: a decision node; its children are evaluated in turn.
value(positive·Launch) is called: a chance node; its children are evaluated in turn.
positive·Launch (chance): 0.842·480 + 0.158·−220 = 369.5.
positive (decision): max(Launch 369.5, Skip −20.0) = 369.5 → Launch.
value(negative) is called: a decision node; its children are evaluated in turn.
value(negative·Launch) is called: a chance node; its children are evaluated in turn.
negative·Launch (chance): 0.129·480 + 0.871·−220 = −129.7.
negative (decision): max(Launch −129.7, Skip −20.0) = −20.0 → Skip.
Research (chance): 0.38·369.5 + 0.62·−20 = 128.0.
value(Direct) is called: a decision node; its children are evaluated in turn.
value(Direct·Launch) is called: a chance node; its children are evaluated in turn.
Direct·Launch (chance): 0.4·500 + 0.6·−200 = 80.0.
Direct (decision): max(Launch 80.0, Skip 0.0) = 80.0 → Launch.
root (decision): max(Research 128.0, Direct 80.0) = 128.0 → Research. Strategy: do research; launch if positive, skip if negative.
def value(n): if not isinstance(n, tuple): # leaf return n kind, branches = n if kind == "C": # chance node return sum(p * value(c) for p, c in branches) return max(value(c) for name, c in branches) # decision
Decision Trees and the Value of Information
Expected Value of Information: EVPI and EVSI
| State | Computation | Value |
|---|---|---|
| No information (decide now) | Launch: 0.4·500 + 0.6·(−200) | 80 |
| · | · | · |
| · | · | · |
| · | · | · |
| · | · | · |
Without information the best action is to launch: expected value 80.
| State | Computation | Value |
|---|---|---|
| No information (decide now) | Launch: 0.4·500 + 0.6·(−200) | 80 |
| Perfect information (oracle) | good → launch 500, bad → skip 0: 0.4·500 + 0.6·0 | 200 |
| · | · | · |
| · | · | · |
| · | · | · |
If an oracle revealed the market in advance, the firm would launch in a good market (500) and skip in a bad one (0). Expected 200.
| State | Computation | Value |
|---|---|---|
| No information (decide now) | Launch: 0.4·500 + 0.6·(−200) | 80 |
| Perfect information (oracle) | good → launch 500, bad → skip 0: 0.4·500 + 0.6·0 | 200 |
| EVPI = expected value of perfect information | 200 − 80 | 120 |
| · | · | · |
| · | · | · |
EVPI = 200 − 80 = 120: no information source is worth more than this (an upper bound).
| State | Computation | Value |
|---|---|---|
| No information (decide now) | Launch: 0.4·500 + 0.6·(−200) | 80 |
| Perfect information (oracle) | good → launch 500, bad → skip 0: 0.4·500 + 0.6·0 | 200 |
| EVPI = expected value of perfect information | 200 − 80 | 120 |
| Sample information (research, free) | 128 from the tree + research cost 20 | 148 |
| · | · | · |
Gross value of the imperfect research: add its cost back to the tree's 128 ⇒ 148.
| State | Computation | Value |
|---|---|---|
| No information (decide now) | Launch: 0.4·500 + 0.6·(−200) | 80 |
| Perfect information (oracle) | good → launch 500, bad → skip 0: 0.4·500 + 0.6·0 | 200 |
| EVPI = expected value of perfect information | 200 − 80 | 120 |
| Sample information (research, free) | 128 from the tree + research cost 20 | 148 |
| EVSI = expected value of sample information | 148 − 80 | 68 |
EVSI = 148 − 80 = 68 > cost 20 ⇒ do the research (net gain 48). Efficiency EVSI/EVPI = 57%.
Information has value only if it can change the decision: a negative report leads to "skip", so the research has positive value.
Summary
Summary
Summary
Better Decisions under Risk and Uncertainty
Summary
Practice Problems
- Lottery: 2,500 TL with probability 20%, otherwise 100 TL. Expected value? Certainty equivalent and risk premium for u(x) = √x?
- In the insurance example, what would the maximum premium be if the loss probability were 20%?
- In the medical test, what would P(ill | +) be with prevalence 10% (sensitivity 90%, false positives 9%)?
- In the decision tree, what would the best strategy be if research cost 70 thousand TL?
Show answers
- E[X] = 580; E[u] = 0.2·50 + 0.8·10 = 18; CE = 324; risk premium = 256.
- E[u] = 0.8·30 + 0.2·10 = 26; CE = 676; premium ≤ 224 (expected loss 160).
- P = 0.09 / (0.09 + 0.081) = 10/19 ≈ 52.6%: with a higher base rate, a positive result means much more.
- EVSI = 68 < 70 ⇒ no research; launch directly (expected 80).
Summary
References
- J. von Neumann, O. Morgenstern, Theory of Games and Economic Behavior, Princeton University Press, 1944.
- S. Russell, P. Norvig, Artificial Intelligence: A Modern Approach, 4th ed., Pearson, 2020 (Chapters 12–16: uncertainty, Bayes, decision theory).
- H. Raiffa, Decision Analysis, 1968; D. Bernoulli, “Specimen theoriae novae de mensura sortis”, 1738.
- M. Allais, “Le comportement de l'homme rationnel devant le risque”, Econometrica 21, 1953; L. J. Savage, “The theory of statistical decision”, JASA 46, 1951; D. Ellsberg, Quarterly J. Economics 75, 1961.
- G. Gigerenzer, U. Hoffrage, "How to improve Bayesian reasoning without instruction: Frequency formats", Psychological Review 102(4), 1995.
- W. Casscells, A. Schoenberger, T. B. Graboys, “Interpretation by physicians of clinical laboratory results”, NEJM 299, 1978.
- D. Easley, J. Kleinberg, Networks, Crowds, and Markets, Cambridge University Press, 2010 (Chapter 16: information cascades and Bayes' rule). Tool: Gambit (draws trees with chance nodes and solves them backward).
The End
Next: Chapter 2: Games and Representations