Chapter 1: Decision Theory

Algorithmic Game Theory
Expected utilityUtility scalesRisk attitudesDecision criteria under uncertaintyBayesian decisionsDecision treesValue of information
Sercan KÜLCÜ · Giresun University · Computer Engineering

Expected Utility

Warm-up Game: Bayesian Decision Lab

The rules

You know the prior of a hidden state, see a noisy signal and choose one of two actions; then the state is revealed and you receive the payoff in the table.

Think about it
  • A forecast that is right 90% of the time says "rain". What is the probability of rain?
  • If a decision leads to a bad outcome, was the decision bad?

Bayesian inference →

RainDryOutdoors−68Indoors43Prior: P(rain) = 0.35Signal: forecast, 90% accurate (Level 1)Forecast: "rain"

Expected Utility

Learning Outcomes

Expected utility and riskCompute the expected value and expected utility of a lottery; interpret risk attitudes via certainty equivalents and risk premia; apply the maximin, Laplace and Savage criteria when probabilities are unknown.
Utility scalesExplain the vNM axioms and the Allais and Ellsberg choices; show which transformations of utility numbers preserve decisions.
Bayesian decisionsUpdate a prior with a test result; show base-rate neglect in a numerical example; choose the action with the smallest expected loss.
Decision treesFind the optimal strategy by backward induction; compute the value of perfect and sample information.

Expected Utility

Decision Making: Three Kinds

Decision making is the process of choosing among options by evaluating their consequences. Three kinds, by what drives the outcome:

KindOutcome driven byToolsExample
Individual, under riskNature / chance (probabilities known or estimated)Expected utility, Bayes, decision treesInvestment, insurance, medical tests
StrategicChoices of other rational agentsBest response, Nash equilibriumPrice competition, bargaining, sports
CollectiveJoint choice and interaction of a groupVoting, common goodsPolicy, corporate strategy
In an individual decision the decision maker plays only against nature (chance). Expected utility, Bayes' rule and decision trees are also the building blocks of strategic and collective decisions.

Expected Utility

Rational Choice: Expected Value or Expected Utility?

  • Lottery: a set of outcomes with probabilities. A = 900 TL with probability 50%, 0 TL with probability 50%.
  • Expected value: E[X] = Σ pᵢ xᵢ. E[A] = ½·900 + ½·0 = 450 TL > a sure B = 400 TL.
  • Expected utility (von Neumann–Morgenstern): E[u] = Σ pᵢ u(xᵢ). A rational decision maker maximizes expected utility.
  • With u(x) = √x: E[u(A)] = ½·30 + ½·0 = 15; u(B) = √400 = 20 ⇒ B is chosen.
A½900 TL½0 TLB400 TLsureE[X] = 450E[√x] = 15E[X] = 400E[√x] = 20

Red: each criterion's choice. Expected value picks A, expected utility (√x) picks B.

Expected monetary value and expected utility may disagree; the gap is driven by the curvature of the utility function, i.e. the risk attitude.

St. Petersburg paradox: nobody pays much for a game with infinite expected value; this is why Daniel Bernoulli (1738) proposed utility (a similar idea: Cramer, 1728).

Expected Utility

Foundations of Expected Utility and the Allais Paradox

  • von Neumann–Morgenstern: if preferences satisfy four axioms, some u exists and the decision maker maximizes expected utility. A ≻ B: A preferred to B.
  • Completeness: any two options are comparable. Transitivity: A ≻ B and B ≻ C imply A ≻ C. Continuity: a middle option equals some mix of the extremes.
  • Independence: mixing the same component (same probability) into two lotteries keeps their ranking.
  • Allais (1953): most people pick A in choice 1, D in choice 2.
Option1 million5 million0
Choice 1A100%——
B89%10%1%
Choice 2C11%—89%
D—10%90%

Swap the common 89% "1 million" in A and B for 89% "0": C and D.

For every u, E[u(A)] − E[u(B)] = 0.11·u(1) − 0.10·u(5) − 0.01·u(0) = E[u(C)] − E[u(D)]. So A ≻ B forces C ≻ D; "A and D" fits no utility function. Overweighting certainty (the certainty effect) is a starting point of prospect theory.

Expected Utility

What Do Utility Numbers Measure? Uniqueness of the Scale

  • Utility is like a thermometer: its zero and its unit are arbitrary. Celsius and Fahrenheit rank temperatures the same way.
  • Uniqueness (vNM): a transformation v = a·u + b (a > 0) is called an affine transformation: it rescales and shifts, but changes no decision between lotteries. Other (non-affine) transformations can change decisions.
  • Link to game theory: multiplying all of a player's payoffs by 2 and adding 5 changes none of its decisions.
Utilitylottery Asure Bchoice
u = √x1520B
v = 10√x + 5155205B
w = (√x)² = x450400A

A: 900 TL with probability ½, 0 TL with probability ½; B: 400 TL for sure. The affine v preserves the decision; the increasing but non-affine w reverses it.

Common mistake: saying "u(B) − u(A) = 5 units of happiness" or "Ayşe's utility is 20 and Bora's 15, so Ayşe is happier". The size of differences and comparisons across people depend on the choice of scale; what is meaningful is one person's ranking of options.

Risk Attitudes

Certainty Equivalent and Risk Premium, Step by Step

0450900030x (TL)uu(x) = √x

Concave utility u(x) = √x: each extra lira adds less utility than the one before (diminishing marginal utility). Endpoints: u(0) = 0, u(900) = 30.

045090001530x (TL)uu(x) = √xE[u] = 15

Chord: the line joining the two ends of the curve. Lottery A's expected utility is the chord's midpoint: E[u] = ½·0 + ½·30 = 15 (at x = 450).

045090001530x (TL)uu(x) = √xE[u] = 15u(450) ≈ 21.2

The same 450 TL for sure gives u(450) ≈ 21.2 > 15. On a concave curve the chord lies below the curve, hence E[u(X)] < u(E[X]) (Jensen's inequality).

022545090001530x (TL)uu(x) = √xE[u] = 15u(450) ≈ 21.2CE = 225

Certainty equivalent (CE): the sure amount with utility 15. √x = 15 ⇒ CE = 225 TL. This person would sell the lottery for 225 TL.

02254004509000152030x (TL)uu(x) = √xE[u] = 15u(450) ≈ 21.2CE = 225risk premium 225sure 400: u = 20

Risk premium = E[X] − CE = 450 − 225 = 225 TL. A sure 400 TL has utility 20 > 15 ⇒ a risk-averse person chooses B.

Risk Attitudes

Three Risk Attitudes: Same Lottery, Different Decisions

AttitudeUtilityE[u(A)]u(400)ChoiceCE (TL)Risk premium
Risk-averseu(x) = √x (concave)1520B225225
Risk-neutralu(x) = x (linear)450400A4500
Risk-seekingu(x) = x²/900 (convex)450177.8A636.4−186.4
Concave → averseCE < E[X], premium > 0: demands a reward for bearing risk. Most people, for large amounts.
Linear → neutralCE = E[X]: looks only at expected value. Large firms, small amounts.
Convex → seekingCE > E[X], premium < 0: pays for risk. Someone buying a lottery ticket.

Lottery A: 900 TL with probability 50%, 0 TL with probability 50% (E[X] = 450). B: 400 TL for sure. CE = u⁻¹(E[u(A)]): u⁻¹ inverts u; for √x, u⁻¹(15) = 15² = 225. Risk premium = E[X] − CE.

Risk Attitudes

Application: Why Does Insurance Exist?

  • Wealth 900 (thousand TL); a loss of 800 (fire) with probability 10%. Utility u(x) = √x.
  • Uninsured: E[u] = 0.9·√900 + 0.1·√100 = 27 + 1 = 28.
  • Certainty equivalent: 28² = 784. The person accepts a sure 784 in place of a risky 900.
  • Maximum premium = 900 − 784 = 116; the expected loss is only 0.1·800 = 80.
740760780800820840860880900bargaining zoneCE = 784premium 116: insured indifferentE[X] = 820premium 80: insurer breaks even900 (no fire)insured wealth = 900 − premium

Insured: paying a premium ≤ 116 to shed the risk raises utility. Insurer: with many independent policies it is nearly risk-neutral; any premium above 80 is profit.

Any premium between 80 and 116 benefits both sides: risk moves to the party best able to bear it.

Risk Attitudes

Known Risk or Unknown Probabilities? The Ellsberg Urn

  • Under risk the probabilities are known (dice, roulette); under uncertainty they are not. People treat the two differently.
  • Urn: 90 balls; 30 are red, the other 60 are black or yellow, but how many are black is unknown. Drawing the winning color pays 100 TL.
  • Typical choices: "red" in bet 1, "black or yellow" in bet 2; in both, the option with a known winning probability.
BetOptionwinning balls
1red30 (known)
blackb (0–60, unknown)
2red or yellow30 + (60 − b)
black or yellow60 (known)

b: number of black balls in the urn. Orange: the majority choice.

Choice 1 means 30 > b, choice 2 means 60 > 90 − b, i.e. b > 30: no probability explains both choices (Ellsberg, 1961). Avoiding options with unknown probabilities is called ambiguity aversion; when probabilities are entirely unknown, choosing to make the worst case as good as possible (maximin) is its extreme form.

Risk Attitudes

Unknown Probabilities: Decisions under Uncertainty

action \ stateGoodFairBadBig investment20050−120Small investment9040−20No investment000

Probabilities are unknown (uncertainty). Rows are actions, columns are states; each criterion reflects a different degree of "pessimism".

action \ stateGoodFairBadminBig investment20050−120−120Small investment9040−20−20No investment0000

Maximin (Wald, plan for the worst): row minima −120, −20, 0 → the largest is 0: No investment.

action \ stateGoodFairBadminmaxBig investment20050−120−120200Small investment9040−20−2090No investment00000

Maximax (optimistic): row maxima 200, 90, 0 → Big investment.

action \ stateGoodFairBadminmaxavg.Big investment20050−120−12020043.3Small investment9040−20−209036.7No investment000000

Laplace (treat states as equally likely): averages 43.3, 36.7, 0.0 → Big investment.

action \ stateGoodFairBadminmaxavg.Big investment20050−120−12020043.3Small investment9040−20−209036.7No investment000000column best:200500

Best achievable result in each state: best = [200, 50, 0]. Regret = best − achieved.

action \ stateGoodFairBadminmaxavg.max regretBig investment20050−120−12020043.3120Small investment9040−20−209036.7110No investment000000200column best:200500

Largest regret per row: Big (bad state, 0 − (−120)) = 120, Small (good state, 200 − 90) = 110, None (good state, 200) = 200.

action \ stateGoodFairBadminmaxavg.max regretBig investment20050−120−12020043.3120Small investment9040−20−209036.7110No investment000000200column best:200500

Savage (minimax regret): the smallest maximum regret is 110 → Small investment.

action \ stateGoodFairBadminmaxavg.max regretBig investment20050−120−12020043.3120Small investment9040−20−209036.7110No investment000000200column best:200500

return (2, 0, 0, 1). Returned indices (0: Big, 1: Small, 2: None): maximin → None, maximax and Laplace → Big, Savage → Small. Three different decisions from one table.

action \ stateGoodFairBadminmaxavg.max regretBig investment20050−120−12020043.3120Small investment9040−20−209036.7110No investment000000200column best:200500

One more criterion, not in the code: Hurwicz sits between the extremes: α·(row max) + (1 − α)·(row min); with optimism coefficient α = 0.5: 40, 35, 0 → Big. If probabilities can be estimated, return to expected utility; if not, the choice of criterion reflects the decision maker's attitude.

def criteria(U):
    A, m = range(len(U)), len(U[0])
    mm = max(A, key=lambda i: min(U[i]))      # maximin
    mx = max(A, key=lambda i: max(U[i]))      # maximax
    lp = max(A, key=lambda i: sum(U[i]) / m)  # Laplace
    best = [max(r[j] for r in U) for j in range(m)]
    reg = [max(best[j] - r[j] for j in range(m))
           for r in U]            # max regret
    sv = reg.index(min(reg))                  # Savage
    return mm, mx, lp, sv

Bayesian Decisions

Medical Test: What Does a Positive Result Mean?

10,000people

Picture a population of 10,000. Using natural frequencies (head counts) instead of probabilities makes Bayesian reasoning intuitive.

10,000people1% ill99% healthy100ill9,900healthy

Prior: prevalence 1% ⇒ 100 ill, 9,900 healthy.

10,000people1% ill99% healthy100ill9,900healthy90% +10% −90ill, test +10ill, test −

Sensitivity 90%: 90 of the 100 ill people test positive, 10 negative.

10,000people1% ill99% healthy100ill9,900healthy90% +10% −90ill, test +10ill, test −9% +91% −891healthy, test +9,009healthy, test −

False positive rate 9%: 891 of the 9,900 healthy people also test positive! Healthy people are so numerous that this number is large.

10,000people1% ill99% healthy100ill9,900healthy90% +10% −90ill, test +10ill, test −9% +91% −891healthy, test +9,009healthy, test −test +981

Everyone testing positive: 90 + 891 = 981 people. Only 90 of them are actually ill.

10,000people1% ill99% healthy100ill9,900healthy90% +10% −90ill, test +10ill, test −9% +91% −891healthy, test +9,009healthy, test −test +9819.2%

Posterior: P(ill | +) = 90 / 981 = 9.2%. Even with 90% sensitivity, a positive result is mostly a false alarm (base-rate neglect).

Bayesian Decisions

Code Trace: bayes(p, sens, fp, tests)

prior0.0100test 1test 2test 3

Call: bayes(0.01, 0.9, 0.09, ["+", "+", "-"]). p starts at the prior (1%); the three test results are processed in turn.

prior0.0100test 1test 2test 3

Test 1 result: +. The current belief p = 0.0100 is this test's prior.

prior0.0100test 1test 2test 3

res == "+" ? → True: use the likelihoods of a positive result. Likelihood: the probability of this result given the disease status, e.g. P(+ | ill) = 0.9.

prior0.0100test 1test 2test 3

num = P(+|ill)·p = 0.9 × 0.0100 = 0.0090; den = num + P(+|healthy)·(1 − p) = 0.0981 (total probability of a positive).

prior0.0100test 10.0917test 2test 3

p = num / den = 0.0917, printed. Despite the positive test, the probability of disease is still below 10%.

prior0.0100test 10.0917test 2test 3

Test 2 result: +. The current belief p = 0.0917 is this test's prior.

prior0.0100test 10.0917test 2test 3

res == "+" ? → True: use the likelihoods of a positive result.

prior0.0100test 10.0917test 2test 3

num = P(+|ill)·p = 0.9 × 0.0917 = 0.0826; den = num + P(+|healthy)·(1 − p) = 0.1643 (total probability of a positive).

prior0.0100test 10.0917test 20.5025test 3

p = num / den = 0.5025, printed. Two positives, independent given the disease status, raised the probability to ~50%.

prior0.0100test 10.0917test 20.5025test 3

Test 3 result: -. The current belief p = 0.5025 is this test's prior.

prior0.0100test 10.0917test 20.5025test 3

res == "+" ? → False: use the likelihoods of a negative result.

prior0.0100test 10.0917test 20.5025test 3

num = P(−|ill)·p = 0.1 × 0.5025 = 0.0503; den = num + P(−|healthy)·(1 − p) = 0.91 × 0.4975 + num = 0.5030.

prior0.0100test 10.0917test 20.5025test 30.0999

p = num / den = 0.0999, printed. The negative result brought the belief back down to 10%.

prior0.0100test 10.0917test 20.5025test 30.0999

The loop ends; the function returns p = 0.0999. Each test uses the previous posterior as its prior (sequential updating).

def bayes(p, sens, fp, tests):
  for res in tests:
    if res == "+":
      num = sens * p
      den = num + fp * (1 - p)
    elif res == "-":
      num = (1 - sens) * p
      den = num + (1 - fp) * (1 - p)
    else: raise ValueError("result must be + or -")
    if den == 0: raise ValueError("impossible result")
    p = num / den
    print(res, round(p, 4))
  return p

Bayesian Decisions

Bayesian Decision Rule: Treat or Wait?

  • Update the prior with the evidence; then compute each action's expected loss under the posterior. Choose the action with the lowest expected loss.
  • Losses: needless treatment of a healthy patient 30, not treating an ill one 200.
  • E[loss | treat] = (1 − p)·30; E[loss | wait] = p·200. Treat if it loses less: (1 − p)·30 < p·200 ⇒ 30 < 230p ⇒ p > 30/230 = 3/23 ≈ 13.0%.
Stateptreatwaitdecision
1 positive0.091727.218.3wait, retest
2 positives0.502514.9100.5treat

Expected losses (orange = smallest)

The decision depends not only on the probability but on the ratio of the losses: the costlier a missed disease, the lower the threshold.

Bayesian Decisions

Back to the Game: Bayesian Decision Lab

Theory's answer: 82.9%

P(rain | "rain") = 0.9·0.35 / (0.9·0.35 + 0.1·0.65) = 0.315 / 0.38 ≈ 0.829.

InformationP(rain)EU(out)EU(in)Decision
prior only0.353.103.35indoors
"rain"0.829−3.613.83indoors
"dry"0.0567.213.06outdoors

EU: expected utility. Outdoors is chosen only when P(rain) < 1/3. At Level 3 (78% accuracy) the signal's value of information is 1.765; it exceeds the signal's price (1), so buying the signal pays off.

In practice: base-rate neglect

In a classic study, physicians and medical students were asked about a test with prevalence 1/1000 and a 5% false positive rate: only 11 of 60 reached the right answer (about 2%); the most common answer was 95% (Casscells, Schoenberger and Graboys, 1978).

The game's "EU-optimal" counter counts not lucky outcomes but decisions that maximize expected utility given the information at hand: a good decision can lead to a bad outcome.

Bayesian Decisions

Bridge to Bayesian Games: Updating the Opponent's Type

  • With incomplete information the unknown "state" is the opponent's type: the belief is the prior, the move is the evidence.
  • Example (poker): the opponent is a bluffer with probability 20%, honest with 80%. A bluffer raises 90% of the time whatever the hand; an honest player raises only with a strong hand (30%).
  • The opponent raised: P(bluffer | raise) = 0.2·0.9 / (0.2·0.9 + 0.8·0.3) = 0.18 / 0.42 ≈ 0.429.
  • Calling earns +30 against a bluffer, −30 against an honest player: E[call] = 0.429·30 − 0.571·30 ≈ −4.3 < 0 ⇒ fold. After a second raise the posterior is 0.692.
ObservationP(bluffer)E[call]decision
prior only0.200−18.0fold
1 raise0.429−4.3fold
2 raises0.692+11.5call

Calling pays only if P(bluffer) > 0.5 (equal gain and loss).

Common mistake: confusing P(raise | bluffer) = 0.9 with P(bluffer | raise). Honest players are more common, so most raises (0.24 / 0.42 ≈ 57%) come from them. A bluff tries to steer this update.

Decision Trees and the Value of Information

Decision Tree: Product Launch and Market Research

  • A firm must decide whether to launch a new product. If the market is good (probability 2/5) it earns +500, if bad (3/5) −200 thousand TL; no launch gives 0.
  • It can first buy market research for 20 thousand TL. The report is imperfect: P(positive | good) = 0.8, P(positive | bad) = 0.1.
  • By Bayes: P(positive) = 0.4·0.8 + 0.6·0.1 = 0.38; P(good | positive) = 0.32/0.38 = 16/19; P(good | negative) = 0.08/0.62 = 4/31.
□ Decision nodeThe decision maker chooses → the largest value among the branches.
○ Chance nodeNature chooses → probability-weighted average.
LeafThe net result of a path.
A decision tree is a game tree in which nature also moves (chance nodes); it is solved from the leaves to the root by backward induction.

Decision Trees and the Value of Information

Backward Induction in a Decision Tree

ResearchDirectpositive 19/50negative 31/50LaunchLaunchLaunchGood 16/19480Bad 3/19−220Skip−20Good 4/31480Bad 27/31−220Skip−20Good 2/5500Bad 3/5−200Skip0net payoff (thousand TL)

Square = decision node, circle = chance node. Leaves show net payoffs (20 deducted if research is bought). The two root branches: Research and Direct (decide without research). Backward induction: expected value at chance nodes, maximum at decision nodes.

ResearchDirectpositive 19/50negative 31/50LaunchLaunchLaunchGood 16/19480Bad 3/19−220Skip−20Good 4/31480Bad 27/31−220Skip−20Good 2/5500Bad 3/5−200Skip080net payoff (thousand TL)

Launching without research: 2/5·500 + 3/5·(−200) = 200 − 120 = 80.

ResearchDirectpositive 19/50negative 31/50LaunchLaunchLaunchGood 16/19480Bad 3/19−220Skip−20Good 4/31480Bad 27/31−220Skip−20Good 2/5500Bad 3/5−200Skip08080net payoff (thousand TL)

Decision: max(Launch 80, Skip 0) = 80 ⇒ Launch. The Skip branch is pruned.

ResearchDirectpositive 19/50negative 31/50LaunchLaunchLaunchGood 16/19480Bad 3/19−220Skip−20Good 4/31480Bad 27/31−220Skip−20Good 2/5500Bad 3/5−200Skip0369.58080net payoff (thousand TL)

After a positive report, P(Good | positive) = 0.32/0.38 = 16/19 (Bayes). Expected: 16/19·480 + 3/19·(−220) ≈ 369.5.

ResearchDirectpositive 19/50negative 31/50LaunchLaunchLaunchGood 16/19480Bad 3/19−220Skip−20Good 4/31480Bad 27/31−220Skip−20Good 2/5500Bad 3/5−200Skip0369.5369.58080net payoff (thousand TL)

Decision: max(369.5, −20) ⇒ Launch after a positive report.

ResearchDirectpositive 19/50negative 31/50LaunchLaunchLaunchGood 16/19480Bad 3/19−220Skip−20Good 4/31480Bad 27/31−220Skip−20Good 2/5500Bad 3/5−200Skip0369.5369.5−129.78080net payoff (thousand TL)

After a negative report, P(Good | negative) = 0.08/0.62 = 4/31. Expected: 4/31·480 + 27/31·(−220) ≈ −129.7.

ResearchDirectpositive 19/50negative 31/50LaunchLaunchLaunchGood 16/19480Bad 3/19−220Skip−20Good 4/31480Bad 27/31−220Skip−20Good 2/5500Bad 3/5−200Skip0369.5369.5−20−129.78080net payoff (thousand TL)

Decision: max(−129.7, −20) = −20 ⇒ Skip after a negative report (only the research cost is lost).

ResearchDirectpositive 19/50negative 31/50LaunchLaunchLaunchGood 16/19480Bad 3/19−220Skip−20Good 4/31480Bad 27/31−220Skip−20Good 2/5500Bad 3/5−200Skip0128369.5369.5−20−129.78080net payoff (thousand TL)

Report chance node: 0.38·369.5 + 0.62·(−20) = 128.

ResearchDirectpositive 19/50negative 31/50LaunchLaunchLaunchGood 16/19480Bad 3/19−220Skip−20Good 4/31480Bad 27/31−220Skip−20Good 2/5500Bad 3/5−200Skip0128128369.5369.5−20−129.78080net payoff (thousand TL)

Root: max(Research 128, Direct 80) ⇒ Research. Strategy: do research; launch if positive, skip if negative.

Decision Trees and the Value of Information

Code Trace: Recursive Backward Induction

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200□ decision: maximum ○ chance: expected value

value(root) is called. The tree is nested tuples: root = ("D", [("Research", ("C", [(0.38, …), (0.62, …)])), ("Direct", ("D", […]))]); "D" is a decision node, "C" a chance node, a leaf is a number.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200□ decision: maximum ○ chance: expected value

value(Research) is called: a chance node; its children are evaluated in turn.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200□ decision: maximum ○ chance: expected value

value(positive) is called: a decision node; its children are evaluated in turn.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200□ decision: maximum ○ chance: expected value

value(positive·Launch) is called: a chance node; its children are evaluated in turn.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200369.5□ decision: maximum ○ chance: expected value

positive·Launch (chance): 0.842·480 + 0.158·−220 = 369.5.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200369.5369.5□ decision: maximum ○ chance: expected value

positive (decision): max(Launch 369.5, Skip −20.0) = 369.5 → Launch.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200369.5369.5□ decision: maximum ○ chance: expected value

value(negative) is called: a decision node; its children are evaluated in turn.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200369.5369.5□ decision: maximum ○ chance: expected value

value(negative·Launch) is called: a chance node; its children are evaluated in turn.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200369.5369.5−129.7□ decision: maximum ○ chance: expected value

negative·Launch (chance): 0.129·480 + 0.871·−220 = −129.7.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200369.5−20.0369.5−129.7□ decision: maximum ○ chance: expected value

negative (decision): max(Launch −129.7, Skip −20.0) = −20.0 → Skip.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200128.0369.5−20.0369.5−129.7□ decision: maximum ○ chance: expected value

Research (chance): 0.38·369.5 + 0.62·−20 = 128.0.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200128.0369.5−20.0369.5−129.7□ decision: maximum ○ chance: expected value

value(Direct) is called: a decision node; its children are evaluated in turn.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200128.0369.5−20.0369.5−129.7□ decision: maximum ○ chance: expected value

value(Direct·Launch) is called: a chance node; its children are evaluated in turn.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200128.0369.5−20.080.0369.5−129.7□ decision: maximum ○ chance: expected value

Direct·Launch (chance): 0.4·500 + 0.6·−200 = 80.0.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200128.080.0369.5−20.080.0369.5−129.7□ decision: maximum ○ chance: expected value

Direct (decision): max(Launch 80.0, Skip 0.0) = 80.0 → Launch.

ResearchDirectpositive 0.38negative 0.62Skip −20Skip −20good 480bad −220good 480bad −220Skip 0good 500bad −200128.0128.080.0369.5−20.080.0369.5−129.7□ decision: maximum ○ chance: expected value

root (decision): max(Research 128.0, Direct 80.0) = 128.0 → Research. Strategy: do research; launch if positive, skip if negative.

def value(n):
    if not isinstance(n, tuple):    # leaf
        return n
    kind, branches = n
    if kind == "C":                  # chance node
        return sum(p * value(c) for p, c in branches)
    return max(value(c) for name, c in branches)  # decision

Decision Trees and the Value of Information

Expected Value of Information: EVPI and EVSI

StateComputationValue
No information (decide now)Launch: 0.4·500 + 0.6·(−200)80
···
···
···
···

Without information the best action is to launch: expected value 80.

StateComputationValue
No information (decide now)Launch: 0.4·500 + 0.6·(−200)80
Perfect information (oracle)good → launch 500, bad → skip 0: 0.4·500 + 0.6·0200
···
···
···

If an oracle revealed the market in advance, the firm would launch in a good market (500) and skip in a bad one (0). Expected 200.

StateComputationValue
No information (decide now)Launch: 0.4·500 + 0.6·(−200)80
Perfect information (oracle)good → launch 500, bad → skip 0: 0.4·500 + 0.6·0200
EVPI = expected value of perfect information200 − 80120
···
···

EVPI = 200 − 80 = 120: no information source is worth more than this (an upper bound).

StateComputationValue
No information (decide now)Launch: 0.4·500 + 0.6·(−200)80
Perfect information (oracle)good → launch 500, bad → skip 0: 0.4·500 + 0.6·0200
EVPI = expected value of perfect information200 − 80120
Sample information (research, free)128 from the tree + research cost 20148
···

Gross value of the imperfect research: add its cost back to the tree's 128 ⇒ 148.

StateComputationValue
No information (decide now)Launch: 0.4·500 + 0.6·(−200)80
Perfect information (oracle)good → launch 500, bad → skip 0: 0.4·500 + 0.6·0200
EVPI = expected value of perfect information200 − 80120
Sample information (research, free)128 from the tree + research cost 20148
EVSI = expected value of sample information148 − 8068

EVSI = 148 − 80 = 68 > cost 20 ⇒ do the research (net gain 48). Efficiency EVSI/EVPI = 57%.

Information has value only if it can change the decision: a negative report leads to "skip", so the research has positive value.

Summary

Summary

Expected utilityE[u] = Σ pᵢ u(xᵢ). Lottery A: E[X] = 450, but with √x, E[u] = 15 < u(400) = 20.
Utility scalesThe transformation a·u + b (a > 0) preserves decisions; the Allais and Ellsberg choices deviate systematically from expected utility.
Risk attitudeConcave → averse (CE 225, premium 225); linear → neutral; convex → seeking.
Uncertainty criteriaOn the same table, maximin picks No investment, maximax and Laplace Big, Savage Small.
Bayesian decisionsPrevalence 1%, sensitivity 90%, false positives 9% ⇒ P(ill | +) ≈ 9.2%; treat only if p > 3/23.
Decision treesAverage at chance nodes, maximize at decisions. Research (128) > no research (80); EVPI = 120, EVSI = 68.

Summary

Better Decisions under Risk and Uncertainty

Value informationCurrent data improves decisions; gathering information pays only if EVSI exceeds its cost.
Scenarios and decision treesOnce the possible states and their probabilities are written down, the tree can be solved backward.
Account for the priorBase rates enter the posterior through Bayes' rule; thinking in natural frequencies reduces errors.
Opponents and diversificationGame theory and simulation model opponents' behavior; portfolio diversification reduces risk.

Summary

Practice Problems

  1. Lottery: 2,500 TL with probability 20%, otherwise 100 TL. Expected value? Certainty equivalent and risk premium for u(x) = √x?
  2. In the insurance example, what would the maximum premium be if the loss probability were 20%?
  3. In the medical test, what would P(ill | +) be with prevalence 10% (sensitivity 90%, false positives 9%)?
  4. In the decision tree, what would the best strategy be if research cost 70 thousand TL?
Show answers
  1. E[X] = 580; E[u] = 0.2·50 + 0.8·10 = 18; CE = 324; risk premium = 256.
  2. E[u] = 0.8·30 + 0.2·10 = 26; CE = 676; premium ≤ 224 (expected loss 160).
  3. P = 0.09 / (0.09 + 0.081) = 10/19 ≈ 52.6%: with a higher base rate, a positive result means much more.
  4. EVSI = 68 < 70 ⇒ no research; launch directly (expected 80).

Summary

References

  • J. von Neumann, O. Morgenstern, Theory of Games and Economic Behavior, Princeton University Press, 1944.
  • S. Russell, P. Norvig, Artificial Intelligence: A Modern Approach, 4th ed., Pearson, 2020 (Chapters 12–16: uncertainty, Bayes, decision theory).
  • H. Raiffa, Decision Analysis, 1968; D. Bernoulli, “Specimen theoriae novae de mensura sortis”, 1738.
  • M. Allais, “Le comportement de l'homme rationnel devant le risque”, Econometrica 21, 1953; L. J. Savage, “The theory of statistical decision”, JASA 46, 1951; D. Ellsberg, Quarterly J. Economics 75, 1961.
  • G. Gigerenzer, U. Hoffrage, "How to improve Bayesian reasoning without instruction: Frequency formats", Psychological Review 102(4), 1995.
  • W. Casscells, A. Schoenberger, T. B. Graboys, “Interpretation by physicians of clinical laboratory results”, NEJM 299, 1978.
  • D. Easley, J. Kleinberg, Networks, Crowds, and Markets, Cambridge University Press, 2010 (Chapter 16: information cascades and Bayes' rule). Tool: Gambit (draws trees with chance nodes and solves them backward).

The End

Chapter 1: Decision Theory
Previous: Chapter 0: Preliminaries
Next: Chapter 2: Games and Representations
1 / 1 Sercan KÜLCÜ. All rights reserved.