Back
New paper: "Functional Decision Theory" - Machine Intelligence Research Institute
webCredibility Rating
3/5
Good(3)Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.
Rating inherited from publication venue: MIRI
This MIRI announcement introduces the Yudkowsky & Soares FDT paper, relevant to AI alignment as agent foundations work on decision theory informs how rational AI agents should reason about their own decision-making processes.
Metadata
Importance: 72/100blog postprimary source
Summary
MIRI researchers Eliezer Yudkowsky and Nate Soares introduce Functional Decision Theory (FDT), a decision theory that treats one's decision as the output of a fixed mathematical function optimizing for best outcomes. FDT outperforms both Causal Decision Theory (CDT) and Evidential Decision Theory (EDT) across canonical decision-theoretic problems including Newcomb's Problem, the Smoking Lesion Problem, and Parfit's Hitchhiker. The paper provides formal definitions, philosophical justifications, and comparative analysis of all three decision theories.
Key Points
- •FDT asks 'which output of this mathematical function would yield the best outcome?' treating decisions as outputs of a fixed function rather than isolated choices.
- •FDT achieves better utility than CDT on Newcomb's problem, better than EDT on the smoking lesion problem, and better than both on Parfit's hitchhiker problem.
- •FDT relies on 'subjunctive dependence' — a broader class of dependencies than CDT's causal dependencies — to determine which variables the agent's decision influences.
- •Three arguments favor FDT: the argument from precommitment, argument from information value, and argument from utility.
- •This paper is the most complete stand-alone introduction to FDT, formalizing algorithms for EDT, CDT, and FDT to precisely specify each theory's prescriptions.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Agent Foundations | Approach | 59.0 |
Cached Content Preview
HTTP 200Fetched Mar 15, 202611 KB
New paper: "Functional Decision Theory" - Machine Intelligence Research Institute
Skip to content
New paper: “Functional Decision Theory”
October 22, 2017
Matthew Gray
MIRI senior researcher Eliezer Yudkowsky and executive director Nate Soares have a new introductory paper out on decision theory: “ Functional decision theory: A new theory of instrumental rationality .”
Abstract:
This paper describes and motivates a new decision theory known as functional decision theory (FDT), as distinct from causal decision theory and evidential decision theory.
Functional decision theorists hold that the normative principle for action is to treat one’s decision as the output of a fixed mathematical function that answers the question, “Which output of this very function would yield the best outcome?” Adhering to this principle delivers a number of benefits, including the ability to maximize wealth in an array of traditional decision-theoretic and game-theoretic problems where CDT and EDT perform poorly. Using one simple and coherent decision rule, functional decision theorists (for example) achieve more utility than CDT on Newcomb’s problem, more utility than EDT on the smoking lesion problem, and more utility than both in Parfit’s hitchhiker problem.
In this paper, we define FDT, explore its prescriptions in a number of different decision problems, compare it to CDT and EDT, and give philosophical justifications for FDT as a normative theory of decision-making.
Our previous introductory paper on FDT, “ Cheating Death in Damascus ,” focused on comparing FDT’s performance to that of CDT and EDT in fairly high-level terms. Yudkowsky and Soares’ new paper puts a much larger focus on FDT’s mechanics and motivations, making “Functional Decision Theory” the most complete stand-alone introduction to the theory. ((“Functional Decision Theory” was originally drafted prior to “ Cheating Death in Damascus ,” and was significantly longer before we received various rounds of feedback from the philosophical community. “Cheating Death in Damascus” was produced from material that was cut from early drafts; other cut material included a discussion of proof-based decision theory , and some Death in Damascus variants left on the cutting room floor for being needlessly cruel to CDT.))
Contents:
1. Overview.
2. Newcomb’s Problem and the Smoking Lesion Problem. In terms of utility gained, conventional EDT outperforms CDT in Newcomb’s problem, while underperforming CDT in the smoking lesion problem. Both CDT and EDT have therefore appeared unsastisfactory as expected utility theories, and the debate between the two has
... (truncated, 11 KB total)Resource ID:
170ed59807cc7b2f | Stable ID: ZDg4MzNlMD