人工智能风险的超越P（厄运）：无概率量化不确定性

Executive Summary Many observers wonder aboutthepotentialfor artificial intelligenceto causecatastrophic risks, but fortunately, there is little empirical evidenceon which tobasethose assessments. Absent such evidence, experts often use their best guesses toestimate the probability of an AI-induced catastrophe or apocalypse (i.e.,p-doom).Although subjective expert assessment may be the best evidence available,policymakers and risk analysts are not restricted to asking for probabilities. This briefpromotes additional tools for handling uncertainty in AI risk assessments. Imagine you are asked to roll a six-sided die but you have only seen three sides;oneside has a star etched on it, two sides are blank, and the other three are unknown.Predicting the outcome involves part randomness and part ignorance. If you are askedto give the probability of a star, you might note that one of the three sides you’ve seenhas a star and answer1/3. Asked how confident to be that a star will come up, there isonly one side that you know has a star, so1/6would be a reasonable answer. Askedwhether a star could come up, you might note that four sides could have a star, so4/6would be a reasonable answer. These questions appear similar, but their differencesare important if you are adecision-maker who cares about stars. In AI risk, rather than in dice rolls, ignorance is the dominant form of uncertainty, notrandomness, so the best techniques are not always probabilistic. There are alternativemathematical techniques that are just as rigorous as probability. They also usefamiliarterms from common discourse, such as Belief and Plausibility, allowing them to easilybecome part of popular AI risk vernacular and to be communicated to decision makers. The way to think of the mathematical term Belief is that it expresses how confidentone can be based on the evidence. For instance, the evidence allows us to have a 1/6degree of belief that the die will come up stars. Plausibility expresses what is left afterremoving the counter-evidence. Two of the six sides cannot be stars, so the Plausibilityof stars is 4/6. The gap between Belief and Plausibility is due to ignorance. Withoutignorance, Belief and Plausibility become the same number, equal to probability. This issue brief explains why analysts and decision-makers need alternatives toprobability for handling the uncertainty in AI risk. It explains Belief, Plausibility, andhow they relate to probability in an intuitively accessible way. And it demonstrateshow to calculate Belief and Plausibility in the context of expert assessments of AI risk. At a high level, enacting the change sought by this brief is easy. Policymakers onlyneed to add two additional questions when discussing AI risks. The first is either,howcertain are you that this risk will occur, or even better,howstrong is the evidencesupporting this hypothetical outcome? The second is,how certain are you that this riskwillnotoccur,orhowstrong is the evidenceagainstthis hypothetical outcome? Asking those two additional questions will force analysts to confront their sources ofuncertainty more directly and drive analysts to expand their risk analysis toolbox.Answering those questions, and communicating those answers, is also a low liftbecause the analytical techniques already exist and because the vocabulary is alreadyfamiliar. This brief provides an introduction to those techniques and vocabulary. Table of Contents Executive Summary................................................................................................................................1Introduction...............................................................................................................................................4Probability Is Not the Only Option....................................................................................................5Aleatoric and Epistemic Uncertainty............................................................................................5Alternatives to Probability...............................................................................................................7Belief, Plausibility, and Probability....................................................................................................8Belief.......................................................................................................................................................8Plausibility............................................................................................................................................8Probability.............................................................................................................................................8Structure of Evidence.............................................................................................................................9Indirect, Subjective, and Conflicting......................................................