Apr 9, 2018

Affinities between national defense analysis and retirement?

I thought this paper (Handling and Mishandling Estimative Probability: Likelihood,Confidence, and the Search for Bin Laden Jeffrey A. Friedman and Richard Zeckhauser  Intelligence and National Security, 2014) was an interesting short cover of an area of decision theory dealing with presenting analysis to decision makers in situations with both uncertainty and time-sensitive outcomes.  In particular I thought there were some fun affinities with retirement finance where the seperability of the idea of risk from uncertainty (or confidence) makes some sense.  Retirement is nothing if not probabilities confounded by uncertainty.  Here are some excerpts tied to my highlighting and annotation: 


"a range of probabilities can always be condensed into a single point estimate that is clearer (but logically no different) than standard intelligence reporting, and why assessments of confidence are most useful when they indicate the extent to which estimative probabilities might shift in response to newly gathered information. 


the president’s self-professed frustration about the Abbottabad debate suggests the continuing need for scholars, analysts, and policymakers to improve their conceptual understanding of how to express and interpret uncertainty. 

assessments of confidence should be separated from assessments of likelihood, and that expressions of confidence are most useful when they serve the specific function of describing how much predictions might shift in response to new information.

the traditional function of expressing confidence as a way to hedge estimates or describe their ‘information content’ should have little influence over the decision making process per se. 

[the analyst] must consciously police the consistency of his subjective inputs and calculate their implications for action. Such an approach is designed to help us reason and act a bit more systematically… -- Hward Raiffa,  Decision Analysis: introductory lectures on choices under uncertainty  
 risk aversion appropriately applies to evaluating outcomes and not their probabilities.
This is consistent with the notion that people often tilt probability estimates towards caution. It is also consistent with a common bias in which individuals default to 50/50 assessments when presented with complex situations, regardless of how well that stance fits with the evidence at hand. Both of these tendencies can be exacerbated by presenting a range of predicted probabilities rather than a single point estimate.

anyone confronted with multiple estimates should combine them in a manner that weights each one in proportion to its credibility. 

since a range of estimative probabilities always implies a single point estimate, then not giving a single point estimate may only confuse a decision maker, while simultaneously preventing analysts from refining and controlling the message they convey. 

If analysts wish to convey information about ambiguity and confidence, that information should be offered separately and explicitly. [Emphasis added]

Thus, the relevant question is not necessarily where the ‘information content’ of assessments stands at the moment, but how much that information might change and how far those assessments might shift moving forward. Existing intelligence tradecraft does not express this information directly. 


First, analysts should convey likelihood using point estimates rather than ranges of predicted probabilities such as those offered by President Obama’s advisors in the Abbottabad debate. This article showed that, even if analysts do not state a point estimate, they cannot avoid at least implying one; therefore, they might as well present a point estimate explicitly so as to avoid unnecessary confusion.

Second, analysts should articulate confidence by assessing the ‘responsiveness’ of their estimates in a manner that explains how much their views might shift in light of new intelligence. [the evaluative tool implied earlier in the paper was either "how much will the estimate change with new information over what time frame" or "is there a benefit of delay." Maybe think: options value of delaying annuitization or something like that...]

the ideas suggested in this article impose demands on analysts that again are less about how individuals should form their views about ambiguous information than they are about how to express those views in ways that decision makers will find useful. Decision makers already have to form opinions about the potential value of gathering more information, and in doing so they already must think through different scenarios for how much existing assessments might respond as new material comes to light. That determination will always involve subjective factors, but those factors can at least be identified and conveyed in a structured fashion. "




No comments:

Post a Comment