Part 6 -- The Decisionmaker's Perspective (Evaluating Intelligence)

Part 1 -- Introduction

Part 2 -- A Tale Of Two Weathermen
Part 3 -- A Model For Evaluating Intelligence
Part 4 -- The Problems With Evaluating Intelligence Products
Part 5 -- The Problems With Evaluating Intelligence Process

Decisionmakers are charged with making decisions. While this statement is blindingly obvious, its logical extension actually has some far reaching consequences.

First, even if the decision is to "do nothing" in a particular instance, it is still a decision. Likewise, with the authority to make a decision comes (or should come) responsibility and accountability for that decision's consequences (The recent kerfluffle surrounding the withdrawal of Tom Daschle for an appointment in the Obama cabinet is instructive in this matter -- see the video below).



Driving these decisions are typically two kinds of forces. The first is largely internal to the individual or organization making the decision. The focus here is on the capabilities and limitations of the organization itself: How well-trained are my soldiers? How competent are my salespeople? How competitive are my prices, how efficient are my logistics or how well equipped are my police units? Good decisionmakers are often comfortable here. They know themselves quite well. Oftentimes they have risen through the ranks of an organization or even started the organization on their own. The internal workings of a decisionmaker's own organization are easiest (if not easy) to see, predict and control.

The same cannot be said of external forces. The current upheaval in the global market is likely, for example, to affect even good, well-run businesses in ways that are extremely difficult to predict, much less control. The opaque strategies of state and non-state actors threaten national security plans and the changing tactics of organized criminal activity routinely frustrate law enforcement professionals. Understanding these external forces is a defining characteristic of intelligence and the complexity of these forces is often used to justify substantial intelligence budgets.

Experienced decisionmakers do not expect intelligence professionals to be able to understand external forces to the same degree that it is possible to understand internal forces. They do expect intelligence to reduce their uncertainty, in tangible ways, regarding these external forces. Sometimes intelligence provides up-to-date descriptive information, unavailable previously to the decisionmaker (such as the U2 photographs in the run-up to the Cuban Missile Crisis). Decisionmakers, however, find it much more useful when analysts provide estimative intelligence -- assessments about how the relevant external forces are likely to change.

• Note: I do not intend to address concerns regarding bad or stupid decisionmakers in this series of posts, though both clearly exist. These concerns are outside the scope of a discussion about evaluating intelligence and fall more naturally into the realms of management studies or psychology. I have a slightly different take on inexperienced (with intel) decisionmakers, however. I teach my students that intel professionals have an obligation to teach the decisionmakers they support about what intel can and cannot do in the same way the grizzled old platoon sergeant has an obligation to teach the newly minted second lieutenant about the ins and outs of the army.
  

Obviously then, knowing, with absolute certainty, where the relevant external forces will be and what they will be doing is of primary importance to decisionmakers. Experienced decisionmakers also know that to expect such precision from intelligence is unrealistic. Rather, they expect that the estimates they receive will only reduce their uncertainty about those external forces, allowing them to plan and decide with greater clarity.

Imagine, for example, a battalion commander on a mission to defend a particular piece of terrain. The intelligence officer tells the commander that the enemy has two primary avenues of approach, A and B, and that it is "likely" that the enemy will choose avenue A. How does this intelligence inform the commander's decision about how to defend the objective?

For the sake of argument, let's assume that the battalion commander interprets the word "likely" as meaning "about 60%". Does this mean that the battalion commander should allocate about 60% of his forces to defending Avenue A and 40% to defending Avenue B? That is a solution but there are many, many ways in which such a decision would make no sense at all. The worst case scenario for the battalion commander, however, is if he only has enough forces to adequately cover one of the two avenues of approach. In this case, diverting any forces at all will guarantee failure.

Assuming an accurate analytic process and all other things being equal (and I can do that because this is a thought experiment), the commander should align his forces along Avenue A in this situation. This gives him the best chance of stopping the enemy forces. This decisionmaker, with his limited forces, is essentially forced by the situation to treat a 60% probability as 100% accurate for planning purposes. Since many decisions are (or appear to be to the decisionmaker) to be of this type, it is no wonder that decisionmakers, when they evaluate intelligence, tend to focus on the ability to discriminate between possible outcomes over the ability to calibrate the estimative conclusion.

Tomorrow: The Iraq NIEs

Reblog this post [with Zemanta]">