Tuesday, January 15, 2008

Part 10 -- The Problem With "If" (The Revolution Begins On Page Five: The Changing Nature Of The NIE And Its Implications For Intelligence)

Part 1 -- Welcome To The Revolution
Part 2 -- Some History
Part 3 -- The Revolution Begins
Part 4 -- Page Five In Detail
Part 5 -- Enough Exposition, Let's Get Down To It...
Part 6 -- Digging Deeper
Part 7 -- Looking At The Fine Print
Part 8 -- Confidence Is Not the Only Issue
Part 9 -- Waffle Words And Intel-Speak

Part 10 -- The Problem With “If”

So, “could” doesn't work. Nor does “may”, “might” and “possible” (If I had a nickel for every time a decisionmaker has said to me, “Son, anything is possible”, I would be wealthy). Even the only occasionally used “we cannot dismiss” or “hard pressed” create such a strong sense of a lack of definition that analysts should restrict or eliminate them as well from their vocabulary. What’s more, they are all unnecessary if the guidelines laid down in the Explanation Of Estimative Language – the Page 5 of the title – are followed more closely. What, then, is the problem with “if” and the other words on the list? Consider this statement from the recent Iran National Intelligence Estimate (NIE):

  • We assess with high confidence that Iran has the scientific, technical and industrial capacity eventually to produce nuclear weapons if it decides to do so.
Lots of problems here, of course: The use of a statement of confidence without a corresponding Word of Estimative Probability, the use of the word “eventually” (“Hell, son, the world will come to an end…eventually.” Another one I could have funded my retirement with). The “if” clause is particularly problematic, though. "If" clauses have a tendency to beg the real question. What is the real question here? Isn’t it “Will Iran decide to build a nuclear weapon or not?” That is a much more important and interesting question than the question this sentence actually answers concerning the scientific, technical and industrial capacity of Iran. It is sort of like your doctor coming in and saying, “We assess with high confidence that eventually you will not be able to drive your car, if you have cancer.” Glad to hear it, Doc, but can you elaborate on that last part a bit…

“Unless” and “Should” clauses used like “if” are equally worrisome. Consider these two sentences, the first from the Global Terror NIE and the second from the 2nd Iraq Stability NIE:
  • Should al-Zarqawi continue to evade capture and scale back attacks against Muslims, we assess he could broaden his popular appeal and present a global threat.
  • Broadly accepted political compromises required for sustained security, long-term political progress, and economic development are unlikely to emerge unless there is a fundamental shift in the factors driving Iraqi political and security developments.
In the first instance, the unanswered intel question is whether or not Zarqawi will scale back his attacks. In the second, the real question is whether there is likely to be the fundamental shift that the IC has identified as necessary. Admittedly, both questions contain elements that are probably outside of the NICs purview. In the first instance, the issue of whether or not Zarqawi will continue to evade capture falls largely within the realm of those charged with hunting him and going beyond this carefully phrased clause might somehow jeopardize those operations. In the second case it is less clear if the "fundamental shift in factors" is a euphemism for the potential results of planned US and allied action or not. The IC is, I think, rightly cautious about commenting on the possible success or failure of US plans. While the IC is well aware, in general, of the capabilities and limitations of the US government, it spends most of its time and energy focused externally, on threats to and opportunities for the United States. It is not and should not try to also be the expert in applying diplomatic, informational, military or economic pressure outside of the narrow bounds traditionally labeled "covert action" (and maybe not there either...).

Despite this, there are clear intelligence questions here that have gone unanswered through the use of "should" and "unless". Is Zarqawi likely to scale back his attacks or not? Do the factors driving Iraqi political and security developments that are independent of US action likely favor the broadly accepted political compromises deemed necessary? It seems clear that, using the guidance from the EEL page, the IC could make these type estimates more useful to decisionmakers.

Not all “if” clauses are awful, though. There are some, like this one from the Iran NIE, “Barring such acquisitions, if Iran wants to have nuclear weapons it would need to produce sufficient amounts of fissile material indigenously—which we judge with high confidence it has not yet done” where the analysis actually answers the question implied by the “if” clause. Therefore, in computing the percentages in the table in Part 9, I only included “if” clauses that fit the “waffle-word” category.

Part 11 -- One More Thing

No comments: