Part 1 -- Welcome To The Revolution
Part 2 -- Some History
Part 3 -- The Revolution Begins
Part 4 -- Page Five In Detail
Part 5 -- Enough Exposition! Let’s Get Down To It…
In order to benchmark the estimates, I started by counting the number of sentences in each of the last several NIEs. For comparison purposes, I included the 2002 WMD NIE and the 1990 Yugoslavia NIE as well. I only looked at the Key Judgments of each NIE. While I understand that there is a good bit more information in the NIE than in the Key Judgments, I am virtually certain that the bulk of the NIE is consistent with the Key Judgments whether they are made public or not.
As you can see by the chart above the numbers are fairly consistent across the NIEs. The minimum number of sentences is 25 with a maximum of 52. The average is 38. I don’t consider these differences in length to be important. They are likely explained by the nature of the subject matter and the level of detail of the full text NIE. I wanted, however, to be able to compare words and phrases defined in the Explanation of Estimative Language (EEL) pages (See Part 4 for more information on these) across multiple NIEs and I knew that mere numbers of uses of the word “likely”, for example, could be skewed by the length of the estimate (i.e. the longer the estimate, the more times a certain word would probably be used). I also considered it unlikely that Words of Estimative Probability (WEPs) and other special words (as defined by the EEL page) would be used multiple times in a single sentence. Number of sentences, therefore, while not perfect, seemed to me to be a useful denominator.
If the number of sentences is the denominator, what about statements of confidence, the numerator? I generated this number by searching each of the Judgments for each of the words or phrases highlighted in the EELs and then going back through and reading for words that might serve the same purpose but were not specifically mentioned in the EEL. Looking at the number of sentences with explicit levels of confidence in each of the identified estimates tells a startling story:
What immediately jumps out is the number of sentences that contain statements of confidence in the Iran NIE – 19 out of the 31 total sentences or almost 2/3 of the sentences in the Iran NIE – compared with other NIEs. None come even close and the majority of them (including the three other NIEs that contained EEL pages that specifically said expressions of confidence were going to be used) contain none. Not one.
Frankly (and abandoning any pretense of academic detachment for a minute) this stuns me. I have a good deal of respect for the the analysts in the NIC and throughout the IC but what were they thinking when they put the previous NIEs in 2007 together? That no one would notice? If that is the case, they were probably correct. Certainly I have not seen a single critique that highlights the absolute inconsistency of stating, “We intend to assess confidence” and then not doing it – three times in a row. That they never said they had to use statements of confidence? That is quibbling. The whole purpose of the EEL is to define “What We Mean When We Say...”. It’s like including a Spanish glossary at the front of a Chinese textbook and then saying, “We never said we had to write the textbook in Spanish.” Why make a point of including an explanation of statements of confidence and then not using them at all? That the expressions of confidence are in the classified but not the unclassified version? That makes even less sense and I don’t think the IC is that dumb. That they had never done it before? Nonsense. The Iraq WMD estimate contained explicit references to confidence. I'm sorry, but three times in a row is too strong of a pattern to ignore. The failure to do what the Intelligence Community said it would do was intentional.
The worst case scenario is that the IC suspects that no one is reading these things anyway or, if they are, they believe the readers are only going to cherry-pick the parts that serve their policy or political purposes. In this context, carefully nuancing your statements and enforcing strict consistency, indeed any consistency, in your use of words is just wasted effort. This is a cynics-eye view yet, sadly, the evidence seems to support it.
That is a shame. Many very smart, dedicated people work in the IC. Many of them are putting their lives on the line to collect, process and analyze the information our decisionmakers need to make good decisions. Certainly the taxpayer has borne a not insignificant burden funding it. The IC's work should be applause-worthy but saying one thing and doing another is not a cause for either confidence or approbation.
Water under the bridge, at this point, of course. The Iran NIE obviously fixed all that, you might say.
Not so fast.
Tomorrow: Part 6 -- Digging Deeper
1 comment:
There is another variable to consider, which is whether the NIE stuck to such safe predictions that no qualification was needed. For example, the wholly unqualified NIE of 1990 regarding Yugoslavia was right on as to what it predicted. On the other hand, it offered no predictions regarding deeper matters, such as the likelihood and violence of the Bosnian civil war, of the prospective siege of Sarajevo, and of prospects that American and NATO forces would be drawn in, the potential for drawing in Islamic fighters nor for reviving "Pan Slavism" in Russia and other Slavic states. It gained certainty by settling for very high level generality.
Post a Comment