Can You Balance The US Budget? Can You Make Decisions Like A Chinese Official? (American Public Media And Wikiversity)

With all the brouhaha concerning DIA's recent announcement concerning the creation of several video games to help train new analysts, I thought it would be worth mentioning a couple of other "serious games" I have run across (one yesterday and one quite some time ago).

The first, Budget Hero, is a flash based game (see screenshot below) that lets you pick options to help balance the US Federal Budget. Basically, after a short intro brief, you get to pick tax and spending policies in order to try to accomplish your goals without busting the budget. Once you have selected your series of policies, the game calculates how well you have done against a couple of benchmarks and then reports back to you on how well you have accomplished your goals and how well in tune you are with others who have played the game.

The game really gives you a good idea of the relative values of different policies. It also highlights the fact that some policies are important for their emotional impact but have little real impact on the total budget (By the way, if this game is at all accurate, the next administration will have its work cut out for it. The kind of answers that tend to work are not going to be very palatable). The best I have been able to do is to keep the government afloat until 2044. If anyone figures out a way to beat that, drop a comment.

The second game is one I have been meaning to blog about for some time. One of our former students (he just graduated), Pat Noble, an analyst with the FBI, put together an interesting simulation of Chinese political decisionmaking processes called Forbidden Kingdom (he did this last year in my Advanced Analytic Techniques class so, no, there is no relation to the movie of the same name). The screenshot below shows the game board (which you have to build) and the rules and all the other materials you need are available for free over at Wikiversity.

All simulations have to balance playability with realism. Pat's challenge was to make a game that captured some of the essence of modern Chinese political thinking in a short, interesting game. As someone who watched the research and development of the game, has played the game and heard countless stories about others playing the game, I think Pat accomplished just that.

Excellent "Lesser Taught" Language Resource (DLI via DangerRoom)

Wired.com's DangerRoom featured an excellent language resource from the Defense Language Institute for some of the lesser known (only because they are lesser taught in the US) languages. Not everything on the site is accessible to non-military but the stuff that is available is very good. In addition to the fine language resources, I was particularly impressed with the "Countries In Perspective" section that provides short, descriptive country studies on about 20 countries.

China's Demographic Problems, India As A Counterbalance To China And A Revolution In Political Economy Against Jihad (Fora.tv)

A few recent gems from Fora.tv:

Nicholas Eberstadt: China's Looming Demographic Troubles
AEI Fellow Nicholas Eberstadt discusses the effect of an aging population on China's workforce, the skewed gender demographics and its implications, and the future of China's social capitol.
Date: Mon, 28 Apr 2008
Location: Hoover Institution, Stanford, CA,
http://fora.tv/2008/04/28/Nicholas_Eberstadt_China_s_Looming_Demographic

Mark Helprin on India Counterbalancing China
Mark Helprin answers whether India, an up and coming power in the world, will act as a counterbalance to China.
Date: Mon, 28 Apr 2008
Location: Hoover Institution, Stanford, CA,
http://fora.tv/2008/04/28/Mark_Helprin_on_India_Counterbalancing_China

A Revolution in Political Economy Against Jihad
Walid Phares calls for a revolution in political economy in the war against jihad.
Date: Wed, 23 Apr 2008
Location: Foundation for the Defense of Democracies, Washington D.C.,
http://fora.tv/2008/04/23/A_Revolution_in_Political_Economy_Against_Jihad

Saying One Thing And Doing Another: A Look Back At Nearly 60 Years Of Estimative Language (Original Research)

US News and World Reports has an interesting story about the current state of intelligence reform. According to the article, CIA Director, Mike Hayden, said,

• "Some months ago, I met with a small group of investment bankers and one of them asked me, 'On a scale of 1 to 10, how good is our intelligence today?'" recalled Hayden. "I said the first thing to understand is that anything above 7 isn't on our scale. If we're at 8, 9, or 10, we're not in the realm of intelligence—no one is asking us the questions that can yield such confidence. We only get the hard sliders on the corner of the plate. Our profession deals with subjects that are inherently ambiguous, and often deliberately hidden. Even when we're at the top of our game, we can offer policymakers insight, we can provide context, and we can give them a clearer picture of the issue at hand, but we cannot claim certainty for our judgments."
  

(For those of you keeping score at home, Hayden said much the same thing last year during an interview with CSPAN...)

Frankly, I don't know anyone knowledgeable about the strengths and weaknesses of intelligence that doesn't agree with this statement. Certitude is impossible. That is what makes the chart below so darn interesting:


The chart is from Rachel Kesselman's recently completed thesis, Verbal Probability Expressions In National Intelligence Estimates: A Comprehensive Analysis Of Trends From The Fifties Through Post 9/11. The chart shows the number of times the word "will" has been used, in an estimative sense (e.g "X will happen"), in the Key Judgments of 120 National Intelligence Estimates (NIE) over the last 58 years (20 per decade) that she examined.

In fact, at 717 times, the word "will" was the single most commonly used estimative word, by a very large margin, in NIEs. Not only was it the single most commonly used word, it was also one of the most consistently used words across the decades (tests Rachel ran showed that the variances across the decades were not statistically significant).

So...if certitude is impossible, why does the Intelligence Community use "will" -- a word that reeks of certitude -- so often in its estimates? Such a result is absolutely inconsistent with statements, such as Hayden's above, made by virtually everyone who has ever jumped up to defend intelligence's predictive track record.

This was only one of the many fascinating results that came out of Rachel's exhaustive study of the words that analysts have used over the years to verbally express probabilities

Rachel's lit review, for example, makes for very interesting reading. She has done a thorough search of not only the intelligence but also the business, linguistics and other literatures in order to find out how other disciplines have dealt with the problem of "What do we mean when we say something is 'likely'..." She uncovered, for example, that, in medicine, words of estimative probability such as "likely", "remote" and "probably" have taken on more or less fixed meanings due primarily to outside intervention or, as she put it, "legal ramifications". Her comparative analysis of the results and approaches taken by these other disciplines is required reading for anyone in the Intelligence Community trying to understand how verbal expressions of probability are actually interpreted.

Another of my favorite charts is the one below:


This chart examines the use of the NIC's nine currently "approved" words of estimative probability (See page 5 of this document for additional discussion) across the decades. The NICs list only became final in the last several years so it is arguable whether this list of nine words really captures the breadth of estimative word usage across the decades. Rather, it would be arguable if this chart didn't make it crystal clear that the Intelligence Community has really relied on just two words, "probably" and "likely" to express its estimates of probabilities for the last 60 years. All other words are used rarely or not at all.

Based on her research of what works and what doesn't and which words seem to have the most consistent meanings to users, Rachel even offers her own list of estimative words along with their associated probabilities:


Rachel's work tracks well with my own examination of word usage in recent NIEs and with some of the findings in Mike Lyden's thesis on Accelerated Analysis, but her thesis really stands on its own and my brief description and summary of some of the highlights does not do it justice. It is a first-of-its-kind, longitudinal study of estimative word usage by the intelligence community and has contributed significantly to my own understanding of where the Intelligence Community has been over the last 58 years. I think readers of this blog will be more than a little interested in her results and recommendations as well.

Related Posts:
The Revolution Begins On Page Five...
Accelerated Analysis: A New And Promising Intelligence Process
What Do Words Of Estimative Probability Mean?

Great...Well, Funny, At Least...Commencement Addresses

With Commencements of all kinds in the air, I thought it would be fun to list a couple of the more "interesting" commencement addresses I have run across surfing the web:

John Stewart (2004 -- William and Mary)
Conan O'Brien (2000 -- Harvard)
Stephen Colbert (2006 -- Knox College)