Extremely Cool Interactive Breakdown Of The UK's Intelligence Community

Nick Hare (the Head of Futures and Analytic Methods at the UK Ministry of Defense) and his team of analysts put together this interesting presentation that breaks down the UK Intel community.  He notes in the LinkedIn post where he first announced it that "The sizes of the parts roughly correspond to numbers of people and / or budgets."

Good stuff!

2nd Annual Mercyhurst Report On Hiring Prospects For Entry Level Intelligence Analysts In The National Security Community Is Out! (And It Is Not Pretty...)

http://www.iismu.org/white-papers

Sequester, draw-down, RIF, early retirement - these are the buzzwords that are dominating the hiring discussions in the US government these days.  While last year was marginal, the next 12 months are shaping up to be very tough ones for entry-level intelligence analysts trying to break into the US national security intelligence community. 

Greg Marchwinski, one of our current crop of graduate student all-stars, prepared this year's report which, like last year's report, is based on collected survey data, interviews and emails from knowledgeable individuals, and relevant secondary sources. 

Here is what Greg has to say about the prospects for the next 12 months (from the executive summary):

"Due to uncertainty over federal government deficit reduction initiatives and a decreasing military presence globally, it is highly likely that overall hiring of entry-level intelligence analysts within the US Intelligence Community (IC) will decrease significantly from recent levels until the next budget cycle begins in October, 2013. The only exception to this general trend is cyber-related positions which are likely to see a moderate increase despite budget cuts. Additionally, it is highly likely that sequestration throughout the IC will significantly limit hiring entry-level intelligence analysts in all analytic functions until defense funding negotiations are resolved."

Click on this link to download the full report.

Look for our reports on entry-level intel analyst hiring in law enforcement and in business next month.

Part 6 -- So, How Did It All Work Out? (Teaching Strategic Intel Through Games)

Part 1 -- Introduction

Part 2 -- What Is Strategy And What Are Strategic Decisions?

Part 3 -- What Is Intelligence And What Is The Role Of Intelligence In The Formulation Of Strategy? 

Part 4 -- Why Games? 

Part 5 -- How, Specifically, Were Games Used in Class?

While mostly anecdotal, the available evidence suggests that students significantly increased their ability to see patterns and connections buried deeply in unstructured data sets, my first goal.  This was particularly obvious in the graduate class where I required students to jot down their conclusions prior to class. 

An example of the growth I witnessed from week one to week ten would likely be helpful at this point.  The same student wrote both examples below and I consider this example to be representative of the whole:

Week 2 Response – “This game (The Space Game: Missions) was predominantly about budgets and space allocation….  Strategy and forethought go into where you place lasers, missile launchers and solar stations, so that you don’t run out of minerals to power those machines and so repair stations are available for those that are rundown.  It’s clear that resource and space allocation are key elements for a player to win this game, just as it is for the Intelligence Community and analysts to win a war.”

Week 8 Response:  “I think if Chess dropped acid it’d become the Thinking Machine. When the computer player was contemplating its next move colorful lines and curves covered the board… To me, Chess was always a one-on-one game; a conflict, if you will, between black versus white… Samuel Huntington states up front that he believes that conflict will no longer be about Princes and Emperors expanding empires or influencing ideologies, but rather about conflicts among different cultures:  “The great divisions among humankind and the dominating source of conflict will be cultural.” Civilizations and cultures are not black and white, however; they’re not defined by nation-state borders. There are colors and nuances in culture requiring a change in mindset and in strategy to approach these new problems.”

While difficult to assess quantitatively, literature from the critical thinking community helps assess the degree of change here.

Note:  There is a widespread belief among intelligence professionals that teaching critical thinking methods will improve intelligence analysis (See David Moore’s comprehensive examination of this thesis in his book Critical Thinking And Intelligence Analysis).   A minority of authors are less willing to jump on this particular bandwagon (See Behavioral and Psychosocial Considerations in Intelligence Analysis: A Preliminary Review of Literature on Critical Thinking Skills by the Air Force Research Laboratory, Human Effectiveness Division) citing a lack of empirical evidence pointing to a correlation between critical thinking skills and improved analysis.

In particular, the Guide to Rating Critical Thinking developed by Washington State University identifies seven broad categories for assessing if and to what degree critical thinking is taking place: 

-          Identification of the question or problem

-          Willingness to articulate a personal position or argument

-          Willingness to consider other positions or perspectives

-          Identification and assessment of key assumptions

-          Identification and assessment of supporting data

-          Considers the influence of context on the problem

-          Identifies and assesses conclusions, implications and consequences

While such a list may not be perfect, there is certainly nothing on it that is inconsistent with good intelligence practice.  Likewise, when reading the representative example above with these criteria in mind, the increase in nuance, the willingness to challenge an acknowledged authority, the nimble leaps from one concept to another all become even more obvious. The growth evident in the second example is even more impressive when you consider that the Huntington reading was not required.  The majority of the students in the class showed this kind of growth over the course of the term both in the quality of the classroom discussions and in their written reports.

In addition to seeing an improvement in students’ ability to detect deep patterns in complex and disparate data sets, I also wanted that increased ability to translate into better quality intelligence products for the decisionmakers who were sponsoring projects in the class. 

Here the task was somewhat easier.  I have solicited and received good feedback from each of the decisionmakers involved in the 78 strategic intelligence projects my students have worked on over the last 7 years.  This feedback, leavened with a sense of the cognitive complexity of the requirement, yields a rough but useful assessment of how “good” each final project turned out to be.

Mapping this overall assessment onto a 5 point scale (where a 3 indicates average “A” work, a 2 and 1 indicates below and well below "A" work respectively, a 4 indicates "A+" or young professional work and a 5 indicates professional quality work), permits a comparison of the average quality of the work across various years.  

Note:  “A” is average for the Mercyhurst Intelligence Studies seniors and second year graduate students permitted to take Strategic Intelligence.  In order to be employable in this highly competitive field, the program requires students to maintain a cumulative 3.0 GPA simply to stay in the program and encourages students to maintain a 3.5 or better.  In addition, the major is widely considered to be “challenging” and those who do not see themselves in the career of intelligence analysis, upon reflection, often change majors.  As a result, GPAs of the seniors and second year graduate students who remain with the program are often quite high.  The graduating class of 2010, for example, averaged a 3.66 GPA.

The chart above summarizes the results for each year.  While the subjectivity inherent in some of the evaluations possibly influenced some of the individual scores, the size of the data pool suggests that some of these variations will be eliminated or at least smoothed out through averaging.

There are, to be sure, a number of possible reasons to explain the surge in quality evidenced by the most recent year group.  The students could be naturally better analysts, the quality of instruction leading up to the strategic course could have dramatically improved, the projects could have been simpler or the results could be a statistical artifact.

None of these reasons, in my mind, however, hold true.  While additional statistical analysis has yet to be completed, the hypothesis that games-based learning improves the quality of an intelligence product appears to have some validity and is, at least, worthy of further exploration.

My third goal for a games-based approach was to better lock in those ideas that would likely be relevant to future strategic intelligence projects attempted by the students, most likely after graduation.  To get some sense if the games-based approach was successful in this regard, I sent each of the students in the three classes a letter requesting their general input regarding the class along with any suggestions for change or improvement.  I sent these letters approximately five months after the undergraduate classes had finished and approximately 2.5 months after the end of the graduate class.

Seventeen of the 75 students (23%) who took one of the three courses responded to the email and a number of students stopped by to speak to me in person.  In the end, over 40% of those who took the class responded to my request for feedback in one way or another.  This evidence, while still anecdotal, was consistent – games helped the students remember the concepts better.

Comments such as, “Looking back, I can remember a lot of the concepts simply because the games remind me of them” or “I am of the opinion that the only reason that the [lessons] stood out was because they were different from any other class most students have taken” were often mixed in with suggestions on how to improve the course.  The verbal feedback was even more encouraging, with reports of discussions and even arguments centered on the games and their “meaning” weeks and months after the course was completed.

The evidentiary record, in summary, is clearly incomplete but encouraging.  Games–based learning appears to have increased intelligence students’ capacity for sensemaking, to have improved the results of their intelligence analysis and to allow the lessons learned to persist and even encourage new exploration of strategic topics months after the course had ended.

Next:  
What did the students think about it?

How Large Is The World's Intelligence Industry? Now We Know... (Thesis Month)

One of the questions that has really bothered me over the years concerns the size of the national security intelligence "industry" worldwide. When you add it all up, how much money do the states of the world spend on intelligence and how many people are involved in government intelligence work?

These questions are important. There is a popular impression in much of the world that "intelligence is everywhere", that it is both all powerful and omnipresent. Creating or encouraging this impression in dictatorial countries might even be part of the system of repression. Knowing the answers to these questions could help reformers more accurately assess their risks.

Even in democratic countries, however, understanding the resource limits of the national security intelligence apparatus at the broadest possible levels, where the need for citizens to know where their money is being spent can be appropriately balanced with the legitimate operational concerns of the working intelligence professional, seems to make sense.

From a more provincial standpoint, it also seems important for educational institutions to have some sort of a feel for the need for trained professionals in intelligence work if the university model is ever going to supplant government training as the primary way into the intelligence communities of the world.

The answer to this question, however, is obviously difficult to uncover. Most countries do not want to discuss how much they spend on intel each year. Oftentimes, it is even difficult to figure out which organizations within a country are actively engaged in intelligence work.

It is with great pleasure, then, that I announce the final results:

The national security intelligence industry accounts for about $106 billion dollars a year and employs about a million people worldwide.

These are the numbers generated by Chris Hippner in his interesting and exhaustive thesis titled, A Study Into The Size Of The World's Intelligence Industry.

While Chris has done a good (extraordinary, really) job of collecting as many facts and figures as he could regarding the intel budgets of every country on the planet, he had to rely on estimates for many of them.

These estimates are based on GDP and on the spending patterns of countries where the data is available, a method which Chris readily admits is fraught with some difficulty (I note with some interest, though, that Chris has posted a note to his online thesis encouraging people to send him more accurate figures. It will be interesting to see how many people take him up on the offer...).

I am also sure that Chris has missed some organizations. It is virtually certain that there are organizations out there which are well known to people living in a particular country to be wholly controlled by that country's intelligence apparatus for which Chris has not accounted. Such errors are essentially unavoidable given the global scope of his thesis work.

Likewise, Chris simply did not have time to examine either the growing presence of intelligence units in law enforcement or business (My own guess is that this would approximately double the total value of the industry).

All that said, this thesis does exactly what needed to be done -- give us all a starting point for further research and refinements.

A full copy of the thesis is located below or you can go to Chris's site on Scribd.com for other viewing and download options.

A Study Into the Size of the World's Intelligence Industry

Mark Your Calendars! US Intelligence Community's Virtual Career Fair 16 March! (ICVIRTUALFAIR.com)

The US National Security Intelligence Community (IC) is about to hold a virtual career fair on 16 March, 2010. Nine of the largest agencies within the US IC will be represented and registration is free and open to anyone (though only US citizens can get jobs in the US IC).

Career opportunities in the following fields will be available:

• Cybersecurity/Information Assurance
• Engineering and Physical Science
• Foreign Languages (Arabic, Dari, Farsi, Mandarin, Pashto, Somali, or Urdu are the top choices)

• Information Technology
• Intelligence Analysis
• Law Enforcement
• Many others (though not specified on the Virtual Career Fair website).

Registration is easy. Just go to the registration page and fill in the blanks (Note: When it asks for "Event Login ID" and "Event Password" those are things that you make up, not things you need to get from the event organizers):

There are some browser and plug-in issues so be sure to click on the Computer Requirements link to check out your system. Also don't rely on the text alone. Make sure you click on the System Check link. For example, the text says Firefox 1.5 and above is supported but when I clicked on the Check System link it told me my Firefox 3.5 system was no good.

According to some of the press releases about the Fair, if you can't make it on the 16th, the site will still be up for 90 days after the event but there won't be any recruiters there to chat with.

Heuer: How To Fix Intelligence Analysis With Structured Methods (NationalAcademies.org)

Richards Heuer (of Psychology Of Intelligence Analysis fame...) spoke last month at the National Academy of Sciences regarding his thoughts on how to improve intelligence analysis through the increased use of structured methods.

In the aftermath of the attempted bombing of Flight 253 on Christmas Day, it is worth reading Dick's words on how to improve the analytic side of the intelligence equation. I don't agree with everything he says (and say so in italicized parenthetical comments below) but he has been thinking clearly about these kinds of things for far longer than most of us. If you are concerned at all with reforming the way we do analysis, then this is a much better place to start than with all the noise being generated by the talking heads on TV.

I have embedded the whole document below this post or you can go to the National Academies site and listen to Dick's speech yourself. For those of you with too much to do and too little time, I have tried to pull out some of the highlights from Dick's paper below. I am not going to do it justice though, so, if you have the time, please read the entire document.

• "If there is one thing you take away from my presentation to day, please let it be that structured analytic techniques are enablers of collaboration. They are the process by which effective collaboration occurs. Structured techniques and collaboration fit together like hand in glove, and they need to be promoted and developed together." (This is very consistent with what we see with our students here at Mercyhurst. Dick reports some anecdotal evidence to support his claim and it is exactly the same kinds of things we see with our young analysts).
  

• "Unfortunately, the DNI leadership has not recognized this. For example, the DNI’s National Intelligence Strategy, Enterprise Objective 4 on improving integration and sharing, makes no mention of improving analytic methods."

• "CIA, not the DNI, is the agency that has been pushing structured analysis. One important innovation at CIA is the development in various analytic offices of what are called tradecraft cells. These are small groups of analysts whose job it is to help other analysts decide which techniques are most appropriate, help guide the use of such techniques by inexperienced analysts, and often serve as facilitators of group processes. These tradecraft cells are a very helpful innovation that should spread to the other agencies." (Interesting. We called these "analytic coaches" and tried to get funding for them in our contract work for the government in 2005 -- and failed).
  

• "I understand you are all concerned about evaluating whether these structured techniques actually work. So am I. I’d love to see our methods tested, especially the structured analytic techniques Randy and I have written about. The only testing the Intelligence Community has done is through the experience of using them, and I think we all agree that’s not adequate." (I suppose this is the comment that bothers me at the deepest level. It implies, to me, at least, that the IC doesn't know if any of its analytic methods work. What other 75 billion dollar a year enterprise can say that? What other 75 billion dollar a year enterprise wants to say that?)
  

• "Some of you have emphasized the need to test the accuracy of these techniques. That would certainly be the ideal, but ideals are not always achievable." (Here I have to disagree with Dick. Philip Tetlock and Bruce Bueno De Mesquita have both made progress in this area and there is every reason to think that, with proper funding, such an effort would ultimately be successful. Tetlock recommended as much in a recent article. The key is to get started. The amount of money necessary to conduct this research is trivial compared to the amount spent on intel overall. Likewise, the payoff is enormous. As an investment it is a no-brainer, but until you try, you will not know)
  

• "Unfortunately, there are major difficulties in testing structured techniques for accuracy, (for an outline of some of these, see my series of posts on evaluating intelligence) and the chances of such an approach having a significant favorable impact on how analysis is done are not very good. I see four reasons for this."

• "1. Testing for accuracy is difficult because it assumes that the accuracy of intelligence judgments can be measured."

• "2. There is a subset of analytic problems such as elections, when a definitive answer will be known in 6 or 12 months. Even in these cases there is a problem in measuring accuracy, because intelligence judgments are almost always probabilistic."

• "3. A third reason why a major effort to evaluate the accuracy of structured analytic techniques may not be feasible stems from our experience that these techniques are most effective when used as part of a group process."

• "4. If you are trying to change analysts’ behavior, which has to be the goal of such research, you are starting with at least one strike against you, as much of your target audience already has a firm opinion, based on their personal experience that they believe is more trustworthy than your research." (Here I have to disagree with Dick again. I think the goal of this research has to be to improve forecasting accuracy. If you can show analysts a method that has been demonstrated to improve forecasting accuracy -- to improve the analyst's "batting average" -- in real world conditions, I don't think you will have any problem changing their behavior.)

• "As with the other examples, however, the Intel Community has no organizational unit that is funded and qualified to do that sort of testing."
  

• "It (a referenced Wall St. Journal article) suggested that instead of estimating the likelihood that their plans will work, financial analysts should estimate the probability they might fail. That’s a good idea that could also be applied to intelligence analysis." (I am not sure why we can't do both. We currently teach at Mercyhurst that a "complete" estimate consists of both a statement of probability (i.e. the likelihood that X will or will not happen) and a statement of analytic confidence (i.e. how likely is that you, the analyst, are wrong in your estimate.)
  

• "The kind of research I just talked about can and should be done in-house with the assistance of those who are directly responsible for implementing the findings." (I think that Dick is correct, that, at some point, it has to be done in-house. I do think, however, that the preliminary testing could be effectively done by colleges, universities and other research institutions. This has three big benefits. First, it means that many methods could be tested quickly and that only the most promising would move forward. Second, it would likely be less expensive to do the first stage testing in the open community than in the IC. Third, it allows the IC to extend its partnering and engagement activities with colleges, universities and research institutions.)
  

• "Our forthcoming book has two major recommendations for DNI actions that we believe are needed to achieve the analytic transformation we would all like to see."

• "1. The DNI needs to require that the National Intelligence Council set an example about the importance of analytic tradecraft. NIC projects are exactly the kind of projects for which structured techniques should always be used, and this is not happening now."

• "2. The second recommendation is that the DNI should create what might be called a center for analytic tradecraft."

Complete text below:

The Evolution of Structured Analytic Techniques -- Richards Heuer -- 8 DEC 2009