Sources And Methods

Mercyhurst College Is Now Mercyhurst University (WOOT!)

2012-01-26T14:57:00.000-05:00

For the last several years Mercyhurst has been going through the administrative process of changing our status from a "College" to a "University". Yesterday, we received notice from the State of Pennsylvania that our application had been approved.

(Note: I am going to talk about what I think this means and some of the new things that are likely to emerge as a result but I am also interested in what you have to say. so please leave a comment if you have one!)

This process actually began even before the first piece of paperwork went in. We changed our Carnegie Classification a number of years ago to better represent the courses, degree programs and research opportunities we offered.

Now that the status issue is resolved, we are all dealing with the inevitable administrative details that result from such a change, the first of which is our name. The Mercyhurst College Institute for Intelligence Studies (MCIIS) is likely to change to the overwhelmingly favored Institute for Intelligence Studies at Mercyhurst University (IIS-MU) at some point. The institute's website (mciis.org) will likely get an upgrade at about the same time.

More interesting than the administrative details are some of our longer range initiatives. The University (going to take me awhile to get used to writing that...) intends to pursue doctorate programs in some of our majors, for example. Our hope is to be one of the first. I don't think we want to pursue a PhD type program, however, as there are already a number of PhD programs that would allow a student to focus on the academic aspects of studying intelligence.

Rather, I think we should, like MDs and JDs, offer a professional doctorate (DI? ID? InD?). The focus of such a degree, if I had my way, would be on application -- the actual doing of intelligence analysis -- rather than just talking about it. Specifically, I would like us to provide our doctoral students a chance to become better leaders and managers in addition to gaining increased skills as analysts.

We are obviously some distance from this objective but the conversation about the future direction of the intelligence studies program is starting. Now is the time to chime in!

What Makes An Easy Question Easy? (DAGGRE.org)

2012-01-09T08:30:00.000-05:00

For most of last year I have had the privilege of working with the DAGGRE Team on the Intelligence Advanced Research Projects Activity's (IARPA's) Aggregative Contingent Estimation (ACE) Project. While all the real scientists have been busy exploring research questions involving Bayesian networks and combinatorial markets, this old soldier has been focusing on more mundane things like "What makes an easy question easy?"

(Note: If you have not had a chance to check out the DAGGRE.org site and its mind-numbingly cool companion blog, Future Perfect, you should. Three reasons: First, it is pretty interesting research that could impact the future of the intel community. Second, you can actually participate in it. Third (and maybe most important to many of the readers of this blog), the DAGGRE team has gone the extra mile to make sure your personal data, etc is secure while participating (check out the FAQ page for all the details)).

As I explained in this post, having some way to evaluate and even rank intelligence reqyuirements according to difficulty is important. Analysts are supposed to be accurate but if you aren't also evaluating the difficulty of the underlying question, two equally accurate analysts could be miles apart in terms of overall quality. It would be kind of like saying a little leaguer who hits .400 is as good as a major leaguer hitting .400.

While that distinction is easy to see in baseball, it is much more difficult in intelligence analysis. Questions come in all shapes and sizes and vary in an enormous number of ways. There is also a psychological, subjective aspect to it: Questions that seem tough to some analysts may seem very easy to others. On its face, it appears difficult if not impossible to come up with a system that can reliably evaluate and categorize questions by difficulty level.

Which is why I want to try.

And I may be making some progress. I think I have figured out how to spot an "easy" question. DAGGRE, you see, is a predictive market. This means that people assign probabilities to the outcomes of questions. Imagine, for example, I asked if Sarkozy would still be the president of France on 1 JUN 2012 (He is running for re-election in April and May). Now imagine that you thought the odds of Sarkozy's re-election were 80%. You could establish your position in the market at that "price" and others would be free to do the same (The Iowa Electronic Markets do this for the US election, by the way).

The market would reward people that were right on 1 JUN and would heavily reward those that were right when lots of others were wrong. Studies have shown that, on average, these types of markets are pretty good at making these kinds of estimates.

Now, imagine if I asked you to estimate the chances that Sarkozy would still be president of France by 1700 tomorrow? Sarkozy is not sick (at least I hope he is not) and there are no direct, immediate threats to his presidency. There is no reason to expect that he would not still be president tomorrow. Likewise, successfully predicting that he will still be in office tomorrow is no sign of great analytic ability. The question is too easy.

Generalizing this pattern, I think it is worth exploring the idea that "easy" questions are those that start and end their run on a predictive market close to either 0% or 100% probability, do not vary much during the course of that run and, finally, resolve in accordance with their probabilities (i.e. they happen if close to 100% and don't happen if close to 0%. See the picture that accompanies this post for an idea of what such patterns might look like). Furthermore, I think that these kinds of questions will see much less trading activity than other ("not easy") questions.

Of course, the problem with this definition is that it only identifies easy questions after the fact, after the question has been resolved. My hope, however, is that by examining the set of questions that we already know are easy (at least under this definition), that we might be able to see other patterns that will allow us to identify easy questions when they are asked rather than only after they are answered.

Our (I say "our" because I am working on this with one of our superb grad students, Brian Manning) first attempt to get at these patterns will be a simple one -- question length. We hypothesize that, on average, questions that match the "easy" pattern I described above will be shorter than other questions. When you think about it, it makes some sense. After all, "What time is it?" seems like an easier question to answer than "What time is it in Nigeria?"

Brian has found some research that says that, subjectively, people don't perceive longer questions as necessarily more difficult. The difference, of course, is that we have a definition of "easy" that is based on objective criteria. Still, I think it best to start with the easiest possible measurement and then go from there. Not sure where I will end up or if this will be a dead end but I will keep you posted...

Is ISI Really The Best Intelligence Agency In The World?

2011-12-14T11:57:00.001-05:00

http://nationalpostnews.files.wordpress.com/2011/11/spies.gif

According to the National Post, Canada's conservative newspaper, it is.

That is just one of the interesting tidbits reported in this graphic titled, The State of The Global Spy Game (Download the PDF here).

Following Pakistan's ISI comes Mossad in the number 2 slot, MI 6 taking third and the CIA following up in fourth place.

In addition to the Top Ten list, most of the graphic outlines a series of assassinations, explosions, spying, cyber spying and "convenient accidents" that the Post ties to various intelligence organizations over the last ten years.

Finally, there are some charts which claim to be based on some of Richards Heuer's work regarding the demographics of spies; where they come from in government, how much education they have, etc. The graphic provides no comparative data to see if any of the categories identified are larger or smaller than they are in the relevant population from which they are drawn so it is difficult to draw conclusions but intriguing nonetheless.

Given the nature of the article and difficulty associated with making these kinds of judgements, I am not surprised at the results but it is still an interesting question to ask: Who has the world's best intel service?

(Hat tip to Christophe Deschamps at Outils Froid and his must follow Twitter feed!)

1st Annual Entry-level Intel Analyst Jobs Report Out Now!

2011-12-09T10:12:00.001-05:00

How good is the job market for entry-level intelligence analysts over the next 12 months?

Good question, right? If you are a recent graduate from college or you are graduating in 2012, and you are interested in working as an analyst in the US national security intelligence community, it is probably one of the questions you are asking yourself.

The answer is contained in this document.

We tasked one of our outstanding grad students, Whitney Bergendahl, with examining this question back in the early fall. He put together a survey (which some of you may remember) and conducted some fairly extensive secondary research to put together this report. It is obviously a tough nut to crack but Whitney has done yeoman's work on this first ever, job market report for entry-level intel analysts.

Whitney and I are both interested in your feedback, of course. After you have had a chance to read the report, please leave a comment!

But wait, there's more!

(I know that sounds cheesy but there is, in fact, more...)

This report only covers the job market for entry-level analysts in the national security intelligence community. Between now and the end of February, we hope to publish two other reports on the job markets for entry-level law enforcement intellience analysts and for entry level intelligence analysts in the business community.

But wait! There's even more!

(Had to do it...)

Benjamin Wittorf, who publishes occasionally on the blog, Netzwerk-Organisatorische Formen, but is probably best followed via Twitter and makes a living as a researcher for eVenture Capital Partners, has turned my series of blog posts, How To Get A Job In Intelligence, into an epub for easy (and free!) download. (Note: I am sure there is something clever I could say here about "the kindness of strangers" but I can't think of it so I will just say, Thanks, Ben!)

While this series is a little old, I think much of the general guidance ought to still be good. If you want to read it, you will, of course, need to have an epub reader to access it. If you don't, there is also a pdf version or, of course, you can still access the original series on this blog.

If You Think You Understand The Role Of Social Media In The Arab Spring Uprisings (And Particularly If You Don't), Watch This Video...

2011-12-06T11:39:00.001-05:00

This video is an hour long and, frankly, I didn't think I would have the time this morning. Started watching nonetheless and became riveted by one of the most cogent explanations of the role of social media in activism I have heard. Even if you disagree (and this is not my area of expertise so I hope those that do disagree will do so in the comments so we can all learn), it is well worth the hour it takes to watch.

(Many thanks to my friends at Sharp for this!)

Testing A New Mindmapping Tool...And You Can Join In! (Popplet.com)

2011-12-01T12:39:00.001-05:00

One of our amazing alums (Thanks, Justin!) sent me a link to a beta version of a new, web-based, collaborative mindmapping tool called Popplet.

I have been playing around with it for the last half hour or so and found it easy to use and potentially very useful.

What I don't know is how well it works as a collaborative tool (which is where my real interest lies -- there is plenty of good stand-alone mindmapping software), so I thought I would throw it out there for anyone to examine (Popplet makes this simple with an embed code. Yeah!).

If you are interested in trying it out, however, you will need to drop me an email (kwheaton at Mercyhurst dot edu) and I will send you an invite.

These kinds of open, collaborative tools that are easy to set up and quick to learn are great for classroom exercises; they are interactive and engaging. In my experience, students love them (If you are looking for another example, try Willyou.typewith.me)

It is also a great way to build a mental model of an intelligence problem. This app is in beta though and has no way (that I could find) to safely share or export the data. There is an offline reader application called Popplet Presenter which would allow a single individual to show his/her work securely (-ish) to others, I suppose.

I suspect these features are coming (offering these features for a modest price is the way most of these kinds of apps, like Mindmeister or Webspiration, make their money) but until then, this is probably best confined to the classroom.

Welcome USA Today Readers!

2011-11-29T21:16:00.001-05:00

If you have found your way to my little slice of the internet today, it is probably because of this article in USA Today.

Hello!

If you are interested in learning more about my use of games in my intelligence studies classes, you will probably want to read my online article, Teaching Strategic Intelligence Through Games.

If you are interested in a specific example of a game and how I used it in class, you will probably find this article, Spot Report From The Future, to be worth reading. If you want all of the games-related posts, just search for "games" in the search bar at the top left of this page.

If you are interested in where I teach, you can find out more at the Mercyhurst website. If you are interested in the intelligence studies program at Mercyhurst, you can find out more about it on the Mercyhurst College Institute For Intelligence Studies website.

If you want to look at some of my other research or observations, just poke around this site. I post updates concerning most of the things I am working on here as well as some links to various projects on which my students have worked. Finally, if you have a question for me, please post a comment or drop me an email directly at kwheaton at mercyhurst dot edu.

Thanks for stopping by! You can subscribe to the RSS feed for this site directly at the link on the bottom right or through your RSS feed reader. If you use Twitter, you can follow all my postings @kwheaton.

Evaluating The Reliability Of Social Media Sources, An Amazing Technology Roadmap And Reagan's Visual SFARs (Link List)

2011-11-16T10:56:00.001-05:00

I love having friends and alumni that send me interesting links and this week contained an extraordinary crop! Here are three of the best things that happened to cross my desktop:

Some of the Ushahidi Deployments

How To Verify Social Media Content. We have known for some time how to evaluate online sources for credibility in a general sense (See Dax Norman's thesis and checklist here. Not only it is a brilliant piece of research, it is also the only such document designed by an intelligence analyst for use by other intelligence analysts). When it comes to understanding how to evaluate social media sources, however, the question becomes much trickier.

Patrick Meier is the Director of Crisis Mapping at Ushahidi and previously co-directed Harvard's Program on Crisis Mapping and Early Warning (If you are not familiar with the crisis mapping platform Ushahidi, stop now and go here). He has had extensive real-world experience with social media sources in the hundreds of uses of the Ushahidi platform in crises world-wide and he has translated that experience into an outstanding list of hints and tips for evaluating social media (Twitter specifcially).

While his insights into evaluating social media are born of this experience rather than more rigorous statistical analyses (like Dax's), his findings ring true and certainly operate as an excellent general purpose checklist until the science catches up.

http://envisioningtech.com/

Envisioning Emerging Technology For 2012 And Beyond. Through a series of serendipitous accidents, I have worked on a number of projects looking at technology trends.

While I normally start with Gartner on these types of questions, I have just added Michael Zappa and his excellent work at Envisioning Technology to my short list of go-to sources.

The technology roadmap he has built is awesome (you can see the compact version to the right but I strongly recommend you take a look at the interactive version here (Note to Michael Zappa: If you are going to make it Creative Commons, you might as well make it embeddable as well...Please!)).

Ronald Reagan: Intelligence and the End of the Cold War. Finally, I like to emphasize the importance of production skills for my students with a variety of stories about high-level decisionmakers who preferred their intelligence in "alternative" formats.

For example, John F. Kennedy had the President's Intelligence Checklist (the PICL -- analysts who worked on the product were said to work in the "PICL Factory"). Ronald Reagan, on the other hand, liked to see some of his intelligence, at least, in the form of short videos.

(Note: One of the kinds of analytic report writing we teach at Mercyhurst is called, generically, the Short Form Analytic Report, usually pronounced "Ess-Far". When this type of report contains more visual elements than written ones, we call it a visual SFAR, hence the title to this post).

Many have speculated that this was because Reagan was an actor and naturally gravitated to film but, whatever the reason, it is an interesting lesson in the importance of producing intelligence -- that is, the ability to fully communicate the results of analysis to the decisionmaker that the intel unit is supporting.

You can see the full report here or watch the videos on the CIA's YouTube channel (!). I have embedded my favorite (because I lived through it...) below:

An Internet Murder?

2011-11-11T11:51:00.001-05:00

I ran across this remarkable video today and it occurred to me that I might be able to use it to offer a brief but challenging thought experiment:

Given this video, what question would you ask first?

Leave your answer in the comments!

How To Improve Your Open Source Search Skills -- One Day At A Time!

2011-11-09T13:52:00.000-05:00

If you scratch the surface of the web you will find a ton of articles, websites and videos about how to search for information more efficiently.

Not that this makes any difference.

Students, according to the Project For Information Literacy, "while curious in the beginning stages of research, employed a consistent and predictable research strategy for finding information, whether theywere conducting course-related or everyday life research." The report goes on to say, "Almost all students used course readings and Google first for course-related research and Google and Wikipedia for everyday life research."

There are a number of reasons to be worried about this. Many people tend to focus on the over-reliance on Google as the search engine of first resort. While some of that logic is true (for an interesting and illuminating experiment that makes the point, I recommend this site...), I find the inability to use even Google very well to be one of my largest concerns. I mean, if you are going to rely only on Google, you ought to be incredibly good at finding stuff with it.

The best way, in my mind, to get good at finding stuff with Google is to practice doing it and Google apparently feels the same way. The Wizards of Mountain View are now offering a game that is designed to use "your creativity and clever search skills". The premise is simple. They ask a question and then you use Google to find the answer. The game is called "Google a Day". You can try today's challenge below:

I can easily imagine this as an icebreaker in a computer equipped classroom or as a homework or extra credit assignment. Leave your own ideas in the comments!

How Many Entry-level Analysts Does The US IC Need This Year? (Survey)

2011-11-08T12:09:00.000-05:00

Good question, right?

If you have direct knowledge of information that might help answer the question in the title or you have indirect knowledge that is relevant to the answer to the question in the title, please take 2 minutes to complete this survey.

What do I mean by direct and indirect knowledge?
Direct knowledge means that you know personally or have good information concerning the hiring plans of your agency or organization (or at least your section or division). You might work in HR or be a manager with hiring responsibilities.

Indirect knowledge is information that is relevant to the question that is not due to your direct responsibilities. You might have spoken with an HR manager or have been involved in meetings where this issue was discussed.

We are NOT looking for opinion based on purely circumstantial information. If you are not involved in the hiring process either directly or indirectly, please DO NOT take this survey.

Why are we interested?

Every year, other disciplines announce hiring projections for the year: "This year's hot jobs are for engineers and chimney sweeps." That sort of thing. Entry level intelligence analysts who are searching for a job, on the other hand, receive no such guidance.

We hope to change that. Working with one of our hot-shot grad students, Whitney Bergendahl, and my colleague and marketing expert, Shelly Freyn, we put together this survey to get a better feel for the the job market for entry level analysts for the year ahead.

Once we get enough survey data, Whitney will compile it and combine it with the macro-level, mostly qualitative data that we already have and put together a "jobs report" for the year ahead. I will publish it here once we are done.

We understand that there are some legitimate security concerns here so we have tried to frame the questions such that they are focused on broad developments and general trends. We are not interested in the kind of deep details that might compromise security.

Finally, we intend to follow this study up with similar surveys of the law enforcement and business job markets for entry-level intelligence analysts as well.

Thanks for your participation!

Two New "Intelligence In Business" Blogs Of Note

2011-11-01T10:55:00.000-04:00

Competitive Intelligence? Too narrowly defined. Strategic and Competitive Intelligence? Too long. Industrial Espionage? Illegal. Commercial Intelligence? Better but easy to confuse with the other two "CIs".

Whatever you tend to call it (and, as the title to this post indicates, I prefer "Intelligence In Business" or IIB), intelligence analysis as a function in the business world is growing very quickly. More and more companies are waking up to the idea that internally focused operational efficiency is not enough -- they need to better understand the world of events, people and organizations that are relevant to their success or failure but are outside their control as well.

To paraphrase a well known military thinker: "Know the competitor, the customer, the regulatory environment, your supply chain's vulnerabilities, the limits of your hiring pool, etc. and know yourself and you will succeed in all your battles."

That is a bit of a lengthy intro to two relatively new blogs that can help anyone new to the wild west of IIB understand it a bit better:

i-intelligence. There are few people I know who are both as articulate and consistently right as Chris Pallaris. His company, i-intelligence, is based in Switzerland but operates all over the world. He and his team do not post often (so don't worry about getting overwhelmed by spam) but it always makes for interesting reading. Of even more value, perhaps, is i-intelligence's twitter feed which can be (conveniently) found at @i-intelligence. There Chris and his team cull and curate a stream of articles that should be interesting to anyone involved in intelligence.

MGT Analytics. Whether it is making the theoretical real to the business person or just explaining the basics of intel, this new blog, run by Mike Thomas, looks like it is going to definitely be worth following. I have known Mike since he was a student here at Mercyhurst and have always been impressed with the clarity of his insight. More important than my intuitions, however, are Mike's wide range of real-world experience. He has known both success and failure so there will be no pollyanna-ish leanings here. Likewise, he has traveled extensively, taught accounting in China, gotten his masters, worked for the TSA, been a cyberthreat analyst and is now founding his second company. Clearly worth a look.

The Intelligence Analyst's Deck Of Cards Is Here! (Pictures And Update)

2011-10-26T11:08:00.000-04:00

The cards came in last week and look fantastic! Take a look at the unboxing pics below:

Cool, eh?

We should begin shipping out the complimentary copies to everyone who submitted a quote today. If you don't get your copy by 15 NOV, drop me a note (kwheaton at mercyhurst dot edu) with your mailing address.

Pre-orders made through the Mercyhurst Bookstore's online ordering service have already begun to be shipped. If you are interested in ordering fewer than 10 copies, you can continue to do so through the bookstore.

We are offering pretty sizable bulk discounts as well. If you want to order 10 or more copies (for all your training/morale building/Christmas needs!), I recommend you contact the increasingly harried editor of the Institute's Press, Domenic Vallone, at dvallo66@lakers.mercyhurst.edu

As a reminder, all profits from the sale of the cards will go to fund the activities of our three student intel clubs.

RFI: Should Intelligence Analysis Be More Like Competitive Diving?

2011-10-18T13:49:00.000-04:00

Quick! Which is more difficult: A jackknife or three and a half somersaults from a tuck position? In case you are not familiar with these dives, you can see videos of both below.

<p>&amp;amp;amp;lt;p&amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;lt;br&amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;lt;br&amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;lt;br&amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;amp;amp;amp;lt;br&amp;amp;amp;amp;amp;amp;gt;NO, &amp;amp;amp;amp;amp;amp;lt;br&amp;amp;amp;amp;amp;amp;gt;&amp;amp;amp;lt;/p&amp;amp;amp;gt;</p>

Now, here is the more difficult question: How much more difficult is a 3.5 from the tuck than a jackknife?

The answer is about 2.3 times more difficult. How do I know this? Because I checked out the handy diving tables at FINA (the international organization that regulates diving). I'm no expert but my reading of the tables says that a 3.5 from the tuck is a dive with a 3 point difficulty while a forward dive from the pike position (a jackknife?) is a 1.3 point dive.

Note that the degree of difficulty is simply a multiplier for the actual score of the dive. It is theoretically possible that a perfect jackknife would beat a lousy 3.5 somersault.

Intelligence, right now, is all about scoring the dive. Degree of difficulty? Not so much.

I am hoping to change that...

We spend a good bit of time in intelligence talking about forecasting accuracy and we should. Saying accurate things about the future is arguably much more valuable to decisionmakers than saying accurate things about the present or past. It is also inherently more difficult.

Even when we are trying to say accurate things about the future, though, some questions are easier to answer than others. Quick! Which is more difficult to answer: Is there likely to be a war somewhere in the Middle East in the next 100 years or is there likely to be a war between Israel and Egypt within the next 24 months? I am no Middle East expert but it seems to me that the first question is much easier than the second. I am guessing that most readers of this blog would say the same thing.

Why? What are the essential elements of a question that make it obviously more or less difficult to answer? How do we generalize these criteria across all questions?

I am not the only person to recognize the inherent difficulties in different kinds of questions. Michael Hayden, the former Director of the CIA and NSA, likes to tell this story:

"Some months ago, I met with a small group of investment bankers and one of them asked me, 'On a scale of 1 to 10, how good is our intelligence today?' I said the first thing to understand is that anything above 7 isn't on our scale. If we're at 8, 9, or 10, we're not in the realm of intelligence—no one is asking us the questions that can yield such confidence. We only get the hard sliders on the corner of the plate."

Note that Hayden highlighted the degree of difficulty of the questions (not the difficulty of obtaining the information or the complications associated with supporting political appointees or the lack of area experts or anything else) as the reason for more realistic expectations for the intelligence community's analysts.

So...if degree of question difficulty is the missing half of the "evaluating intelligence" equation, shouldn't someone be working on a diving-like degree of dfficulty table for intel analysts?

That is precisely what I, along with one of our top graduate students, Brian Manning, have set out to do this year. This research question piqued our interest primarily because of our involvement in the DAGGRE Research Project (more on that soon).

In that project, we are asking questions (lots of them) that all have to be resolvable. That is, they have to all have an answer eventually ("Will Moammar Ghaddafi be President of Libya after 31 DEC 2011?" is a resolvable question -- he either will or he won't be president after that date).

My concern was that this is not the way that most questions are actually asked by the decisionmakers that intel typically supports. For example, I would expect that the Ghaddafi question would come at me in the form of "So, what is going to happen with Ghaddafi?" A very different question and, intuitively, much more difficult to answer.

So far our research has turned up some interesting answers from the fields of linguistics, artificial intelligence and, from all places, marketing. We expect to find interesting answers in other fields (like philosophy) but have not yet.

Our goal is to sort through this research and figure out if any of the existing answers to this "question about questions" makes any sense for intel professionals. Alternatively, we might take elements from each answer and kludge them together into some steampunk-looking difficulty of question generator. We just don't know at this point.

What we are looking for is good ideas, in general, and, in particular, any real research into how to rank questions for difficulty.

The comments section is now open!

Smokin' Hot Online Cyberthreat Analysis Grad Class About To Start! (Shameless Self Promotion)

2011-10-13T13:51:00.000-04:00

I know it is shameless. I know it is self-promoting but I think our online cyberthreat analysis course is an extremely cool, very awesome class.

Why?

First, it is taught by Billy Rios. Billy is the current team lead for web and product security at Google. Before his gig with Google, he worked at Microsoft. The prof for this class is a soldier who comes in from the trenches to teach each class and goes back to the trenches when class is over. It simply does not get any more real than that.

Second, the class is not just theory. Billy brings a healthy dose of application to the class. To quote the brochure: "Students will be introduced to the key concepts, tools, and terminologies used by professionals in the field and apply what they learn in lab exercises that model real-world events."

Third, it means something to me when good students speak highly of a class. Numerous high quality students who I know and trust have taken this class -- and loved it.

Fourth, it is online and asynchronous. That doesn't mean you can blow off assignments until the last week and then catch up. What it does mean is that, if you are deployed or in a distant time zone or work odd hours, you can still take this class.

Fifth, it is an introductory level class and there are no pre-requisites. It is specifically designed for someone who knows little to nothing about cyber. It is perfect, for example, for an analyst who is interested in moving into the cyberthreat field or who just wants to have some grounding in the issues but does not have a technical background.

Sixth, it is a graduate level class that is accredited and transfers just like any other grad class.

Seventh...oh, you get the idea. It is a damn good class. If you are interested in taking it or want more information, contact Linda Bremmer: lbremmer@mercyhurst.edu

Class begins 28 NOV and runs for 10 weeks. The deadline for applications is 4 NOV.

If you are interested in cyberthreat analysis and are looking for a introductory course, I can think of no better place to start.

Intelligence Is All About A Partially Observable, Stochastic, Continuous, Adversarial Space...Really!

2011-10-12T11:00:00.001-04:00

At least that is what I learned in lesson 1 of Stanford's free, online, Introduction to Artificial Intelligence Course.

Along with 160,000 of my classmates from all over the world, I am trying to learn the basics of artificial intelligence (AI). Besides a long-time personal interest in AI (for my capstone project in law school, I designed an expert system that helped college students navigate landlord-tenant law (While it may seem trivial today, believe me, back in 1983, this was considered enormously sexy...)), at least half of AI is about how to better understand uncertain environments, something with which intelligence professionals are intimately familiar. In fact, in lesson 1, as one of our professors, Sebastian Thrun, points out, AI can be thought of as "uncertainty management" -- words that should also resonate with most intelligence analysts.

My hope is that some of the formal ways in which AI scientists go about doing their business might have direct application to what we often tend to think of as the very squishy world of intelligence. Since many AI applications are already concerned with what are traditionally intelligence problems, my assumption is that the language and systems used by AI professionals will help me understand and explain my own work better.

So far (I am well into Unit 2 of the course), I have not been disappointed. While the production values are more Khan Academy than Nova, I find the short video clips, frequent quizzes and my own interest in the material to be enough to keep my attention. The concepts that underlie AI are so embedded into almost any predictive system on the market or in the works that it is hard not to recommend the course to virtually all intelligence professionals.

While it may be too late to sign up for the course, you can view all of the videos on YouTube (I have embedded the introductory video below):

Pre-order Your "Intelligence Analyst's Deck of Cards" Now!

2011-10-10T11:22:00.000-04:00

In just a few days we expect to have the hotly anticipated Intelligence Analyst's Deck of Cards in the building.

While everyone who submitted a quote is still getting a free deck, many of you have asked to buy additional decks for yourselves or for your organization.

With those requests in mind, we have decided to open up a pre-order option for both individual and bulk orders.

Note: We will fill all pre-orders first and we may run out of decks. We intend to print more, of course, but you would have to wait. Bottomline: If you or your organization knows it is going to want some/many of these decks, you probably ought to go ahead and pre-order.

If you would like to pre-order less than 10 decks, please do so through the Mercyhurst bookstore.

If you would like pre-order 10 decks or more, please contact our editor, Domenic Vallone (dvallo66 at lakers dot mercyhurst dot edu) and he will be able to give you information about the discount available at the quantity you desire.

If none of this makes any sense to you, search this blog for "Intelligence Analyst's Deck of Cards" to get the rest of the story.

Getting Intel From Twitter, Getting Help From Online Communities, Getting A Job And Target-Centric Astronomy (Link List)

2011-10-05T10:43:00.000-04:00

The great things about having friends is that they send you stuff. I love getting links and articles from contacts all over the world. It is like Christmas every time I open my inbox...

OK, maybe not exactly like Christmas, but close!

Anyway, with no particular theme and in no particular order, here is a sampling of just some of the stuff that engaged my brain cells over the last few days:

Ten Simple Rules for Getting Help from Online Scientific Communities. This is actually a good list for anyone trying to get help from any online community.

Truthy. According to the site, "Truthy is a research project that helps you understand how memes spread online. Our first application was the study of astroturf campaigns in elections." They have just recently added a feature that allows you to map any Twitter hashtags that you find interesting (It was down earlier but might be up by the time you read this).

Why grad schools should be more like steel mills. Interesting essay on the current focus of graduate education and how it should change. Not a bad read if you are a grad student or a teacher of grad students. From Craig Zelizer on the Peace and Collaborative Development Network.

Do 'the Risky Thing' in Digital Humanities. More grad student advice. While this author focuses on digital humanities, I think the points apply to other disciplines as well. I am particularly drawn to the advice, "Make sure that someone's got your back, but do the risky thing." (Thanks to my colleague, Diane Chido!)

Preparing Grad Students for Careers Outside of Academia. This post aggregates comments from Chronicle of Higher Education readers about the opportunities and challenges facing graduate students.

Target-Centric Astronomy. Saved the best for last! My colleague, Steve Marrin, sent this out over the always useful IAFIE list. It compares how astronomers do their business to how Robert Clark says intelligence analysts should do theirs. Brilliant stuff!

9 Types Of Collaborators (CentralDesktop.com)

2011-09-30T11:20:00.000-04:00

Recently I posted some of the early results of our research into using wikis as collaborative tools for managing and producing intelligence. While my focus has been on the process, it makes just as much sense to focus on the people involved in the collaborative effort.

CentralDesktop, a provider of a wiki+ solution to small and medium sized businesses, has produced an interesting infographic (see small version below but you will probably have to go to the site to see the blown up version) that captures their own experience with the various different kinds of collaborators. I suspect the evidence for the typology is anecdotal and that some of the intent behind the infographic below was humorous. That said, I found the idea of thinking about the kinds of collaborative personalities involved in a project to be an interesting one.

From a design standpoint, it would seem important to address the needs of the different types of personalities in order to engage as many as possible in the effort. From a management standpoint, however, it would seem important to focus training and cash on software that offered just enough variety (but not too much). While these two needs conflict with each other to a certain extent, there is likely a sweet spot where they overlap.

http://www.centraldesktop.com/infographics/collaboration-personas-the-9-types-of-collaborators.jpg

The Intelligence Analyst's Deck Of Cards: An Update

2011-09-23T11:54:00.001-04:00

Several people have asked me about the status of Intelligence Analyst's Deck Of Cards that we collectively worked on this summer, so I wanted to provide a quick update (If you have no idea what I am talking about, go here).

The proofs have just come back from the printer (as you can see in the picture to the right). We hope to go to print next week and have the cards back by mid-October. Shortly after that we should begin to send out free decks to anyone who submitted a quotation (whether it made it into the deck or not).

I think I have contact info for most of the people who submitted quotes but in case we are missing some, Domenic Vallone, the current editor of the MCIIS Press, will be sending anyone we don't have an email to make sure we have a good mailing address.

Of course, the remaining decks will go on sale with the profits going to fund the three intelligence oriented student clubs we have at Mercyhurst.

Is This The End Of Textbooks? (MakeUseOf.com)

2011-09-22T10:55:00.000-04:00

The recent launch of Mercyhurst's The Analyst's Cookbook, Vol 2 as a Kindle only edition ignited an interesting side discussion on the always informative IAFIE listserve regarding the relative value of paper vs. electronic materials for training and education.

Broadly, the arguments there break down along the lines of cost and convenience for electronic versions vs. durability and prestige (with a smattering of tradition thrown in) for paper.

The discussion is still ongoing for IAFIE members so I thought I would widen the audience a bit to include SAM's diverse readership. The infographic below, recently published in MakeUseOf, is designed to add some facts to the mix.

Via: OnlineEducation.net

By the way, our own decision was driven almost entirely by cost. Volume 1 of the Cookbook has been enormously popular (it is in its third printing and still available here) but it is expensive to produce, store and distribute. The Kindle version of Volume 2 allows us to drastically cut the price. This makes it an easier buy for most people and likely ensures a wider distribution.

Analyst's Cookbook, Volume 2, Now Available!

2011-09-20T14:54:00.000-04:00

The Analyst's Cookbook, Volume 2, is out right now(!) and can be downloaded from Amazon.com to your Kindle or, if you don't have a Kindle, to one of the free Kindle readers for PC, iPhone, etc.

We went with a Kindle edition of the Cookbook this time around for all the reasons anyone goes to digital publishing -- it is less expensive for you to buy (only $4.99) and easier for us to manage than paper books.

For those of you familiar with the first Cookbook, thanks for your support ... and for waiting so long! Your loyalty has made The Analyst's Cookbook the best selling book in MCIIS' inventory (it is now in its third printing!).

For those of you not familiar with the first volume of The Analyst's Cookbook (still available in hardcopy here), it is a series of short articles that outline the basics of a variety of different analytic techniques. Each chapter was written by a different analyst and addresses one specific method or technique, provides a short description, a how-to, and a sense of the pros and cons of the method. The second volume follows the same pattern.

What really makes the chapters interesting, though, is the experience each individual analyst had when they tried to apply the method to a particular problem. In the past, these method/problem match-ups make for some fascinating reading (like when one analyst applied the business methodology of benchmarking to European terrorist groups). The current collection is no exception in this regard.

The real exception in this volume is that, in the past, the Cookbook was a venue to show off graduate student writing, this volume shows off graduate student editing as well. It was put together almost entirely by two recent editors for the MCIIS Press, Nicole Pillar and Domenic Vallone.

Finally, while we had many good suggestions for improving the format of the Cookbook over the years since Volume 1 was published, in the end, we decided to stick with the less formal, "cookbook", approach of Volume 1. The goal for us is to capture the experience of using a particular analytic method on a real problem, to give the reader a sense of how these methods work. The purpose is not to provide a definitive evaluation of one approach vs. another. It is a starting place for thinking about analytic methods, not the end point.

I hope you enjoy the new Cookbook!

To purchase The Analyst's Cookbook, Volume 2: Go here!
To download free Kindle Reader software: Go here!

"You Are Not So Smart" -- The Book I Am Most Looking Forward To This Fall

2011-09-12T14:54:00.000-04:00

For those of you who are unfamiliar with the brilliant "You Are Not So Smart" blog, check out this brilliant example.

Go ahead. I'll wait.

OK, so hopefully you came back and did not get tied down (the way I do) following one insightful exploration of our own biases after another.

This stuff is particularly (read: incredibly) useful for intelligence analysts to understand and the only thing that could make the blog any better would be if the author, David McRaney, put it all into a book -- which he just did and it will be out next month.

Below is the official movie trailer for the book (his words, not mine) that explores the causes and preventative strategies surrounding procrastination:

Lessons Learned: Managing Intelligence Projects And Producing Intelligence Products With Wikis (The Good, The Bad, The Reality)

2011-09-08T14:29:00.001-04:00

Since late 2006 we here at Mercyhurst have been using wikis to manage intelligence projects and produce intelligence products. To date, we have done well over 100 wiki-based projects for real world customers in the national security, law enforcement, business and NGO sectors.

Note I am not talking about using wikis to produce wikipedia/intellipedia-like descriptive articles. I am using wikis in groups as small as 4-5 and as large as 50 to produce an intelligence estimate similar in scope and format to a National Intelligence Estimate. You can see an early example of our work here and some later examples here and here.

Not only have we worked for many different clients, we have used a variety of different wiki platforms such as MediaWiki (the platform that supports both Wikipedia and Intellipedia), Wikispaces and Google Sites.

As a result, we have learned a few things about both wikis and collaboration.

One of my goals during my sabbatical (I am on sabbatical now) is to finalize a "How-to" book on managing intelligence projects with wikis I co-authored a few years (!) ago with one of the best teams of students I have ever had the pleasure of working with.

Given my track record of getting stuff done (sigh...), I decided to post some of the more interesting findings in advance of getting the book completed.

Finally, just to make my own position clear: I am overwhelmingly in favor of using wiki software to manage and produce intelligence products. However, as the title to this post indicates, there is going to be both good and bad here. Both our experience and our more formal studies indicate pretty clearly that the "good" is significant. That said, it would be wrong to claim that using wikis to produce intelligence is a perfect solution and that there is no corresponding "bad" that needs to be identified, managed and minimized.

Without further ado, then, here is a slightly abridged list of the good, the bad and the reality (I will let you figure out which is which...) of using wikis to manage and produce intelligence products:

1. Wikis don't make me faster, they make us faster...
One of the most frustrating things for many novice users is that a wiki can actually be slower for the individual, in many cases, than doing comparable work more traditionally. Even when comfortable with the wiki software and the new workflow it often requires, many wiki users believe -- and quite rightly -- that they could go faster if they were just allowed to do it "my way".

The significant increases in productivity do not come in most wiki work by making the individual faster. Instead, the key to a wiki's success lies in eliminating many of the transaction costs of doing collaborative work.

Wikis make an intelligence team faster by streamlining communication, storing data in one easily accessible location, and allowing a team to work together in a near real-time environment. The seconds required to open, edit, save and send a file, are all removed from this process, adding up over the course of the project and saving the team valuable time and ending confusion. Team members can operate on multiple tasks simultaneously and once a task is done -- once a paragraph is written or a file uploaded -- it can be easily referenced and copied by others; it never has to be done again.

2. You can't break a wiki...
... or, at least, most wiki software makes it really hard to do so. Unlike an installed computer program, where you can corrupt a file or create the need to enact a complicated series of actions to fix a mistake, a wiki has built-in safeguards to protect against loss of information.

For example, a blank page can serve as a "sandbox" where those new to the technology can practice uploading pictures or try their hand at formatting text.

Poor edits or erased information are also easily undone or recreated by the "revert" function common to most wiki platforms. Wikis typically save a history of changes made to it, so when a first time user attempts to correct a spelling error and finds that they erased an entire paragraph, all is not lost. Reverts allow users to restore the wiki to its last saved version.

Most importantly, however, wikis encourage "play". This means that users, once they overcome the technological learning curve, are empowered to try new things. This type of empowerment is what leads to new and innovative uses of the technology that can enhance the intelligence process.

3. Wikis reflect the reality of the intelligence process
As I have been discussing in my "Let's Kill The Intelligence Cycle" posts, the modern intelligence process requires analysts to perform many tasks in parallel rather than completing individual tasks in sequence. Wikis accommodate this more accurate depiction of the intelligence process.

One action performed in a wiki simultaneously accomplishes several other goals. The decision to create and the act of creating a page, for example, is also a decision and act of modeling, collecting, analysis and shaping the final product. Wikis complement the real intelligence process almost perfectly.

4. Wikis remove the "box"

The new realities of intelligence require that analysts think "outside the box" with regards to threats and opportunities. This means breaking out of mental traps such as "mindset" and "groupthink", and to challenge one's biases.

Wikis help in that they provide a number of ways for many different contributors to participate. From the full time team member to the casual reviewer, there are many simple ways for new information, differing opinions and alternative analysis to work their way into the wiki format.

Wiki discussion pages, for example, encourage analysts to question both method and process. The instant updates to information can inspire an analyst to reconsider a thought or follow a new lead without confrontations or lengthy meetings.

Many wikis feature wikimail or wikichat features so that contributors can have both private discussions and real-time interaction. Most wiki platforms also have notification services that allow users to watch key pages or even the entire wiki for substantive changes. Likewise, many modern wki platforms have a number of "membership" options allowing some people to contribute but not administer the wiki, some people to administer and contribute to the project and others to merely access the final product.

All these options (and others) serve to enlarge and deepen the intellectual space in which the intelligence unit can work, without giving up control of the core aspects of the process or product.

5. Wikis make managers more efficient
Wikis give managers an unprecedented window into the intelligence process. They can track progress, deal with problems, and assess results from the receipt of the requirement to the delivery of the final product. This allows managers to provide more timely feedback or even redirect the project as needed.

A manager can be involved as little or as much as required without having to call a physical meeting, pick up a phone or write an email. Discussion pages, chat functions, and access to direct editing of the wiki allow managers to help analysts refine and shape the product as it progresses and maintain contact with the analytic team without taking up time better spent elsewhere.

At the end of the project, the wiki serves as an incredibly complete audit trail for evaluating the results of the analysis and for implementing changes with regard to future projects.

6. Wikis answer the intelligence requirement better
An intelligence product on a wiki that is well-structured, well-designed, optimizes hyper-linking and takes maximum advantage of relevant multimedia achieves a level of transparency and depth that traditional intelligence products simply can not provide.

A decisionmaker can immediately click a link to discern where information came from, check the history of a page, or read over discussion pages to see how a train of thought evolved throughout the project.

A wiki provides more readily available information than the standard printed final product. Moreover, the wiki can even be formatted to produce a printed product to accommodate the preferences of the decisionmaker.

7. Wikis have a learning curve

With wikis, there is not only a technological learning curve but a social dynamic learning curve as well. The collaborative environment that wikis create can be an unsettling experience for some people.

Analysts used to a certain working environment or working alone at a desk with the occasional meeting must now learn to work with other analysts in an online environment in which every sentence that is posted is immediately available for all to see and comment upon. A shift in thinking and work habits is required to absorb, what is for some, a radical change in how they perform their duties.

As far as the technological curve, since most wikis work similarly to a word processing application, many may adapt to the technology with relative ease. Still, it is one more application for intelligence professionals to learn.

Likewise, the internet or intranet nature of wikis makes it virtually certain that they are not as sophisticated as standalone word processing or presentation software packages. The frustration that accompanies the sense that "I can't do what I want to with this damn thing" is real and often underlies the dissatisfaction intelligence professionals sometimes have when using wikis.

The solution for both issues is immersion and most successful wiki-based intelligence projects insist that users put everything on the wiki from the very start of the project. Wikis need a critical mass of information on them in order for their utility to become obvious to a team new to working with a wiki.

Casual or hybrid approaches to using the wiki may work but often do so in spite of the team's use of a wiki rather than because of it. The wiki learning curve is steepest at the beginning of the project and the sooner and more aggressively the team begins to climb it, the better. Managers that insist their teams climb this curve are likely to be unpopular at first but the benefits of the hard line approach will typically become evident to all well before the mid-point of the project.

8. Not all wikis are created equal
All wikis allow a user to easily create and edit a page. However, some software packages are easier on the user than others and this adds to the difficulties inherent in the learning curve.

More importantly, a truly great final product on a wiki doesn't just happen. Words are not magically fed into an online machine that automatically creates a relevant and substantial report.

A good and creative analytical team can craft a visually appealing, cohesive and insightful final intelligence product using a wiki -- but it takes work. Even good software can't fix poor analysis, sloppy editing and unappealing formatting.

9. More interaction ≠ peaceful collaboration
Enhanced interaction amongst analysts does not always mean that emotions and good manners will be kept in check. Written communication is easily misconstrued and, unfortunately, some people will find it easier to write scathing criticisms than speak it aloud.

Some wikis have developed their own sense of culture and social norms. Wikipedia, for example, maintains a strict code of civility which contributors and editors are expected to follow. Just because some wikis have such rules and social norms does not mean that your wiki will have such rules or social norms, though.

Nor might they emerge from the routine interactions of the participants in the wiki. Establishing rules of the online workspace and good management can overcome poor social dynamics that threaten the success of the wiki.

10. Wikis permit micro-managing
The transparency and usability of wikis allow managers to follow a project from start to finish. This can make managing one or more teams infinitely more efficient, but some managers take it as an opportunity to micro-manage.

Managers who comment on every post and every analytical statement, or continuously edit work are considered disruptive editors. This kind of behavior hinders analysts' progress and discourages them from using the wiki.

It can be difficult to know how much managerial involvement is too much, but an invasive manager or a "do-it-all" can prevent analytic wikis from evolving past a place to store information or the personal insights of the manager.

11. Wikis lack a substantive look and feel

Instead of dropping a thick report on a decisionmaker's desk (accompanied, of course by an "Executive Summary", a final product based on a wiki is merely a link to what looks like a single web page. The decisionmaker is deprived of the tactile feeling of depth of thought that a printed report can inspire.

In the book, we discuss more fully some of the details regarding the preparation of a wiki for delivery as a final product to a decisionmaker. No matter how it is structured, however, it is very difficult to convey the depth of a wiki-based product. While this effect may become less acute with the passing of the generations, for many readers it will continue to look like a single page.

12. There are times when you shouldn't use a wiki
Reconsider using a wiki when the technology will not be available to all members of the analytical team. Two or more people who do not have consistent access to the wiki on a small team during the course of the project can hinder work flow and positive group dynamics.
Reconsider using a wiki if anyone on the team is in a position such that they can refuse to adopt the technology. While wikis are easy to learn and employ, some people may not be sold on their benefits and people operating outside of the group workspace will make it much harder to create a successful wiki-based intelligence product.
In the case of simply needing to create a database or other highly specialized product, consider that the project may be better served with specialized software.

There's lots more, of course, and I hope to be able to get to it in the next few months.

(Many thanks, again Kathleen, Kevin, Stephanie, Joe Ellen, Ethan, Emily and Emily! I promise -- I am working on it!)

Mercyhurst Students, Faculty, Alums, Friends: IAI? Or IIS? Help Us Decide!

2011-09-01T12:13:00.001-04:00

It is shaping up to be a heckuva year for intelligence studies at Mercyhurst. This fall we are welcoming our largest freshman class ever (we now have over 400 students in the program!), next summer we will be celebrating our 20th anniversary and 12 months from September 1, we should be moving into a new building.

One other major development that could also occur impacts more than just our little slice of heaven, though -- Mercyhurst College could become Mercyhurst University (though the timeline for this is much more variable, obviously).

With all this happening, it occurred to several of the faculty that we ought to re-examine our name. It started with the genuine naming problem presented by having an "MU" to deal with but expanded into a fairly deep (for us, that is) discussion of who and what we are.

With that in mind, I decided to see what our students, faculty, alumni and friends might think of our two current top choices. Using the wonderful Swayable tool, I put it to you: Which do you prefer, the more traditional "The Institute for Intelligence Studies at Mercyhurst University"? Or the alternative, "The Institute for Applied Intelligence at Mercyhurst University"? Cast your vote below!

If you don't like either one, leave a comment below!

The Top 6 Skills For Entry-level Intelligence Analysts

2011-08-30T10:10:00.000-04:00

Top 6 Skills For Entry Level Intelligence Analysts

My colleague, Dave Grabelski, has spear-headed what turned out to be a multi-year research project to identify appropriate skills for entry level analysts across all three intel communities -- national security, law enforcement and business -- and match them against what we were teaching.

As an intelligence studies program that focuses on application as well as theory, a robust understanding of the needs of the communities we support made sense. In the end, it was a lot more work than Dave thought it would be, but, with the help of a number of dedicated students, we have one of the most comprehensive and useful strategic planning documents I have seen.

One of the really interesting tidbits to come out of this effort was the chart embedded at the top of the page. Dave asked some students to scour the job offerings at a variety of institutions across the three communities. He then tasked them to categorize the skills identified in the job postings. Finally, he asked them to rank the skills based on frequency.

His researchers looked at multiple entry-level job offerings in 22 different agencies, companies and organizations. In all, they identified nearly 30 key KSAs - Knowledge, Skills or Abilities -- for entry level analysts. Many of them were only represented in a few postings, however. The six on the list above were broadly represented:

Analytic Methodologies: These included those methods and processes specific to the intelligence community examined. Whether it was SWOT in the business community or ACH in the national security community or crime mapping in the law enforcement community, it is clear that knowledge of specific intelligence methodologies is important.

Written Communication: Obvious and essential. Includes both formal communications (such as finished intelligence reports) and informal communications (such as email).

Research Methods: This is the general name given to a variety of skills that revolved around finding, retrieving, collating, processing and conducting first-level analysis of information.

Teamwork: Again, obvious and essential. The focus was on both small teams of analysts working on a problem and on lone analysts providing close support to operational teams.

Oral Communication: Briefing skill is a must here but so is the ability to communicate effectively and professionally in less formal settings.

Databases: This represented the ability to work with structured databases. While these are often different in content, the underlying structure is often similar. Students clearly need to have a working familiarity with databases and how to get the most out of them.

One of the questions I always ask myself on studies like these goes something like, "83%? Don't the other 17% need people who can write, too?" I think it goes without saying that virtually every organization needs people who can communicate effectively; some just choose to mention it.

In fact, if you look at it through a slightly different lens, it is kind of disappointing that 83% of the organizations looking for entry level intel analysts felt compelled to say that they were looking for people with good written communications skills...

The lesson learned for students hoping to enter the field of intelligence analysis is that these are the skills your potential future employers are looking for. Ignore them at your peril.

Related Post: How To Get A Job In Intelligence

The Mindset List -- Recalibrating Your Mind For The Class Of 2015 (Beloit.edu)

2011-08-25T13:26:00.000-04:00

Every year the good professors at Beloit College publish a mindset list. The purpose is to help old fogeys like me (and maybe you -- if you have kids, check with them) understand why their hip references to Two Live Crew and "Bueller? Bueller?" fall flat with incoming freshmen...uh, sorry, fresh-people.

This year's list is no different and a few of them really jumped out at me:

Andre the Giant, River Phoenix, Frank Zappa, Arthur Ashe and the Commodore 64 have always been dead.
There have nearly always been at least two women on the Supreme Court, and women have always commanded U.S. Navy ships.
As they’ve grown up on websites and cell phones, adult experts have constantly fretted about their alleged deficits of empathy and concentration.
Amazon has never been just a river in South America.
Grown-ups have always been arguing about health care policy.
Russian courts have always had juries.
They’ve always wanted to be like Shaq or Kobe: Michael Who?

The full list is available on the Beloit website.

Purity First: The Anti-Human Augmentation Movement Of 2027 (DeusEx.com)

2011-08-24T14:17:00.000-04:00

While the video below is a promo for a new video game called Deus Ex, this post is not about the game.

The game is a "typical" (though the initial reviews indicate it is actually pretty good...) action RPG video game. As with many games these days, there is an effort to create a promotional "buzz" in advance of launch. One marketing tactic often used is to create faux reports of disasters or crises that haven't happened (in the great tradition of War of the Worlds, The Day After and Cloverfield). It is particularly effective for these near-future kinds of story. This is such an effort (for another, see the trailer to The Curfew).

What caught my eye, though, was the subject of this "revolutionary propaganda film" -- human augmentation. Take a look (Note: This video is fairly graphic and not for the young or squeamish)...

In my post last week I noted how "human augmentation" -- the process of using technology to make humans better, smarter, faster (as we used to say back when this stuff only cost 6 million or so...) -- had made it onto the early part of the Gartner Hype Cycle.

Most of us would think, I guess, that human augmentation is generally a good thing. Who wouldn't want an amputee to be able to hold his child's hand or someone who was without legs to be able to walk?

The idea that these kinds of prosthetics and implants are getting so "good" that they might offer people actual (unfair?) advantages rather than merely make up for a perceived deficiency is relatively new.

Two people who have dealt with the consequences of this are Oscar Pistorius and Aimee Mullins, both runners who have lost their legs. Pistorius is a South African who has had to fight to be able to compete for a slot on the South African Olympics team. Mullins ran college track with the use of specially designed prosthetic legs back in the 90's.

Mullins is particularly interesting as she has gone on to a career as a fashion model and motivational speaker. She actually makes the case in the speech below that her artificial limbs give her an advantage.

It is hard to imagine that the dystopian view of human augmentation depicted in Deus Ex will ever come to pass but both these videos are worth watching for the counterpoints they represent (and the kinds of decisions we might be faced with in the future).

New Gartner Hype Cycle Out; Some Interesting Changes From Last Year (Gartner.com)

2011-08-12T12:59:00.000-04:00

Gartner provides, for my money, the most comprehensive and systematic coverage of technology trends among the commercial research providers. One of their best free products is the annual Gartner Hype Cycle.

The Hype Cycle is a useful way of thinking about how typical technologies evolve and mature. The 2011 version is displayed below (with a more complete report and video here):

http://www.gartner.com/DisplayDocument?id=1758314

I covered the Hype Cycle last year on SAM and comparing 2010's cycle with this year's is an interesting exercise. In the first place, there are a number of technologies that are not on both cycles. Gartner covers 1900 technologies so it is clearly impossible to put them all on a single Hype Cycle graphic.

Secondly, most of the technologies have not moved very much in the last year. This makes some sense given that many of the technologies aren't expected to mature for "5 to 10 years" or "more than 10 years".

A couple of notable exceptions include augmented reality and the media tablet which have both crested the first big wave of expectations. If Gartner is right, we should start seeing an increasing number of reports about the limitations of media tablets and the problems with augmented reality over the next 12 months.

I also always pay attention to what is coming in at the beginning of the Hype Cycle and what is about to leave the Hype Cycle. There are some interesting new additions this year: 3D Bioprinting and quantum computing. Location aware applications, speech recognition and (surprisingly) predictive analytics are all set to leave the stage -- they have become mainstream in Gartner's eyes.

How Students Use Technology (Mashable.com and OnlineEducation.net)

2011-08-11T12:47:00.000-04:00

As we approach the new school year, this graphic will resonate with the educators out there (via Mashable and OnlineEducation.net ).

Via: OnlineEducation.net

The Last Decision We Have To Make... (Intel Analyst "Words To Live By")

2011-08-08T14:06:00.000-04:00

The quotes have been collected, the surveys submitted, the results gathered and analyzed.

The Intelligence Analyst's Deck of Cards is almost finished.

All we need now is a design for the back of the cards.

I hired a local design firm to come up with some ideas and through a wholly unscientific process have down-selected to the two you see in the Swayable below.

I am interested in your opinion! Please vote and pass the site on to others who should vote on this crucial decision of national -- nay, international! -- importance.

More Data Cake, Please! (EpicGraphic.com)

2011-07-27T11:45:00.000-04:00

I would change "data" to "information" and "information" to "intelligence" but other than that, I like this cake!

(Hat tip to Nimalan Paul for the link)

Image by EpicGraphic

DC-based Counterintelligence Courses Now Open For Registration! (Shameless Self Promotion)

2011-07-22T12:14:00.000-04:00

How would you like to take counterintel courses for graduate credit from some of the most experienced CI professionals in the world? Now you can...

Mercyhurst College has been teaching counterintelligence courses in the DC area for a number of years. Traditionally, we have done this in coordination with a government or private partner. This year, however, we have decided to open these classes up to qualified students throughout the DC area.

These courses (there are three of them) make up our Graduate Certificate in Counterintelligence. All of the classes are accredited, of course, and are taught one per quarter for three quarters. If you want the certificate, you would need to take all three and take them in series. If you are just interested in learning more about the CI field, then you could opt to take just the first course -- it is entirely your call.

The first course in this series will be offered in early September and run for approximately 10 weeks. The course is not offered online and will meet once a week in the evening at a location in the vicinity of Tyson's Corner. There are no pre-requisites other than a bachelors degree from an accredited university.

The courses are taught by Brian Kelley, Bob Stephan and Ray Batvinis. Kelley spent more than 20 years in the DO at CIA working both defensive and offensive CI ops; Stephan is the author of Stalin's Secret War and has over 20 years expereince with CIA, DIA and the USAF; Batvinis spent 25 years at the FBI working on CI issues at the highest levels.

Because of their experience and contacts, one of the unique aspects of these classes are the kinds of guest speakers they can draw. For example, in the last class they were able to bring together Plato Cacheris and John Martin (the defense attorney and the federal prosecutor in the Robert Hanssen case) for a one-of-a-kind roundtable.

This brief note cannot do either this course or the guys teaching it justice. If you want more information, please do not hesitate to contact the director of our online and distance learning programs, Linda Bremmer, at 814 824 2170 or lbremmer at mercyhurst dot edu.

Registration is now open!

Intelligence Analyst "Words To Live By" -- #10, Voting Closes 19 JUL 11!

2011-07-07T12:13:00.000-04:00

This is the tenth, and last, list of "Intelligence Analyst Words To Live By."

The tenth list is below and links to the previous nine lists are below that. I am off to the Global Intelligence Forum next week and will not be back until 18 JUL so I decided to keep the voting open until then. Please pass these lists on to any analysts you know!

Create your free online surveys with SurveyMonkey, the world's leading questionnaire tool.

Links to the previous nine lists:

List 1
List 2
List 3
List 4
List 5
List 6
List 7
List 8
List 9

The current frontrunners from the ninth list include (with the percentage representing the percent of respondents who said move it up the list):

"The greatest derangement of mind is to believe in something because one wishes it to be so." -- Louis Pasteur (77%)
"I suppose that if we in intelligence were one day given three wishes, they would be to know everything, to be believed when we spoke, and in such a way to exercise an influence to the good in the matter of policy." -- Sherman Kent (53%)
"There is no limit to what a man can do or where he can go if he doesn't mind who gets the credit." -- Ronald Reagan (64%)

All other entries are currently below 50%

Intelligence Analyst "Words To Live By" -- #9 (SurveyMonkey.com)

2011-07-07T09:00:00.001-04:00

This is the ninth of 10 surveys I intend to conduct to determine which are the "best" quotes, sayings and words of wisdom for intelligence analysts.

If you want more detail on why I am doing this, see the previous post.

Create your free online surveys with SurveyMonkey, the world's leading questionnaire tool.

Voting on the first eight lists is still open:

List 1
List 2
List 3
List 4
List 5
List 6
List 7
List 8

The current frontrunners from the seventh list -- I am one behind -- include (with the percentage representing the percent of respondents who said move it up the list):

"The intelligence analyst's function might be described as transcending the limits of incomplete information through the exercise of analytical judgment." -- Richards Heuer (58%)
"What experts think matters far less than how they think." -- Philip Tetlock (77%)
"If you can imagine it, the enemy will do it." -- Numerous (70%)
"The noblest service comes from nameless hands, and the best servant does his work unseen." -- Oliver Wendell Holmes (58%)
"Uncertainty, ambiguity, debate, and partial answers to tangled questions will remain an existential condition of the analytical process." -- Loch Johnson (56%)
"Bad news does not improve with age." -- Anonymous (63%)
"The best defense for the analyst who feels pressure to reach a certain judgment, or the case officer pressured to “bend the rules,” is the ethical recognition that no one—analyst, interrogator, or policymaker—is well served by such corruption." -- William Nolte (67%)

All other entries are currently below 50%.

Intelligence Analyst "Words To Live By" -- #8 (SurveyMonkey.com)

2011-07-06T09:00:00.003-04:00

This is the eighth of 10 surveys I intend to conduct to determine which are the "best" quotes, sayings and words of wisdom for intelligence analysts.

If you want more detail on why I am doing this, see the previous post.

Create your free online surveys with SurveyMonkey, the world's leading questionnaire tool.

Voting on the first seven lists is still open:

List 1
List 2
List 3
List 4
List 5
List 6
List 7

Intelligence Analyst "Words To Live By" -- #7 (SurveyMonkey.com)

2011-07-05T13:38:00.000-04:00

This is the seventh of 10 surveys I intend to conduct to determine which are the "best" quotes, sayings and words of wisdom for intelligence analysts.

If you want more detail on why I am doing this, see the previous post.

Create your free online surveys with SurveyMonkey, the world's leading questionnaire tool.

Voting on the first six lists is still open:
List 1
List 2
List 3
List 4
List 5
List 6

The current frontrunners from the sixth list include (with the percentage representing the percent of respondents who said move it up the list):

"It is better to be vaguely right than precisely wrong." -- Carveth Read (55%)
"When the facts change, I change my mind. What do you do, sir?" -- John Maynard Keynes (76%)
"Summarizing is not the same as analyzing." -- Kathleen Cassedy (76%)
"What gets us into trouble is not what we don't know, it's what we know for sure that just ain't so." -- Mark Twain (67%)
"When you have eliminated the impossible, whatever remains, however improbable, must be the truth. -- Sherlock Holmes (59%)
"Have the courage for what you already know." -- Frederich Nietzsche (52%)
"Things are not always what they seem; the first appearance deceives many. The intelligence of a few perceives what has been carefully hidden." -- Phaedrus (64%)

All other entries are currently below 50%.

Intelligence Analyst "Words To Live By" -- #6 (SurveyMonkey.com)

2011-07-01T13:52:00.000-04:00

This is the sixth of 10 surveys I intend to conduct to determine which are the "best" quotes, sayings and words of wisdom for intelligence analysts.

If you want more detail on why I am doing this, see the previous post.

Create your free online surveys with SurveyMonkey, the world's leading questionnaire tool.

Voting on the first five lists is still open:
List 1
List 2
List 3
List 4
List 5

The current frontrunners from the fifth list include (with the percentage representing the percent of respondents who said move it up the list):

"90% of everything you need is public." -- Numerous (52%)
"There is nothing more necessary than good intelligence to frustrate a designing enemy and nothing requires greater pains to obtain." -- George Washington (74%)
"In theory, there is no difference between theory and practice. In practice, there is." -- Yogi Berra (52%)
"We are what we repeatedly do. Excellence, therefore, is not an act but a habit." -- Aristotle (70%)
"It is a capital mistake to theorize before one has data. Insensibly, one begins to twist facts to suit theories, instead of theories to suit facts." -- Sherlock Holmes (77%)
"Correlation is not causation." -- Anonymous (72%)
"If you know the enemy and know yourself, you need not fear the result of a hundred battles." -- Sun Tzu (74%)

All other entries are currently below 50%.

Intelligence Analyst "Words To Live By" -- #5 (SurveyMonkey.com)

2011-06-30T16:11:00.000-04:00

This is the fifth of 10 surveys I intend to conduct to determine which are the "best" quotes, sayings and words of wisdom for intelligence analysts.

If you want more detail on why I am doing this, see the previous post.

Create your free online surveys with SurveyMonkey, the world's leading questionnaire tool.

Voting on the first four lists is still open:
List 1
List 2
List 3
List 4

The current frontrunners from the fourth list include (with the percentage representing the percent of respondents who said move it up the list):

"The first principle is that you must not fool yourself - and you are the easiest person to fool." -- Richard Feynman (61%)
"Trust but verify." -- Ronald Reagan (57%)
"Things change. There is no 'final' version of a report. There is only the 'current' version." -- Anonymous (77%)
"You are not doing your job if your research doesn't surprise you." -- Cynthia Storer (62%)
"If you think intelligence is expensive, try ignorance." -- David Jimenez (73%)

All other entries are currently below 50%.

Intelligence Analyst "Words To Live By" -- #4 (SurveyMonkey.com)

2011-06-29T13:57:00.000-04:00

This is the fourth of 10 surveys I intend to conduct to determine which are the "best" quotes, sayings and words of wisdom for intelligence analysts.

If you want more detail on why I am doing this, see the previous post.

Create your free online surveys with SurveyMonkey, the world's leading questionnaire tool.

Voting on the first three lists is still open:
List 1
List 2
List 3

The current frontrunners from the third list include (with the percentage representing the percent of respondents who said move it up the list):

"A problem well formulated is a problem half solved." -- Charles Kettering (69%)
"No matter what your conclusions are, always ask yourself if you have factored in the possibility of deception." -- Jim Breckenridge (73%)
"Knowledge a week too late is the same as ignorance." -- George Friedman (69%)
"Discovery consists of seeing what everybody has seen and thinking what no one has thought." -- Albert Szent-Gyorgyi (74%)

All other entries are currently below 50%.

Is The Games For Change Festival Important To Intelligence Professionals? (Gamesforchange.org)

2011-06-20T13:03:00.000-04:00

I hope to answer this question over the next day or two. I made the decision to go to the Games For Change Festival primarily to see what it was all about. This idea of "serious games" is gaining lots of traction in a variety of different fields -- almost none of which have anything to do with intelligence.

When people think about serious games, they usually think about improving education or health care or changing the world in other positive ways. There is nothing wrong with that, of course, and nothing wholly inconsistent with this view and the discipline of intelligence, but I recently witnessed a pretty heated debate over whether games that explain the basics of macroeconomics "belong". Not sure how anyone will react to the idea of serious games for intelligence professionals...

That said, the intelligence community has been exploring the educational value of games for quite some time (with mixed results, I hear). The recent Intelligence Advanced Research Projects Agency (IARPA) Sirius request for proposals is clear evidence that the US national security intelligence community intends to continue to explore how games can meet the IC's needs.

Of course, my own (personal and research) interests lie along these lines so going to this summit makes sense. I just wish it didn't conflict with another excellent conference, the Game Education Summit, which, this year, is on the west coast. GES is an excellent small conference with a unique opportunity to mingle with academics, designers, artists, developers -- all aspects of the game design and game education fields.

I will probably be posting at least some updates to my twitter account (@kwheaton) during the conference and will post a wrap-up article when I get back later this week.

Evil Geniuses Teach How (NOT) To Use PowerPoint (YouTube via Danger Room)

2011-06-17T11:32:00.000-04:00

Wired magazine's excellent blog, Danger Room takes another look at the use of PowerPoint in the US military. While the entire article is worth reading (Danger Room has covered the relationship between PPT and the military extensively), the video embedded below is priceless...

Intelligence Analyst "Words To Live By" -- #3 (SurveyMonkey.com)

2011-06-16T14:44:00.000-04:00

This is the third of 10 surveys I intend to conduct to determine which are the "best" quotes, sayings and words of wisdom for intelligence analysts.

If you want more detail on why I am doing this, see the previous post.

Create your free online surveys with SurveyMonkey, the world's leading questionnaire tool.

Voting on the first two lists is still open:
List 1
List 2

The current frontrunners from the second list include (with the percentage representing the percent of respondents who said move it up the list):

"Describing a past event is not intelligence analysis; it is history. True intelligence analysis is always predictive." Robert Clark (68%)
"If you don't make any mistakes in life, you don't accomplish anything." -- Aikido Master Ken Ota (65%)
"Always bring something to write with and on." -- Numerous (54%)

All other entries are currently below 50%.

Reviewers Needed For Visual Analytics Science And Technology (VAST) 2011 Challenge (PNNL)

2011-06-15T12:07:00.000-04:00

The National Visualization and Analytics Center at the Pacific Northwest National Laboratories in coordination with the IEEE sponsors a visual analytics challenge each year at the IEEE conference for students and researchers.

In order to judge the output from the participants, the challenge organizers asks for analysts to participate as reviewers of the submissions. Kris Cook, who is on the contest committee, has asked me to put the word out that the contest needs reviewers for this year’s challenge.

This is an unpaid, all volunteer effort to assist a non-profit sponsored contest. Kris’ note to me is reproduced below with additional links. If you are interested in participating or have any additional questions, please contact her directly.

For what it is worth, taking a look at the VAST entries is a very interesting and rewarding way to learn what is happening in the world of visual analytics.

Begin Text of Note:

Once again, we are in need of analysts to help us review submissions to the Visual Analytics Science and Technology (VAST) Challenge. The goal of the VAST Challenge is for teams to design intuitive new visualizations that support analysis of complex data sets.

We are soliciting reviewers for three mini-challenges and a Grand Challenge.
•   Mini-Challenge 1 challenges participants to develop tools to characterize the spread of an epidemic using the geospatial and micro-blogging data provided
•   Mini-Challenge 2 challenges participants to create cyber security situational awareness visualizations to identify events of interest in a collection of computer network logs
•   Mini-Challenge 3 challenges participants to create visual analytics tools to support investigation into criminal activity.
•   The Grand Challenge asks participants to look across all Mini-Challenge data to provide visual insights into an over-arching scenario.
More specific information about the tasks may be found at http://hcil.cs.umd.edu/localphp/hcil/vast11/index.php/.

As a reviewer you would be responsible for viewing 3-4 submissions. Each submission consists of:
•   Documents of several pages in length describing the software and how it was used in solving the mini challenge.
•   Screen shots of the software in use on the data
•   Video showing an example of the analytic process used by the team.
Submissions will be available for review by July 2nd and reviews must be returned no later than COB July 15th. All review materials will be accessible over the internet. Reviews will be conducted using the Precision Conference web-based reviewing system. Reviewers will be registered in the Precision Conference system and will submit their reviews using Precision Conference web pages.

If you are interested in reviewing please send e-mail to kris.cook at pnnl.gov no later than June 22nd. Please indicate which mini challenge you would be most interested in reviewing.

Thank you for your time and consideration!

Intelligence Analyst "Words To Live By" -- #2 (SurveyMonkey.com)

2011-06-14T13:34:00.000-04:00

This is the second of 10 surveys I intend to conduct to determine which are the "best" quotes, sayings and words of wisdom for intelligence analysts.

If you want more detail on why I am doing this, see the previous post.

Create your free online surveys with SurveyMonkey, the world's leading questionnaire tool.

If you haven't had a chance to vote on the first list you can find it here (voting is still open).

The current frontrunners from the first list include (with the percentage representing the percent of respondents who said move it up the list):

"Always be aware that what you know and believe can taint your perception of what really is." -- Bill Russell (84%)
"What is your source?" -- Numerous (67%)
"Chance favors the prepared mind." -- Louis Pasteur (61%)
"Many intelligence reports in war are contradictory; even more are false, and most are uncertain." -- Carl von Clausewitz (58%)
"If you always do what you aways did, you'll always get what you always got." -- Steve Gottlieb (58%)

All other entries are currently below 50%.

Intelligence Analyst "Words To Live By" (SurveyMonkey.com)

2011-06-10T15:00:00.000-04:00

Last week, I asked for intelligence professionals from the business, law enforcement and national security intelligence communities to send me quotes, sayings and other "words of wisdom" that they have found useful over the years. My hope was to be able to put together a short list that would be helpful for intelligence analysts everywhere, students and professionals alike.

Over 100 emails and nearly 1000 responses later, I think I can legitimately say that my request has been answered. Many thanks to all who submitted something!

Now comes the genuinely hard part of figuring out which of these quotes ought to be at the top of the list. Rather than just pick the ones I liked, I thought I would ask the same people -- you -- who made the suggestions.

The short survey below is just one of 10 (each with 10 quotes, so 100 total) that I intend to publish over the next several weeks. I am not asking which quotes are best or worst (I know that this is next to impossible given the large number of very good quotes); I am only asking which ones should be moved closer to the top of the list and which ones should be moved closer to the bottom of the list. Assuming I get enough responses, the truly best quotes should rise to the top of the aggregated results.

Create your free online surveys with SurveyMonkey, the world's leading questionnaire tool.

It is also legitimate to ask, of the nearly 1000 quotes I received, how did I downselect to the 100 I intend to survey? I looked for a couple of things. First, they needed to be general. The quote should arguably apply to intelligence professionals in any of the sub-disciplines of intelligence.

Second, the quote needed to be fairly short. There were several excellent lengthy entries that I hated to exclude but I felt that they fell more into the categories of "essay" or "story" than "saying".

Third, I focused primarily (but not exclusively) on intelligence analysis. I teach intelligence analysis and I want to create this list primarily to help my students and other students (young and old) of intelligence analysis.

My intent, once I have the results of the survey, is to take the top 52 quotes and have them printed on a deck of playing cards -- that's right, the "Intelligence Analyst's Deck of Cards".

We received a small unrestricted grant from a private donor a few months ago (yes, my academic friends, they do exist...) with the vague instructions to do something to improve analysis. This money should be enough to get a small print run completed through our Mercyhurst College Institute For Intelligence Studies Press.

Once they are printed, we will send out free copies to anyone who submitted an entry (whether it made it onto the cards or not). If there are any copies remaining after that, they will go on sale at our bookstore where whatever money we make will be donated to support the activities of our three student intelligence clubs here on campus.

Thanks again to all and please take a second to fill out the surveys as they get published!

New Intel Studies Building Going Up At Mercyhurst! (Yeah!)

2011-06-09T13:35:00.000-04:00

In case you missed it, Mercyhurst College broke ground last weekend on a new 9 million dollar building that will house the intel studies and the hotel and restaurant management studies programs.

There are lots of stories about it in the press and the whole event is linked to the alumni Facebook page and Mercyhurst's main site but I am aware that many of my readers and our friends and alumni may not be frequent visitors to those places (ahem...) which is why I decided to cover it here as well.

The bottomline is that the "new building" (that we have been talking about since almost the day I came to campus eight years ago) is a reality. We should occupy the new premises in the fall of 2012.

(For those of you who still don't believe it, see the picture at the left of Prof. Breckenridge personally engaged in the manual labor necessary to make this happen. We would have put Bob Heibel to work as well, by the way, but he was in Madrid at a conference...)

If you look at the picture of the facility at the top of the page, intel will fully occupy the top two floors. The hotel and restaurant management department will have its kitchens and classrooms in the ground floor (not visible in this picture). The first floor will have a number of shared facilities and classrooms.

(By the way, the GREAT thing about having the hotel and restaurant management department in the building is that it should eliminate the need to live off of food from the vending machines.)

The skybridge in the picture connects the new building to the library. Most of Center for Intelligence Research, Analysis and Training (CIRAT) will be in the fourth floor of the library. In addition, the bridge is designed to be a bit of a destination location itself. It will be extra wide and have chairs and other features that will make it appropriate for small receptions or as a place to study or eat lunch.

While the financing has been secured for the building, we wouldn't be a good college if we weren't looking for more donors and sponsors. So, if you want to support the fine work we do here everyday or if you just want your name on the laser cannons we have planned for the roof (not visible in this picture...), get out your checkbooks and go here!

Part 11 -- The New Intelligence Process: The First Picture (Let's Kill The Intelligence Cycle)

2011-06-08T14:00:00.000-04:00

Part 1 -- Let's Kill The Intelligence Cycle

Part 2 -- "We''ll Return To Our Regularly Scheduled Programming In Just A Minute..."

Part 3 -- The Disconnect Between Theory And Practice

Part 4 -- The "Traditional" Intelligence Cycle And Its History

Part 5 -- Critiques Of The Cycle: Which Intelligence Cycle?

Part 6 -- Critiques Of The Cycle: The Intelligence Cycle vs. Reality

Part 7 -- Critiques Of The Cycle: Cycles, Cycles And More Damn Cycles!

Part 8 -- Tweaking The Intelligence Cycle

Part 9 -- Departures From The Intelligence Cycle
Part 10 -- The New Intelligence Process

One of the most pervasive themes to emerge from this study of the intelligence cycle and, more importantly, its critiques, is that the intelligence process needs to be seen generally, as part of a larger decisionmaking process, and specifically, as something that has its own, unique, functions.

Trying to merge these two aspects of the process, one macro and holistic and the other micro and specific, seemed to me to be too much to ask of one diagram so I have split my understanding of the intelligence process into two pictures.

It is worth asking at this point: Why does the intelligence process need a picture at all? Why not just use simple, straightforward words? Beyond the demonstrated psychological value of images in helping people understand complex topics, whatever replaces the intelligence cycle will also have to replace the image of the intelligence cycle. I see no realistic alternative to fighting fire with fire.

It might also be worth asking, if you are going to use two, why not more? Why not a dozen? Why not a hundred pictures? I recognize the risk of proliferating images. In the anti-PowerPoint sentiments prevalent among many of today's leaders, more pictures may well be viewed as more clutter. In response, I can say that I have been using the two pictures I will propose over the next several posts in my classes, in my project work and in a number of meetings with senior decisionmakers for several years now. The response has been very positive.

The first picture (See below) defines the general relationship between intelligence and operations and with the decisionmaker that they both support. It is very clear (or should be, by now) that the intelligence process cannot be viewed in a vacuum. If it is correct to talk about an “intelligence process” on one side of the coin, it is equally important for intelligence professionals to realize that there is an "operational/planning process", just as large if not larger and equally important if not more so, on the other side.

These two sides overlap in significant ways, particularly with respect to the purpose, goals and people of the organization. The intelligence professional is, however, focused externally and attempts to answer questions such as “What is the enemy/criminal/competition up to?” and “What are the threats and opportunities in my environment?”

The operational side of the coin is more focused on questions such as "Where will we place our police/military forces?", “How will we train our forces/employees?” or "What plans do we need to create to be prepared for likely contingencies?" In many ways, the difference between operations and intelligence is the difference between "we" and "they" and the fundamental intelligence question is “What are they likely to do?” while the fundamental operational question is “What are we going to do?” Embedded in this distinction is also the difference between an estimate and a recommendation.

It is from this shared vision of the organization’s purpose and goals that intelligence requirements emerge. With few exceptions, there does not seem to be much concern among the various authors who have written about the intelligence process about where, exactly, requirements come from. While most acknowledge that they generally come from the decisionmakers who have questions or need estimates to help them make decisions, it also seems to be appropriate for intelligence professionals to raise issues or provide information that was not specifically requested when relevant to the goals and purpose of the organization. In short, there seems to be room for both “I need this” coming from a decisionmaker and for “I thought you would want to know this” coming from the intelligence professional as long as it is relevant to the organization’s goals and purposes.

Theoretically, at least, the shared vision of the goals and purpose of the organization should drive decisionmaker feedback as well. The theoretical possibility of feedback, however, is regularly compared with the common perception of reality, at least within the US national security community, that feedback is ad hoc at best.

There, the intelligence professionals preparing the intelligence are oftentimes so distant from the decisionmakers they are supporting that feedback is a rare occurrence and, if it comes at all, is typically only when there has been a flaw in the analysis or products. Some intelligence professionals have gone as far as to claim that “There are only intelligence failures and policy successes” suggesting that intelligence is often a convenient whipping boy for poor decisions while intelligence rarely gets credit for the eventual decisionmaker successes.

It is questionable whether this perception of reality applies throughout the intelligence discipline or even within the broader national security community. Particularly on a tactical level, where the intelligence professional often shares the same foxhole, as it were, with the decisionmaker, it becomes obvious relatively quickly how accurate and how useful the intelligence provided actually is to the decisionmakers.

While most intelligence professionals subscribe to the poor feedback theory, most intelligence professionals also have a story or two about how they were able to provide intelligence to decisionmakers and how that intelligence made a real difference, a difference willingly acknowledged by that decisionmaker. The key to this kind of feedback seems less related to the issue or to intelligence writ large and more related to how closely tied are the intelligence and decisionmaking functions. The more distance between the two, the less feedback, unsurprisingly, there is likely to be.

*****

While this first picture might be a new way of representing the relationship between intelligence, operations and decisionmaking, the content of this image is unlikely to be surprising to most intelligence professionals. Understanding the content of this image is truly basic stuff -- Intel 101.

Which is precisely why it is so important to capture it in an image and to give it equal visual weight with the more specific image of intelligence I will discuss in the next post. This first picture, in short, is not for the seasoned professional. It is for the student and, more importantly, for the decisionmaker.

Few decisionmakers, outside the military, have any formal experience integrating intelligence into their processes. Elected officials, CEOs and chiefs of police typically have an image of intelligence informed mostly by Hollywood. They may well see intelligence as connected to the rest of their organization by a dotted line (or, in extreme cases, by no line at all).

This image, then, sends a powerful message to these decisionmakers. It tells them that, whatever they may have thought about intelligence, it is really about understanding those things that are relevant to your success or failure but are outside of your control. All decisionmakers, no matter how obstinate, will eventually agree that knowing the enemy is as important as knowing yourself, that intelligence and operations are really just different sides of the decisionmaking coin, and this picture helps get them to that point. Once they have made this cognitive leap, it is much easier to convince them to integrate (and appreciate) the specific functions of intelligence.

Next: The Second Picture

Part 10 -- The New Intelligence Process (Let's Kill The Intelligence Cycle)

2011-06-06T15:16:00.000-04:00

Part 1 -- Let's Kill The Intelligence Cycle

Part 2 -- "We''ll Return To Our Regularly Scheduled Programming In Just A Minute..."

Part 3 -- The Disconnect Between Theory And Practice

Part 4 -- The "Traditional" Intelligence Cycle And Its History

Part 5 -- Critiques Of The Cycle: Which Intelligence Cycle?

Part 6 -- Critiques Of The Cycle: The Intelligence Cycle vs. Reality

Part 7 -- Critiques Of The Cycle: Cycles, Cycles And More Damn Cycles!

Part 8 -- Tweaking The Intelligence Cycle

Part 9 -- Departures From The Intelligence Cycle

All of the examples examined in the previous sections are really just hypotheses, or guesses, about how the intelligence process works (or should work). All are based on anecdotal descriptions of the intelligence process as currently conducted solely within the US national security community.

Few of the models attempted to broaden their applicability to either the business or law enforcement sectors. Very few of these models are based on any sort of systematic, empirically based research so, even if they more or less accurately describe how intelligence is done today, it remains unclear if these models are the best that intelligence professionals can do.

Other fields routinely modify and improve their processes in order to remain more competitive or productive. The traditional model of the intelligence process, the intelligence cycle, has, however, largely remained the same since the 1940's despite the withering criticisms leveled against it and, in a few cases, attempts to completely overthrow it.

While some might see the cycle's staying power as a sign of its strength, I prefer to see its lack of value to decisionmakers, its inability to shed little (if any) light on how intelligence is actually done and the various intelligence communities' failure to be able to even consistently define the cycle as hallmarks of what is little more than a very poor answer to the important -- and open -- theoretical question: "What is the intelligence process?"

It is to resolving this question that I will devote the remaining posts in this series.

Next: The First Picture

Part 9 -- Departures From The Intelligence Cycle (Let's Kill The Intelligence Cycle)

2011-06-02T14:26:00.001-04:00

Part 1 -- Let's Kill The Intelligence Cycle

Part 2 -- "We''ll Return To Our Regularly Scheduled Programming In Just A Minute..."

Part 3 -- The Disconnect Between Theory And Practice

Part 4 -- The "Traditional" Intelligence Cycle And Its History

Part 5 -- Critiques Of The Cycle: Which Intelligence Cycle?

Part 6 -- Critiques Of The Cycle: The Intelligence Cycle vs. Reality

Part 7 -- Critiques Of The Cycle: Cycles, Cycles And More Damn Cycles!

Part 8 -- Tweaking The Intelligence Cycle

Other authors have proposed, however, radically different versions of the intelligence process, overthrowing old notions in an attempt to more accurately describe how intelligence is done in the real world.

The first of these attempts, by longtime academic and former CIA officer, Arthur Hulnick, was the Intelligence Matrix. Hulnick believed that intelligence was better described in terms of a matrix (see image below). For Hulnick there were three main activities, parts of which, in many cases, occurred at the same time. These three “pillars” were collection, production, and support and services. Hulnick's model, while capturing more of the functions of intelligence, does not seem to provide much guidance on how to actually do intelligence.

Peter Pirolli and Stuart Card of the Palo Alto Research Center also attempted to re-define the intelligence process (see image below). This re-definition has gained some traction outside of the intelligence community. While much more complex than the cycle and typically perceived as a departure from it, Pirolli and Card's sensemaking loop is still both very sequential and very circular -- with all the limits that implies.

Probably the most recent and most successful move away from the intelligence cycle, however, has been Robert Clark’s target-centric approach to intelligence analysis (see image below). What makes Clark unique in many respects is that he is not merely attempting to describe the current intelligence process; he is attempting to examine how intelligence should be done.

Clark expressly rejects the intelligence cycle and advocates a more inclusive approach, one that includes all of the “stakeholders”, i.e. the individuals and organizations potentially affected by the intelligence produced. Clark claims that, to include these stakeholders, “the cycle must be redefined, not for the convenience of implementation in a traditional hierarchy but so that the process can take full advantage of evolving information technology and handle complex problems.”

Clark calls this a “target-centric approach” because “the goal is to construct a shared picture of the target, from which all participants can extract the elements they need to do their jobs and to which all can contribute from their resources or knowledge.” This approach does a very good job of describing a healthy relationship between the intelligence professional and the decisionmaker he or she supports.

This description of the way intelligence should work seems to fit well with at least some of the initiatives pursued by the US national security intelligence community. The example of Intellipedia, discussed in a earlier post, seems particularly close to Clark’s vision of the way intelligence should work.

What remains less clear is which came first. Is Intellipedia a natural extension of Clark’s thinking or has Clark merely identified the value of a more inclusive, interactive, Intellipedia-like world? Furthermore, beyond describing an ideal relationship between intelligence and decisionmakers, how does the intelligence product actually come about? On this point, as with Hulnick, the model provides little guidance.

Next: The New Intelligence Process

Part 8 -- Tweaking The Intelligence Cycle (Let's Kill The Intelligence Cycle)

2011-06-01T14:45:00.000-04:00

A number of scholars and practitioners have attempted, over the years, to rectify the problems with the intelligence cycle. While, from a theoretical standpoint, virtually all of these attempts have resulted in a more nuanced understanding of the intelligence process, none has caught on among intelligence professionals and none has been able to de-throne the intelligence cycle as the dominant image of how intelligence works.

These new schools of thought fall into two general patterns: Those that are tweaking the intelligence cycle in order to bring it closer to reality and those that seek to overhaul the entire image of how intelligence works (which I will discuss tomorrow).

Several authors have sought to modify the intelligence cycle in order to create a more realistic image of how intelligence “really” works. While some restructuring of the intelligence cycle is done within virtually every intelligence schoolhouse, the four authors most commonly discussed include Lisa Krizan, Gregory Treverton, Mark Lowenthal and Rob Johnston. These authors seek to build upon the existing model in order to make it more realistic.

From: Intelligence Essentials For Everyone

Krizan, in her 1999 monograph, Intelligence Essentials For Everyone provides a slightly restructured view of the Intelligence Cycle (see image to the right) and, while quoting Douglas Dearth, states “These labels, and the illustration ..., should not be interpreted to mean that intelligence is a uni-dimensional and unidirectional process. ‘In fact, the [process] is a multidimensional, multi-directional, and - most importantly - interactive and iteractive.’”


From: Reshaping National Intelligence

Treverton, in Reshaping National Intelligence In An Age Of Information, outlines a slightly more ambitious version of the cycle. In this adaptation, Treverton seeks to more completely include the decisionmaker in the process. You can see a version of Treverton's cycle to the right.

Lowenthal in his classic, Intelligence: From Secrets To Policy, acknowledges the flaws of the traditional intelligence cycle which he calls “overly simple”. His version, reproduced below, demonstrates “that at any stage in the process it is possible – and sometimes necessary – to go back to an earlier step. Initial collection may prove unsatisfactory and may lead policymakers to change the requirements; processing and exploitation or analysis may reveal gaps, resulting in new collection requirements; consumers may change their needs and ask for more intelligence. And, on occasion, intelligence officers may receive feedback.” Lowenthal's revised model, more than any other, seems to me to capture that the intelligence process takes place in a time constrained environment.

From Intelligence: From Secrets To Policy

Perhaps the most dramatic re-visioning of the intelligence cycle, however, comes from anthropologist Rob Johnston in his book, Analytic Culture In The US Intelligence Community. Johnston spent a year studying the analytic culture of the CIA in the time frame immediately following the events of September 11, 2001.

His unique viewpoint resulted in an equally unique rendition of the traditional intelligence cycle, this time from a systems perspective. This complicated vision (reproduced below) includes “stocks” or accumulations of information; “flows” or certain types of activity; “converters” that change inputs to outputs and “connectors”, which tie all of the other parts together.

While, according to Johnston, “the premise that underlies systems analysis as a basis for understanding phenomena is that the whole is greater than the sum of its parts”, the subsequent model does not seek to replace the intelligence cycle but only to describe it more accurately: “The elements of the Intelligence Cycle are identified in terms of their relationship with each other, the flow of the process and the phenomena that influence the elements and the flow.”

From: Analytic Culture In The US Intelligence Community

While each of these models recognizes and attempts to rectify one or more of the flaws inherent in the traditional intelligence cycle and each of the modified versions is a decided improvement on the original cycle, none of these models seeks to discard the fundamental vision of the intelligence process described by the cycle.

Next: Departures From The Intelligence Cycle

Part 7 -- Critiques Of The Cycle: Cycles, Cycles And More Damn Cycles (Let's Kill The Intelligence Cycle)

2011-05-31T12:46:00.000-04:00

Part 6 -- Critiques Of The Cycle: The Intelligence Cycle Vs. Reality

While intelligence professionals often tout the intelligence cycle as something unique, to experienced business, law enforcement and national security decisionmakers, the intelligence cycle looks like many other linear decisionmaking processes with which decisionmakers are already familiar.

Moreover, this familiarity has bred a certain amount of contempt as all of these disciplines are wrestling with re-defining their own processes in light of 21st century technology and systems thinking. In short, the intelligence cycle not only fails in its attempt to explain the intelligence process but also comes across as an archaic sales pitch to a decisionmaker who is typically all too familiar with the flaws of linear process models.

http://home.ubalt.edu/ntsbarsh/opre640/opre640.htm

Every military officer, policeman or business student who has attended even relatively low level training in their profession is familiar with a model of decisionmaking that typically includes defining the question, collecting information relevant to the question, analyzing alternatives or courses of action, making a recommendation and then communicating or executing the recommendation (see image to the right).

This model, of course, bears a striking resemblance to the “intelligence cycle”; a resemblance that may fool the uninformed but is unlikely to pass unnoticed by the decisionmakers that intelligence supports. These decisionmakers, who are never blank slates and rarely outright fools, are also unlikely to accept such a simplistic explanation of the process unless accepting such an explanation serves their own purposes or they simply don't care.

This, in turn, results in two negative consequences for intelligence. First, decisionmakers will, at best, see intelligence as “nothing special”. The process used appears, from their perspective, to be just a glorified decisionmaking process.

At worst, however, decisionmakers will see the “intelligence cycle” as mere advertising puffery, a fancy way of talking about something which could, in their eyes, be defined much more simply using a linear process model (albeit an out-of-date one) with which they are already familiar. Many private sector intelligence organizations have problems convincing the decisionmakers they support of the importance of intelligence. Over emphasis on the value of the intelligence cycle, particularly when faced by an educated decisionmaker, might well be part of the problem.

More insidiously, however, such a perception clouds the true role of intelligence in the decisionmaking process. Decisionmakers, trained in and used to working with the decisionmaking process, will look for intelligence professionals to provide the same kinds of outputs – recommendations – as their process does.

Intelligence, however, is focused externally, on issues relevant to the success or failure of the organization but fundamentally outside that organization's control. Intelligence does best when it focuses on estimating the capabilities and limitations of those external forces and poorly when it attempts to make recommendations to operators as the intelligence professional is generally less well informed than others in the organization about the capabilities and limitations of the parent entity.

In short, because the intelligence cycle creates the impression in the minds of many decisionmakers (particularly those unfamiliar with intelligence but well -educated in their own operational arts), that intelligence is “just like what I do”, only with a different name, the value of intelligence is more difficult to explain to decisionmakers than it needs to be.

Furthermore, once the decisionmakers think they understand the nature of intelligence, the way that nature has been communicated to them predisposes them to ask questions of intelligence that the intelligence professional is poorly positioned to answer.

Next: Tweaking The Intelligence Cycle

Part 6 -- Critiques Of The Cycle: The Intelligence Cycle Vs. Reality (Let's Kill The Intelligence Cycle)

2011-05-27T15:53:00.000-04:00

Part 1 -- Let's Kill The Intelligence Cycle
Part 2 -- "We''ll Return To Our Regularly Scheduled Programming In Just A Minute..."
Part 3 -- The Disconnect Between Theory And Practice
Part 4 -- The "Traditional" Intelligence Cycle And Its History
Part 5 -- Critiques Of The Cycle: Which Intelligence Cycle?

Were the lack of precision the only criticism of the intelligence cycle, it might be able to weather the storm. As suggested previously, there do appear to be general themes that are relevant, and the cycle’s continued existence suggests that its inconsistencies are outweighed, to some extent, by its simplicity.

Unfortunately, the second type of criticism typically leveled against the cycle is much more damning. In fact, it is fatal. Simply put, there is virtually no knowledgeable practitioner or theorist who claims that the cycle reflects, in any substantial way or in any sub-discipline, the reality of how intelligence is actually done.

Consider these quotes from some of the most authoritative voices in each of the three intelligence communities:

"When it came time to start writing about intelligence, a practice I began in my later years at the CIA, I realized that there were serious problems with the intelligence cycle. It is really not a very good description of the ways in which the intelligence process works." Arthur Hulnick, "What's Wrong With The Intelligence Cycle", Strategic Intelligence, Vol. 1 (Loch Johnson, ed), 2007.

"Although meant to be little more than a quick schematic presentation, the CIA diagram [of the intelligence cycle] misrepresents some aspects and misses many others." -- Mark Lowenthal, Intelligence: From Secrets to Policy (2nd Ed.,2003)

"We must begin by redefining the traditional linear intelligence cycle, which is more a manifestation of the bureaucratic structure of the intelligence community than a description of the intelligence exploitation process." -- Eliot Jardines, former head of the Open Source Center, in prepared testimony in front of Congress, 2005.

"The traditional intelligence cycle has been described as an "ideal-type" process that will always be subject to the real constraints of time." -- Jerry Ratcliffe, Strategic Thinking In Criminal Intelligence, 2004

"The classic intelligence cycle is neat, easily displayed, and quickly understood. The problem is that it doesn't really work that way. It's too static, too rigid, with too much distance between leaders and intelligence professionals." -- T.J. Waters, Hyperformance: Using Competitive Intelligence For Better Strategy and Execution, 2010

"Over the years, the intelligence cycle has become somewhat of a theological concept: No one questions its validity. Yet, when pressed, many intelligence officers admit that the intelligence process, 'really doesn't work that way.'" -- Robert Clark, Intelligence Analysis: A Target-centric Approach, 2010.

Once you start looking for them, it is easy to find detailed critiques of the intelligence cycle (and, please, don't hesitate to add your own). The only argument that still seems worth debating is whether or not the cost of maintaining this flawed model of the process is worth the benefit (a question about which readers of this blog were almost evenly split).

In addition to the quotes above, my colleague, Steve Marrin, provided me with an interesting update shortly after I started this series. According to him, the intelligence cycle was the subject of "vigorous discussion" at a 2005 RAND/ODNI Conference on intelligence theory and that this topic will also be the subject of a panel at the 2012 International Studies Association Conference. For a carefully crafted and articulate dissection of the intelligence cycle, I don't think I could recommend a better article than Steve's own chapter, "Intelligence Analysis and Decision-making: Methodological Challenges from the 2009 book, Intelligence Theory: Key Questions and Debate).

Once again, themes emerge from the general discontent with the inadequacies of the intelligence cycle. Many of these themes I will touch upon as I discuss alternatives to the intelligence cycle in later posts. One theme, however, leaps off each page and tends to dominate the discussion: The intelligence cycle is linear and intelligence, as practiced, is not. Tasks move from one part of the cycle to another like an assembly line, where parts are bolted on in a specific order to create a consistent product.

While this approach might be appropriate for early 20th century manufacturers, it doesn’t work with intelligence, where each product, ideally, contains information that is somehow unique. Consider, for example, this hypothetical dialogue between Mary, the CEO of Acme Widgets and Joe, her chief of competitive intelligence:

Mary: I need to know everything there is to know about the Zed Widgets Company.
Joe: Sure. What’s up?
Mary: We are thinking about introducing a new widget and I want to know what the competition is up to.
Joe: Anything in particular you are interested in?
Mary: Well, I can see their marketing efforts on the TV every day, so I am not really interested in that. I guess the most important thing is their cost structure. I want to know how much it costs them to make their widgets and where those costs are.
Joe: Right. Labor, overhead, materials. Got it. Is one part of the cost structure more important than another to you?
Mary. They pay about the same amount in labor and overhead that we do so I guess I am most interested in the materials; particularly Material X. That is our most expensive material.
Joe: I just read a report that indicated that the cost of material X is set to rise worldwide. Would you also like us to take a harder look at that and give you our estimate?
Mary: Absolutely.

While this example is simplistic, it makes the point. Intelligence, even in this one minor example within only one of the many parts of the traditional intelligence cycle is, or should be, at least, interactive, simultaneous, iterative. In the above example, this interaction between the intelligence professional and the CEO resulted in a more detailed and nuanced intelligence requirement going, as it did, from the very general, “Tell me everything…” requirement to the highly focused, “Tell me about Zed Company’s Material X costs and give me an estimate of where the price of Material X is likely to go.”

It is equally easy to imagine this kind of interaction within and between parts of the cycle as well. Collectors and analysts will inevitably go back and forth as the analysts attempt to add depth to their reporting and as the collector develops new collection capabilities. It is even likely that parts of the cycle that are not adjacent to one another will work very closely together, such as an analyst and the briefer responsible for the final dissemination of the product (in its oral form). Decisionmakers, too, may well remain involved throughout the process, seeking status reports and perhaps even modifying the requirement as new information or preliminary analysis becomes available.

The US military's Joint Staff Publication 2.0, Joint Intelligence, states the case more strongly:

"In many situations, the various intelligence operations occur nearly simultaneous with one another or may be bypassed altogether. For example, a request for imagery will require planning and direction activity but may not involve new collection, processing, or exploitation. In this example, the imagery request could go directly to a production facility where previously collected and exploited imagery is reviewed to determine if it will satisfy the request. Likewise, during processing and exploitation, relevant information may be disseminated directly to the user without first undergoing detailed all-source analysis and intelligence production. Significant unanalyzed combat information must be simultaneously available to both the commander (for time-critical decision-making) and to the intelligence analyst (for production of current intelligence assessments). Additionally, the activities within each type of intelligence operation are conducted continuously and in conjunction with activities in each of the other categories of intelligence operations. For example, intelligence planning is updated based on previous information requirements being satisfied during collection and upon new requirements being identified during analysis and production."

The situation is even more complex when you imagine an intelligence unit without teams of people working each of the discrete parts of the cycle. In situations involving small intelligence shops, where a single indivdual collects, processes, translates, analyzes, formats and produces the intelligence, the cycle breaks down completely.

The human mind simply does not work in this strictly linear fashion. Instead, it jumps from task to task. Imagine your own habits when researching a topic. You think a bit, search a bit, get some information, integrate that into the whole and then search some more. This approach inevitably leads to analytic dead ends, requiring more collection. At the same time, you are thinking about the form of the final report. If you are putting together an intelligence product that will use multimedia in its final form, for example, you are constantly on the lookout for relevant graphics or film footage you can use, regardless of its analytic value. To even suggest that you should collect all of your information, stop, and then go and do analysis without ever doing any further collection, is absurd.

One of the most recent and widely publicized innovations within the US national security community is the advent of “Intellipedia”, a Wikipedia-like tool for the intelligence community. Wikipedia, of course, is the online encyclopedia that is free to use and editable by anyone. It is one of the most popular sites on the web and, according to at least some research, is as accurate as other generally accepted encyclopedias. It has become, in its short lifespan, the tertiary source of first resort for both analysts and academics.

One of the things it is not is linear. There is no "Table of Contents" and researchers, authors and editors choose their own path through the resource. Some people generate full articles; others only dive in occasionally to fix a particular fact or even a grammatical or spelling error. There are even full-fledged “edit wars” where a particular version of an especially hot topic changes back and forth between competing points of view until either one side gets tired and gives up or, more likely, the sides reach a version acceptable to all. In the end, it is openness and interactivity that give Wikipedia its strength.

The US national security community acknowledged the value of such a tool, at least with respect to its descriptive products, when it launched Intellipedia. Begun in April, 2006, Intellipedia, according to information from June, 2010, now has 250,000 registered users and is accessed over 2 million times per week. This effort, which is clearly far beyond the experimental stage, plainly shows that collaboration and interactivity – the anti-intelligence cycle -- are core to any modern description of the intelligence process.

Next: Cycles, Cycles And More Damn Cycles!

Part 5 -- Critiques Of The Cycle: Which Intelligence Cycle? (Let's Kill The Intelligence Cycle)

2011-05-26T14:10:00.000-04:00

Despite its popularity, the intelligence cycle is widely criticized by intelligence professionals. These criticisms generally break down along three lines. First, there are those who say that what appears to be a theoretical monolith is actually open to a wide variety of interpretations, depending on perspective. Indeed, there is not one intelligence cycle but a series of intelligence cycles, each substantively different from the rest.

Second, many authors have claimed, quite convincingly, that the intelligence cycle, as generally described, does not, in many material ways, reflect the reality of how intelligence actually is done. The simplicity of the cycle, to these critics, is both seductive and deceiving.

Finally, there are those critics who claim that the so-called intelligence cycle is simply a marketing tool that rebrands overly simplistic "cycles" from business and leadership courses. I intend to discuss each in turn.

Which Intelligence Cycle?

Compare the diagram of the intelligence cycle which, until recently, graced the Intelligence.gov website (owned by the Director of National Intelligence -- DNI) with the diagram of the intelligence cycle from the Federal Bureau Of Investigation (FBI) below.

FBI Version Of The Intelligence Cycle

Recent DNI Version Of The Intelligence Cycle

Besides the obvious differences in graphic representation, what differences in content do you notice? If you look carefully, you will see that the FBI has decided to include a phase that is not in the DNI’s image, the “requirements phase”.

For a seasoned professional, this difference is trivial. Indeed, as I discussed in my overview of the intelligence cycle in Part 4 of this series, there is an explicit need for requirements and the FBI’s inclusion of them as a separate phase of the process might seem to be a matter of professional choice.

A student of intelligence, particularly a new student, might legitimately question this explanation, however. Perhaps there is a difference. Perhaps the FBI’s characterization represents a new way of thinking about intelligence as a process.

Perhaps, in fact, one description of the process is substantively better than the other. If this is not the case, then what is the explanation for the differences? There does not seem to be a good reason why the FBI’s take on the intelligence cycle should differ from that of the main intelligence site for the US Government, particularly since the FBI’s intelligence function, since the 2004 restructuring of the US intelligence community, is, in many ways, subordinate to that of the Director of National Intelligence. In short, does this difference represent legitimate theoretical differences or is it merely the result of a lack of coordination or, worse, sloppy thinking?

To make matters worse, the DNI's recently updated version of the intelligence cycle confuses the issue even more. You can see the the graphic currently in use at intelligence.gov below:

A quick examination of the current version of the DNI's cycle seems to differ from the previous version is several substantive ways. "Direction", "Exploitation" and "Production" all appear to be subsumed into broader categories of activities. Is there a reason for this? Did the DNI conduct studies to determine the best, most accurate, description of the cycle? Or was this a graphic design decision made becasue there was simply not enough room in the graphic for the additional words?

It gets worse.

On the same page that contains the graphic above, the DNI promotes not one but two additional variations of the cycle. In the first, more modest, variation (contained in the text that describes the picture), the DNI says, "The process begins with identifying the issues in which policy makers are interested and defining the answers they need to make educated decisions regarding those issues. We then lay out a plan for acquiring that infromation and go about collecting it." If this is true, then why doesn't the graphic also "begin" with requirements? Why does the graphic seem to begin with planning?

It gets even worse.

The third variation of the cycle (all on the same page) from the DNI comes at the very top of the page. Here one finds five links, "Management", "Data Gathering", "Interpretation", "Analysis and Reporting", and "Distribution". Clicking on the "Management" link indicates that management -- not requirements, not planning -- "is the initial stage of the intelligence cycle".

Sigh.

I wonder which version is taught in the Intel 101 courses?

I wonder how you grade a student who uses an "alternative" cycle as an answer on a test?

I wonder, if the intelligence cycle is perfect (as about 15% of the people I have polled indicate), which of these cycles is perfect-est?

Were these differences the only differences within the US national security intelligence community, they might be explained away more simply but they are not. In fact, there is very little consistency across and even within a number of important elements of the US national security community. These inconsistencies also exist across disciplines as well.

Examine the chart below. Only one function, collection, is universally attributed to intelligence across all 10 organizations examined.

Within the DNI, CIA and FBI there are minor but important differences – not one of the three is exactly like either of the other two.

Even more baffling are the differences within the US military, however. The Defense Technical Information Center (“the premier provider of Department Of Defense technical information”) has a streamlined four-part description of the cycle, one that largely (but not completely) agrees with the cycle as taught at Fort Huachuca, the Army’s home for its military intelligence professionals. This cycle, however, is substantially different from the process defined in the US Military’s highest-level publication on intelligence doctrine, Joint Publication 2.0.

The differences evident in the US military may well be due to different publation dates or my own lack of access to the most recent revisions of some of these documents. In this regard, though, the 2007 Joint Pub is worthy of further commentary. In it, the US military seems to abandon the intelligence cycle in favor of a more generic intelligence "process". Some have suggested that this proves the military has already killed the intelligence cycle (but it just didn't get the memo...).

While it is (from my viewpoint, at least) a step in the right direction, it only exacerbates the impression that either the left hand is not speaking to the right in the US national security intelligence community or that the DNI doesn't control or doesn't care what the Joint Staff puts out with respect to the intelligence process. All of those alternatives make the US IC look sloppy and disorganized.

I also think the Joint Staff is trying to have its cake and eat it, too. Compare the two images below. The first is from the most recent public version of Joint Pub 2.0. The second is from the 1990 version of the US Army's Field Manual 34-3, Intelligence Analysis. While the words in the two publications contain many significant differences, the pictures seem to say that the military has not backed too far away from its conception of the process as a cycle.

Join Pub 2.0 Intel Process 2007

FM 34-3 Intel Cycle 1990

These descriptions of the cycle differ, again, in significant ways from the descriptions provided by two oversight bodies commissioned to examine intelligence activities listed on the chart, the 1996 Graham Rudman Commission and the 2004 Weapons of Mass Destruction Commission. To round out the confusion, the description of the cycle offered by the International Association Of Law Enforcement Intelligence Analysts and the classic competitive intelligence model (as described by longtime private sector intelligence specialist, John McGonagle) also differ from each other and from the other 8 examples.

This analysis, while interesting, comes across as perhaps a bit more picky than it should. Other processes in other disciplines lend themselves to various descriptions. Indeed, despite the differences, there are clear themes that emerge even from this analysis. Few, for example, would question whether requirements, needs, direction, and planning fell into a single, generic category.

Themes, however, is all these are. A rigid approach to intelligence, implied visually in the pictures above and in many of the descriptions of these processes by each of these intelligence organizations, seems inappropriate under these conditions for teaching these concepts to new members of the intelligence profession or, indeed, explaining the process to the decisionmakers that intelligence supports. Instead, a more nuanced and less absolutist approach appears to be called for.

There is one specific area where this analysis does create cause for concern, however. Only three of the 10 organizations examined include a feedback or evaluation function within their versions of the cycle.

While some of the other organizations did include feedback as a subset of the dissemination process, subordinating this crucial evaluative process is not likely to endear the intelligence function to the decisionmakers that intelligence supports. It seems much better practice to include explicitly the role of feedback in the process, whether the decisionmaker chooses to take advantage of it or not.

Next: The Intelligence Cycle vs. Reality

Part 4 -- The "Traditional" Intelligence Cycle And Its History (Let's Kill The Intelligence Cycle)

2011-05-25T12:58:00.000-04:00

Part 1 -- Let's Kill The Intelligence Cycle

Part 2 -- "We''ll Return To Our Regularly Scheduled Programming In Just A Minute..."
Part 3 -- The Disconnect Between Theory And Practice

Finding descriptions of the intelligence cycle is not difficult. Virtually every organization, company or law enforcement agency that has even a modest intelligence capability has a picture, much like the one to the right (which, until recently, graced the US national security intelligence community’s main web site, Intelligence.gov).

So pervasive is this traditional image of the intelligence process that it comes across as generally accepted theory. Indeed, many private sector practitioners have built much of their marketing campaigns on touting the benefits of the cycle.

Likewise, it is commonplace to see the cycle featured prominently in government publications, statements of doctrine, training publications and even in critiques of intelligence. The entire architecture of intelligence, across all three major sub-disciplines of intelligence, is caught up in this more or less common vision of how intelligence professionals perform the functions of intelligence.

Despite its popularity, the history of the cycle is unclear. US army regulations published during WWI identify collection, collation and dissemination of military intelligence as essential duties of what was then called the Military Intelligence Division (Fun fact: According to Congressional testimony in 1919 the whole budget for military intelligence in 1913 was $10,000 -- or roughly $227,000 in 2011 dollars) but there was no suggestion that these three functions happen in a sequence, much less in a cycle.

By 1926, military intellgence officers were recommending four distinct functions for tactical combat intelligence: Requirements, collection, "utilization" (i.e. analysis), and dissemination, though, again, there was no explicit mention of an intelligence cycle.

The first direct mention of the intelligence cycle (see image to the right) I could find is from the 1948 book, Intelligence Is For Commanders (more on this book here). I hypothesize that the intelligence cycle probably came into use during WWII as a training aid but I have not been able to find any evidence to corroborate this bit of speculation on my part.

Since that time, the cycle, as a model of how intelligence works, has become pervasive. A simple Google image search on the term, "Intelligence Cycle" rapidly gives one a sense of the wide variety of agencies, organizations and businesses that use some variant of the cycle.

(Note: Experienced intelligence professionals might want to skip the next few paragraphs, which outline a more or less generic version of the intelligence cycle based on the image at the top of the post. I include it here for readers who are not familiar with the cycle or for students of intelligence who might need a refresher.)

While the actual details vary dramatically (something we will turn to in the next post) in a typical description of the process, the intelligence cycle usually begins with planning and direction or similar language. Direction usually comes from the decisionmakers that the intelligence activity supports although it can also come from the senior leaders of the intelligence activity or even from the analysts themselves.

It is important to note here that direction and planning can be formal but is often done informally (most variants of the cycle make no distinction). This is seen most often when there is no time for a more formal process. Less formal tasking is also often seen in smaller intelligence units, such as in business environments, or in units where the intelligence professionals and decisionmakers have a long-term relationship.

After the planning and direction phase, the collection phase, in a typical version of the intelligence cycle, begins. Here the intelligence professional begins to execute the plan to collect the types of information necessary to understand and answer the requirement.

Once the intelligence unit collects the information necessary, other intelligence professionals within the same unit might have to process and exploit it. Processing and exploitation takes on a number of forms including decrypting encrypted transmissions, turning a variety of conversations into a single cohesive report, translating source information from one language into another or identifying buildings and other features in an aerial image, among others. In short, processing and exploitation is the phase where very raw information becomes usable to the largest number of people authorized to view it within the intelligence organization.

With planning, direction, collection, exploitation and processing complete, the focus of the traditional intelligence cycle shifts to analysis and production, or the interpretation of the collected data and the creation of a product that best meets the decisionmaker’s needs.

Analysts need the widest possible variety of information sources in order to be able to corroborate other information and to answer the requirement with which they are dealing. The notional source of the information is much less important than its relevance to the requirement at hand and that source’s reliability.

Production carries its own concerns and they are often independent of the analysis. If the analyst is concerned with the content of the analysis, the intelligence production specialist is concerned with its form. Appropriate forms, in turn, depend on the needs and desires of the decisionmaker the intelligence unit supports. For example, while the traditional intelligence product within many areas of the US national security community is a written document with a smattering of pictures and graphs to explain key points, business decisionmakers tend to be much more comfortable with charts, graphs, and numerical data accompanied by a few explanatory bullets.

The final phase according to the diagram is dissemination. This is where the intelligence specialist delivers the final product to the decisionmaker. While this sounds fairly easy, it, too, has a number of pitfalls associated with it. Questions concerning exactly to whom an intelligence document should go to and exactly how it should get there are fundamental to this phase. For example, classification, or the level of secrecy or confidentiality associated with the intelligence product, is one such issue.

Likewise, many people in developed countries take high-speed voice and data communications capabilities for granted. Yet, in many cases, particularly in intelligence work, such capabilities are scarce or degraded due to geographic isolation. In these cases, the bandwidth available may determine where the intelligence product is sent or even if it is sent via electronic means at all.

Next: Critiques Of The Cycle

Part 3 -- The Disconnect Between Theory And Practice (Let's Kill The Intelligence Cycle)

2011-05-24T14:30:00.000-04:00

Previous Posts In This Series:

Part 1 -- Let's Kill The Intelligence Cycle
Part 2 -- "We''ll Return To Our Regularly Scheduled Programming In Just A Minute..."

Intelligence is not something that appears, autogenously; it is something that gets done, a process. This idea, that intelligence is a process, is one of the least controversial among intelligence professionals.

However, a general description of the process of intelligence -- that is, the best way to characterize and classify those consistent elements across intelligence sub-disciplines --is still very much an open theoretical question.

Intelligence professionals have long known that the traditional way of describing the intelligence process, the so-called "intelligence cycle", is flawed; yet none of the alternatives proposed has yet captured the nuance of the process as practiced or, for that matter, the mind of the intelligence community.

This disconnect between theory and practice, between the imperfections of the intelligence cycle and the way intelligence is actually done, has real-world consequences. While I will return to this theme many times throughout this series of posts, it is useful to get a sense of the costs associated with perpetuating a faulty model of the process.

For example, without a consensus on the way in which intelligence "happens" that works across the various sub-disciplines of law enforcement, business and national security intelligence, it is impossible to study the process for potential improvements.
In addition, reforms proposed under flawed models are likely to be flawed reforms, incapable of solving systemic problems because the system itself is so poorly understood.
Furthermore, training students with models of a process that falls apart when first touched by reality reduces the perceived value of training as well as the morale of those trained.
Budgets built around a flawed model are likely to mis-allocate funds and require work-around solutions that consume even more scarce resources.
Hiring people to fill positions created under an unsound model of the process is nearly certain to create a mismatch in terms of skills and competencies needeed vs. skills and competencies acquired.

The list goes on.

In this series of posts, I will begin by examining the intelligence cycle and some of the critiques of it. Next, I will examine the alternatives to the intelligence cycle. Finally, I will lay out my own understanding of the process. While every intelligence project is different, my own experience and the evidence I have collected over the last eight years indicates that there are patterns in this activity, whether in the national security, business or law enforcement fields, that are consistent across the entire intelligence profession.

The final goal of this exercise, then, is to outline this new description of the process as clearly as possible based on intelligence, as it is practiced, across all its sub-disciplines and regardless of the size of the intelligence activity involved. Furthermore, I want to balance the need for both simplicity and detail such that this explanation of the process is accessible to all students of intelligence -- at whatever age or level of experience.

Next: The Generic Cycle

"We'll Return To Our Regularly Scheduled Programming In Just a Minute..." (Let's Kill The Intelligence Cycle).

2011-05-23T11:00:00.000-04:00

My intent today was to jump right into my series on the intelligence cycle and why we should get rid of it (put a wooden stake through its heart were the exact words I think I used...).

However, over the weekend, I received a torrent of emails and the post received a number of comments and it occurred to me that, before I got started in earnest, it might be useful to do a little wholly unscientific sentiment analysis on this issue.

Using the Swayable tool (which many of you have already tested here and here), I intend to first test the underlying assumption behind this study and second to ask two related but independent questions about your perceptions of the intelligence cycle and its place in intelligence theory.

The Assumption Check

The first question is: "Is the traditional intelligence cycle a perfect representation of the current intelligence process? By "perfect" I mean perfect -- does the intelligence cycle accurately model the intelligence process as it is currently done? Trivial issues count here (we will deal with them later).

Something A Bit More Substantial

The second question addresses the degree to which the cycle is imperfect (assuming you thought it was imperfect in the first place): "Do the benefits derived from continuing to use the intelligence cycle as a depiction of the intelligence process outweigh the costs?" I would ask you to think carefully about both the costs and the benefits before answering.

Finally, I want to get at your beliefs: "Without reference to perfection (or imperfection), costs or benefits, do you believe that a better general description of the modern intelligence process is possible?" (Note: Extra credit for guessing why I chose pictures of Leibniz and Voltaire and double secret extra credit for knowing which is which...)

That's it. Please do not hesitate to pass this post and the series on to anyone who might be interested. In addition, please do not hesitate to join in the discussion by dropping me an email or posting a comment (comments are better as they can be seen by all but I understand if that is not possible).

Next: The Disconnect Between Theory And Practice

Let's Kill The Intelligence Cycle (Original Research)

2011-05-20T11:37:00.000-04:00

I mean it.

The "intelligence cycle", as a depiction of how the intelligence process works, is a WWII era relic that is way past its sell-by date. It has become toxic. It no longer informs as much as it infects. It is less a cycle than a cyclops -- ancient, ugly and destructive.

I want it dead and gone, crushed, eliminated.

I don't care, frankly, what we have to do. Remove it from every training manual, delete it from every slide, erase it from every website.

Shoot it with a silver bullet, drive a wooden stake through its heart, burn the remains without ceremony and scatter the ashes.

(Geez, Kris, why don't you tell us how you really feel...)

OK, OK, so, yes, I am being intentionally provocative but I have been doing quite a bit of research on the intelligence process over the last several years and have come to the conclusion -- as have others before me -- that our current best depiction of this process, the so-called "intelligence cycle" is fatally flawed. Moreover, I believe these flaws have become so severe, so grievous, that continued adherence to and promotion of the cycle is actually counterproductive.

My intent, beginning on Monday and over the next several weeks, is to lay out the evidence I have gathered about the cycle itself, about attempts to save it from its worst flaws, about attempts to replace it altogether and let you decide for yourselves.

In the end, I intend to recommend (with no hubris intended and well aware of the possibility of hamartia) my own generalized version of the intelligence process; one which I think is more appropriate for the intelligence tasks of the 21st Century and which works, in both theory and practice, across all three major sub-disciplines of intelligence -- national security, business and law enforcement.

Next: The Disconnect Between Theory And Practice

Why Good Data Isn't Enough (British Medical Journal And The University Of Michigan)

2011-05-19T14:01:00.000-04:00

You are briefing the boss today and you are pretty excited. You were tasked to take a hard look at two different ways of doing the same thing -- the "old way" and the "new way". The old way was OK but your research clearly shows that the new way is much better.

You stand up in front of the boss. You know you are speaking a little quickly (you may not even be pausing all that much) and your voice is probably a little higher than it usually is -- but none of that matters. Your data is rock solid.

In fact, you have even put your great data into a pie graph that clearly identifies the validity of your position. This is your ace in the hole because you know the boss loves pie graphs.

All of this explains why you are stunned when the boss decides to continue to do things the old way.

Two interesting studies, one quite old and one brand new, explain why what you said mattered far less than how you said it.

http://www.bmj.com/content/318/7197/1527.full

The first study, from 1999,"Influence of data display formats on physician investigators' decisions to stop clinical trials: prospective trial with repeated measure" from the British Medical Journal (hat tip to social network analysis expert Valdis Krebs and his prolific Twittering) asked a number of physicians to look at the exact same data using one of four different visualization techniques -- bar graph, pie graph, chart or "icons". You can see the four different charts in the picture to the right. Note: The test subjects only saw one of these, not all four together at once.

Now, I admit, these charts are a little dense at first. Basically you have 2 different groups, those who started the study with a good prognosis and those who started the study with a poor prognosis. You also have those who received the old treatment and those that received the new treatment.

The question was, based on these results, do you continue this study or not? The doctors involved in the study were all research physicians and used to seeing this kind of data and making these kinds of decisions.

Despite the fact that the data was exactly the same in all four images and that the data was overwhelmingly in support of the new treatment option, there was a satistically significant difference in the accuracy rate of the physician's decisions based exclusively on how the data was presented.

The least accurate? Pie and bar graphs. Charts did OK but the best option was the "icons".

This kind of iconic chart is probably new to many readers. It shows the impact of the treatments on every single patient in the study. While this kind of display yielded the most accurate results in the study, it was also the most disliked by the test subjects.

The overwhelming preference was for the chart, while a minority preferred the bar or pie graphs. Not only did none of the participants indicate that they preferred the icons, a significant number of them expressed derision at the format in their after action comments.

This study reminds me of a series of studies conducted by Ulrich Hoffrage and Gerd Gigerenzer at the Max Planck Institute in Berlin that demonstrate that expressing statistics using "natural frequencies" (e.g. 2 out of 20 instead of the more common 10%) leads to better understanding and better (i.e. more "Bayesian") reasoning (Jen Lee, Hema Deshmukh and I were able to replicate these results using a typical analytic problem so I believe that this effect is important in the context of intelligence as well).

The second piece of research is from the University of Michigan's Institute For Social Research and is still in pre-publication review. In what appears to be a very cleverly designed study, researchers looked at 200 telephone interviewers (100 male and 100 female).

They found that interviewers who spoke moderately fast, with lower pitched voices (if male) and with 4 to 5 natural pauses per minute were the most effective at getting people to listen to them.

Combining the results of these studies, it is easy to imagine that the most powerful presentation would be one using icons combined with a proficient speaker. The opposite (as demonstrated in the story that started this post) could reasonably be expected to perform less well -- even if the information were exactly the same.

As I have said before, like it or not, it is not enough to have good info, you have to be able to communicate it effectively as well. The flip side of this coin is equally important for intelligence professionals -- we may well be hard-wired to be biased towards high quality forms of communication, even if the quality of the content is second rate.

Genocide Warning For Kenya And Iran, An Interactive Timeline Of The Arab Spring Protests, New Government Source For Finding Intel Jobs (Link List)

2011-05-13T14:31:00.001-04:00

Lots of powerful stuff landing on my desk this week. Not enough time to write them all up so I just bundled them for the weekend:

The Sentinel Project For Genocide Prevention. This is yet another example of an NGO getting into the business of intelligence (Another good example is the Southern Poverty Law Center's Intelligence Project on hate crimes groups). This group of volunteers has taken the most recent research on indicators of genocide and turned it into an early warning system. They are currently issuing alerts on Kenya and Iran.

The Path Of Protest. The Guardian has put together a fascinating interactive infographic (see screenshot below) that captures all of the major incidents for all of the uprisings and protests in the Middle East and puts them in a single timeline. The pic does not do it justice. You need to see it for yourself. (Hat tip to Kevin F.)

http://www.guardian.co.uk/world/interactive/2011/mar/22/middle-east-protest-interactive-timeline

SearchUSA.gov. I suppose you could use it to search for anything in any of the 50 million or so pages that Federal, state and local governments have made available but I used it to search for "intelligence analyst jobs". I was both pleased and surprised that it not only brought up the usual federal listings but that it also brought up state and local intel analysis jobs that currently need filling. Very useful for the law enforcement intel analyst.

Grad Courses In Nonproliferation Analysis, Leadership In Intel Offered Online This Summer (Mercyhurst.edu)

2011-05-10T13:02:00.000-04:00

We are offering two new, online, graduate level courses this summer: Nonproliferation Analysis and Contemporary Leadership In Intelligence. Both courses are filling rapidly but currently have a few slots left. The courses will start 6 JUN and will run through 12 AUG.

If you already know you are interested, you can contact Linda Bremmer, who administers all our online offerings, at lbremmer@mercyhurst.edu. If you want a bit more info, read on!

Although these courses are part of our Master of Science in Applied Intelligence program, both courses are also open to other qualified individuals. US citizenship is not required and it is not necessary to be enrolled in another Mercyhurst program. All students who successfully complete the classes receive 3 (fully transferable) graduate credits, so participants must have a prior undergraduate degree to enroll.

The leadership course will examine organizational leadership in the context of intelligence organizations and units. Students will explore leadership styles, principles, and models, in addition to developing a personal sense of how to lead groups and how to lead change within an intelligence organization.

Taught by the director of our graduate program, Dawn Wozneak (a former FBI analyst), the course will also examine contemporary issues and ethical challenges facing intelligence leaders, particularly how leadership decisions impact organizations, staff, morale, and public perceptions of intelligence organizations.

The Nonproliferation Analysis course will be taught by Kimberly Gilligan, currently with the Global Nuclear Security Technology Division, International Safeguards Group, at Oak Ridge National Laboratory.

This course was specifically designed for analysts and other interested people who do not specialize in non-proliferation issues but want/need some background in this area.

Topics will include the nuclear fuel cycle, the Nuclear Nonproliferation Treaty (NPT), the International Atomic Energy Agency (IAEA), the Additional Protocol, safeguards (including an overview of verification techniques and the use of open-source information), export control, proliferation incentives (and disincentives), nonproliferation trends, and nuclear terrorism. Two case studies will explore the nuclear black market and proliferation.

If you are interested now, contact Linda Bremmer, who can answer any additional questions, at lbremmer@mercyhurst.edu.

How Accurate Is Your Pundit? (Hamilton.edu)

2011-05-06T08:36:00.001-04:00

Allen Thomson, the unusually keen-eyed observer of all things odd and analytic, brought a study by a group of students at Hamilton College to my attention recently.

Titled, Are Talking Heads Blowing Hot Air: An Analysis Of The Accuracy Of Forecasts In The Political Media, the study, complete with detailed annexes and statistical analysis, assessed the accuracy of 26 pundits in the media with regard to political forecasts made in 2008.

The students used something they called a Prognosticator Value Score (PVS) to rank each of the 26. The PVS factors in how many predictions were made, how many were right, how many were wrong and on how many the prognosticators hedged.

The best? Paul Krugman with a PVS of 8.2 (You can see a screenshot of his score sheet to the right. Note: Score sheets for each of the pundits are in the full text document).

The worst? Cal Thomas, with a PVS of -8.7 (You read that right. Negative eight point seven...).

The students were able to confirm much of what Tetlock has already told us: Many things do not matter -- age, race, gender, employment simply had no effect on forecasting accuracy.

The students did find that liberal, non-lawyer pundits tended to be better forecasters but the overall message of their study is that the pundits they examined, in aggregate, were no better than a coin flip.

This is more interesting than it sounds as one of Tetlock's few negative correlations was between a forecaster and his or her exposure to the press. The more exposure, Tetlock found, the more likely the forecaster was to be incorrect. Here, there may be evidence of some sort of "correction" that is made internally by public pundits, i.e. people who make a living, at least in part, making forecasts in the press.

I have a few methodological quibbles with the study. Number of predictions, for example, did not factor into the PVS. Kathleen Parker, for example, made only 6 testable predictions, got 4 right and had a PVS of 6.7. Nancy Pelosi, on the other hand, made 27 testable predictions, got 20 right, but had a PVS of only 6.2.

Despite these minor details, this study is a bold attempt to hold these commentators accountable for their forecasts and the students deserve praise for their obvious hard work and intriguing results.

Evaluating Analytic Methods: What Counts? What Should Count? (Global Intelligence Forum)

2011-05-03T13:54:00.000-04:00

About a week ago, I highlighted the upcoming Global Intelligence Forum and stated that one of the things I liked most about this conference was the opportunity, indeed, the inevitability of meeting interesting people working outside one's own area of expertise.

A really good example of this was Dr. Justine Schober, a pediatric urologist, who lectured the crowd last year on the the problems the medical profession had in analyzing intersexuality (I'll let you look it up...).

I will be honest with you: Justine's presentation was not what the crowd was expecting (...to say the least).

As I listened, however, to her description of the mistakes that doctors had made in this field, how bias and tradition had allowed these mistakes to continue for decades, and how much effort it had taken to begin to understand, analyze and rectify these errors, I realized just how much her profession and my profession have in common.

Evaluating Medical Practice -- Pyramid of Evidence

One of her most useful slides was a simple pyramid (See picture to the right) that highlighted the kinds of evidence doctors use to validate their methods and approaches to various diseases and disorders. Evidence at the bottom of the pyramid is obviously less valuable to doctors than evidence at the top, but all of this evidence counts in one way or another.

This led me, in turn, to think about how we in intelligence evaluate analytic methods. There appears to me to be two strong schools of thought. In the first are such notables as Sherman Kent and other long time members of the intelligence community who write about how difficult it is to establish "batting averages" for intelligence estimates in general, much less for particular methods.

The other school of thought (of which I am a member) emphasizes rigorous testing of analytic methods under realistic conditions to see which are more likely to improve forecasting accuracy and under which conditions. The recent National Research Council report, Intelligence Analysis For Tomorrow, seems to strongly support this point of view as well.

My colleague, Steve Marrin, has often pointed out in our discussions (and probably in print somewhere as well -- he is nothing if not prolific), that this is a false dichotomy, an approach that presents intelligence professionals with only extreme choices and so is not a very useful guide to action.

Justine's chart made me think the same thing. In short, it seems foolish to focus exclusively at either the top or the bottom of the evidence hierarchy. What makes more sense is to climb the damn pyramid!

What do I mean? Well, first, I think it is important to imagine what such a pyramid might look like for intelligence professionals. You can take a look at my own first cut at it below.

Evaluating Intelligence Methods -- Pyramid of Evidence

Ideally, we should be able to select an analytic method and then match the relevant evidence, such as it is, with that method. This, in turn, allows us to know how much faith we should put in the method in question and what kind of studies might be most useful in either confirming or denying the value of the method and under what circumstances.

Examined from this perspective, there are many, many useful and simple kinds of studies intelligence professionals at all levels and in all areas of the intelligence discipline can do to make a difference in the field and, more importantly, many of these kinds of studies are tailor-made for the growing number of intel studies students in the US and elsewhere.

Intel Deathmatch II: Who Will "Win" In Libya? (Swayable.com)

2011-04-29T11:10:00.000-04:00

OK...so that was fun!

If you missed Wednesday's post, we (me and about 250 readers) have been playing around with a new web/iPhone gadget called Swayable here at SAM.

The simple but brilliantly executed idea behind this tool is to get people in your social network to help you decide between two competing options. While much of the action on the Swayable site seems to center on which picture of Justin Bieber is "cutest", I decided to test it out on something which might be of a bit more interest to intelligence professionals.

Specifically, I wanted to know which of two classic intel texts should I read first (assuming I only had time to read one), Lowenthal or Clark. As of a few minutes ago, it was a dead heat (though, to be fair, Lowenthal has led for most of the last 48 hours by a slight but noticeable margin).

Today, I thought I would push the outside of Swayable's envelope once again. Here are a few more substantial questions for you. First, I want to know who is going to "win" in Libya (you can define winning any way you want):

Second, I want to know, if the pro-Gaddafi forces win, what will be the primary reason for the victory:

Finally, I want to know, if the Anti-Gaddafi forces win, with whom will they primarily align themselves after the conflict is over:

A couple of questions and comments have also come up about the service that I think are worth noting here.

First, I can't see who voted which way. All I can see is what you can see. Second, it is possible to preview the results before you cast your vote only on the web-based version of the tool. On the iPhone version, this feature is disabled.

I have had a few very kind emails from the founder of Swayable, Lindsey Harper, and she intends to run some tests to see how hiding the results before the vote would work on the web app as well.

Finally, she indicated that she plans to have an Android version out by summer!

Intel Deathmatch: Who Do You Read First, Clark Or Lowenthal? (Swayable)

2011-04-27T14:46:00.000-04:00

A new online service just came out called "Swayable".

Given that its primary focus is as an iPhone app, it seems to be designed to help people answer those burning questions while on the move such as which shoes should I buy or which dish should I order off a Chinese menu.

However, I can see some very interesting unconventional uses for something like this, particularly with respect to intelligence in business.

So ... let's try it out. Both of the books below are great and both are "must reads" for intel professionals. However, imagine you only have time to read one of them. Which one do you choose first?

What Do Decisionmakers Want From Intelligence? (Global Intelligence Forum)

2011-04-25T07:04:00.000-04:00

globalintelligenceforum.com

One of the most important questions any intelligence professional can ask is "What do the decisionmakers I support want from me?" Historically, it is also one of the most difficult questions to answer.

That is the reason why the faculty at Mercyhurst nominated Intelligence and the Decisionmaker as the theme for this year's Global Intelligence Forum in Dungarvan, Ireland (July 11-13, 2011).

Looking at the line-up of speakers -- GEN Mike Hayden, Charlie Allen, Hayden Peake from the US, Kathleen O'Toole from Ireland, Anthony Campbell from Canada, Niki Ekman from Sweden, Cees Wiebes from the Netherlands, etc. etc. -- and it easy to see why I think we are going to make some progress on this question.

Two other things make this conference one of a kind. The first is the attendees. This conference draws people from all three major sub-disciplines of intelligence: National security, law enforcement and business. Not only does it draw people from all areas of intelligence, it also draws people from all over the world.

So you are going to see guys like Tom Carr, the executive director of the Washington Baltimore High Intensity Drug Trafficking Area program rubbing shoulders with Liam Fahey, founder and executive director of the Intelligence Leadership Forum, the organization for senior corporate intelligence professionals.

There is simply no other conference that brings together this kind of mix of professionals for this kind of conference. Everyone I spoke with last year took as much away from the interaction between the participants as they did from the speakers.

Which is a nice segue to the second big advantage: Dungarvan. What a beautiful place! Located on the southern Irish coast in the midst of one of the prettiest parts of Ireland, Dungarvan is a perfect place for mixing, making new personal and professional relationships and carving out some time to think about some of the important questions facing the profession. Extraordinary hospitality, simple pleasures and (my personal favorite) an outstanding local beer make this event both physically relaxing and intellectually energizing.

We had to turn away people last year so this year we found a larger space and have expanded to 200 (and only 200) attendees.

Registration and more info about the conference is available online here. We are also offering two, one day workshops to round out the week. More information on the workshops here.

Questions regarding sponsorship opportunities for your company or organization should be directed to Bob Heibel, Executive Director of the Mercyhurst College Institute For Intelligence Studies at rheibel at mercyhurst dot edu.

Extremely Cool Predictive Analytics On House Prices (Trulia.com via Information Aesthetics)

2011-04-15T10:14:00.000-04:00

http://explore.trulia.com/datavis/priceredux/Q1-2011/

One of my favorite data visualization blogs, Information Aesthetics, is featuring a new interactive map from the real estate site, Trulia.com, that should get anyone in the market for a new home (or who is selling their old one) pretty excited.

The map (and I have included a screenshot of the DC area to the right) evaluates recent home sales on a zip code by zip code basis to let you know:

How many days a house was on the market before there was a price reduction?
What was the average percent reduction?
What are the odds of a second reduction?

The data set currently covers most of the US and all of the densely populated areas of the US.

Talk about actionable intel!

While this information is very useful, I hope that Trulia will work to improve the map. For example, I wonder why they decided to use zip codes instead of census tracts (which would have allowed for much finer grain analysis)?

It also makes me wonder how the Paradox of Warning will impact Trulia's product and the buyers and sellers who use it?

http://www.fas.org/irp/doddir/dod/jp2_0.pdf

Nuclear Reactors vs. Earthquakes (NETGIS.de)

2011-04-07T09:53:00.000-04:00

http://www.netgis.de/atom_gmaps/

One of my favorite blogs, Google Maps Mania, is featuring a mash-up, developed by a German firm, NETGIS, of data depicting all the world's operating, planned and defunct nuclear reactors combined with all the earthquakes in the world since 1990 that have registered 5 or above on the Richter scale.

http://www.netgis.de/atom_gmaps/

These data sets are then conveniently overlaid on a Google Map with pop-up bubbles that contain tools for easily defining a 100 km radius around the site and links to the Wikipedia article on the reactor. You can see some screenshots to the right to get a sense of what I am talking about. The site should be very useful to analysts trying to figure out where the next Fukushima might occur.

Understanding Radiation Dosage Levels (XKCD.com)

2011-04-05T14:10:00.000-04:00

While XKCD is best known for its very funny web comic, occasionally Randall Munroe, the author, uses his graphic skills for more serious purposes. Recently, he created one of the most useful charts I have seen. The chart compares radiation doses at various levels to each other and puts these dosages in terms of everyday experience.

While Munroe is quick to point out that "the author is not an expert, and that anyone potentially affected by Fukushima should always defer to the directives of regional health authorities", Munroe has consulted with experts and the chart is still worth sharing.

For more background on the chart and the sources used on the chart, click here; For a full size version of the chart, click here.

http://xkcd.com/radiation/

How To Negotiate A Ceasefire (HDCentre.org)

2011-03-31T14:06:00.000-04:00

The Centre For Humanitarian Dialogue, located in Geneva, Switzerland, has done a really good job of pulling together a concise monograph called "Negotiating Ceasefires".

Only 44 pages from start to finish (including endnotes and a comprehensive list of suggested additional readings), this guidebook is filled with practical advice, concise case studies and quotes from practitioners about the risks and rewards inherent in negotiating a ceasefire.

The author, Luc Chounet-Cambas (who has worked these issues in Afghanistan, Indonesia and the Sudan), does not see the ceasefire process through rose-colored glasses: "Negotiating ceasefires does not imply," he states up front, "that armed groups no longer see their military capability as a core source of leverage with the state." This, instead, is a practical volume, written for someone who needs actionable advice.

Of most interest to me personally were the insights on when to negotiate a ceasefire (when the political situation suggests the ceasefire might be sustainable) and the checklist of things to think about putting into a ceasefire agreement (such as de-escalation measures and the extent to which the ceasefire will extend to non-military activities). Finally, there is a must-read section on options for those thrust into the position of mediator.

I don't know much about the Centre for Humanitarian Dialogue but I intend to continue to watch their activities closely (They have two other volumes in this series, Engaging With Armed Groups and A Guide To Mediation: Enabling Peace Processes In Violent Conflicts for those interested).

Negotiating Ceasefires is a good, solid volume filled with practical advice from someone who has had to do it, not just talk about it. Recommended reading.

Why Am I So Excited About A Game Called "Resistance: Road To Liberation"? (Kickstarter.com)

2011-03-25T09:46:00.000-04:00

Three reasons, actually.

First, Resistance: Road To Liberation is a tabletop role playing game. Yes, yes, like Dungeons and Dragons and Traveler and a whole bunch of other games.

The difference here is that the game intends to be historically accurate and based, initially, on the various resistance movements of WWII.

I have had a chance to speak with the designer, however, and he indicated that his intent is to move beyond WWII and to develop rules and scenarios appropriate to the current spate of revolutionary and resistance movements going on in the Middle East and elsewhere.

Role playing has a long history as an intelligence analysis technique. Dick Heuer and Randy Pherson devote a chapter to it in their recent book, Structured Analytic Techniques For Intelligence Analysis, where they indicate that "role playing is particularly useful for understanding the outcomes of a conflict situation." Research by Kesten Green at the Victoria University of Wellington indicates that analysts who role play are over twice as accurate in their estimates as those who use unaided judgment.

Analytic role playing is typically very unstructured and informal, however. To get more realistic results, it would seem necessary to realistically constrain the "players". Taken to its extreme (See, for example, The Marine Corps' Infantry Immersion Simulator which is, in some sense, just role playing on steroids), it is highly effective but also extremely expensive and time consuming.

It seems to me then that a lightweight role playing game that captured many of the essential constraints without overly burdening the players in either time or money would be a useful tool for exploring resistance movements. It might also be a lot more engaging than listening to another briefing or reading another report.

The second reason I am excited about Resistance is that it is using Kickstarter.com as a way to fund the game. Kickstarter has only been around for a very short time but it has already become a major way to fund creative projects. While most of the projects are small (Resistance is looking for only $4000 in funding to get up and running, for example), some Kickstarter projects have raised over a million dollars.

On the other hand, Kickstarter also has a fairly brutal kill switch. If a project doesn't meet the minimum funding level, Kickstarter cancels the donations (which don't get distributed until the minimum is met) and the creator gets nothing at all.

Microfinancing isn't new (Kiva is my personal favorite example) but microfinancing has been traditionally associated only with developing countries. As an intelligence analyst, anytime I see a new financing model gaining acceptance outside its traditional sphere, I sit up and take notice.

Don't get me wrong, both Kiva and Kickstarter are excellent organizations and completely above board. However, any business model that can be used for good can also be used for ill (can anyone say "JihadStarter"?). Donating $15 or $20 to a worthwhile project on Kiva or Kickstarter is not only a good thing, it is also a cheap education in how these kind of internet based microfinancing sites work.

And the third reason? The designer is my son, Charlie Wheaton.

OK, OK. I hear you. In the interest of full disclosure, yes, I am damn proud of him. How many of us have wanted to make a living doing something we are passionate about? How many of us had a plan for turning that dream into reality at age 20? Yep, "damn proud" about sums it up...

More than that, though, is my interest in what he is doing with the role playing game genre and its possibilities for intelligence analysis. I have played Resistance and it is a good game that is very different than most role playing games and not just in terms of subject matter.

He has deliberately kept the rule set streamlined to give the maximum leeway to the players. He has created a system where groups advance in skills and abilities as well as individuals. He intends to publish the final version on Kindle as well as in hardback. The list goes on...

Charlie has been actively designing games for the last three years and, while he has not had any commercial success, he has learned a good bit about design by both studying it and actually getting his hands dirty. If he can get the money and successfully implement all of his ideas (and, in particular the ones revolving around more modern conflicts), I think we may all have a new tool for analysis and training.

Charlie distributed free copies of the beta version of Resistance at the Origins Game Fair (one of the world's largest) two years ago. He made a point of giving copies to soldiers that were there. The feedback he received was universally positive but everyone indicated that it needed more work on the details -- more scenarios, more options for weaponry and tactics and more possibilities in terms of resistance movements. With the money he gets from Kickstarter (assuming he makes his minimum), he hopes to do all that.

So, if you want to throw a few bucks his way, you can do that here. He has some neat "premiums" for various levels of contribution but Kickstarter also gives you the option to just donate some money to the cause.

Likewise, if you know anyone who might be interested don't hesitate to forward them this link or the link to the Kickstarter page.

How To Write An Awesome Resume, How to Organize Your Internship Search, How To Make Money While You Are Waiting And Other Advice For The Job Hunter (Link List)

2011-03-22T14:40:00.000-04:00

It has been some time since I talked about the unspeakable horrors of the entry level intelligence analyst job search and it will probably be some time before I re-visit that particular issue. Here are some useful sites to tide you over while you wait:

Top 10 Ways To Rock Your Resume. Lifehacker has put together a good list of their advice to resume writers. It is an excellent place to start.

Create A Master Resume For Easy, Targeted Applications. If you are a college student you should be reading Hack College. If you know one, forward them this link. It is one of the best blogs for useful advice about how to get the most out of the college experience and this post is a good example.

MeganMcCollum.com. Resumes, they are a changin'! Meg McCollum, one of our recent grads, is on the job market again and took a very different approach this time to the traditional resume for analysts. It is worth a look if only for some ideas (of course, if you are looking to hire an extraordinary young analyst, you should take a look in order to offer Meg a job...).

RezScore. This site allows you to upload your resume and get feedback on how good or bad it is. It actually gives your resume a letter grade! Even if you are hesitant to upload your resume to the site, you still want to take a look at the "A", "B" and "C" resumes available on RezScore's home page.

The Idealist Guide To Non-Profit Careers For First Time Job Seekers. While not written with intelligence analysts in mind, this online guide provides a wealth of useful guidance for all entry-level job seekers. In addition, I find that many people who are interested in intelligence jobs are also interested in serving their country or their communities in other ways. This guide is even more useful for those students.

Organize Your Internship Applications With A Spreadsheet. Internships not only help secure a job, they are also one of the best ways to help you figure out if intelligence analysis is right for you. These internships are typically pretty difficult to find and to get, so multiple applications are a necessity. This useful tip from Hack College (again) is worth exploring if you find yourself submitting multiple internship applications.

8 Websites To Get Tips On Job Interview Questions And Answers. After the applications and the resume comes the interview. These sites offer some helpful tips for managing that part of the job hiring ordeal with equanimity.

5 Salary Comparison Tools For Your Next Job Search. Another handy list from MakeUseOf.com

A Mind Map Of 100+ Tips For Using LinkedIn More Effectively. Most students are familiar (too familiar?) with Facebook. In my experience, though, the social networking site that might actually help you get a job is LinkedIn. Becoming actively involved in LinkedIn's various groups and taking advantage of its tools for building a professional network are things most students simply do not know how to do. The mind map, available for download from this site, does a good job of organizing and outlining many of the tasks you can accomplish with LinkedIn.

The Monster List Of Freelance Job Sites. One of the big problems with a career as an intelligence analyst is waiting for the clearance process to be completed. While the wait time for this has gone down over the last several years, it is still an unfortunate fact of life. Rather than working at Wendy's, recent grads often have skills they can use in the freelance world. This list of sites is essential for these kinds of short term job searches.

Make Money In Your Spare Time Doing Simple Online Tasks. Another way to help fill some empty months with a little bit of cash flow.

Any other sites I have missed? Leave them in the comments!