How To Grade Sources (Link List)

Dax Norman at the National Cryptologic School has done the most authoritative work on general reliability of online sources with intel specifically in mind.  You can find his JMIC/NDIC thesis on the topic and his matrix here.  We teach Dax’s approach in our classes as his research is the best I have seen on the subject and the research was done with the needs of intel analysts in mind.

In the 1997 version of the CIA’s Analytic Tradecraft Notes, the Agency seems to abandon the traditional A1, B2, etc system in favor of a new, less rigorously defined taxonomy.

FBIS used to (and probably still does within the OSC) publish source notes with each of its translated articles.  The BBC does some of this at the end of its Country Reports (See this example).

The website of any library associated with a research university will likely have a section that talks about source reliability.  Here is an example from Cal Berkeley on online sources and here is an example from George Mason on non-web-based sources.

One of my students, Kristina Jakomin, recently completed a study in my Advanced Analytic Techniques class into what law enforcement analysts tend to refer to as “statement analysis”.  She synthesized the “rules’ (such as they are) into a handy cheat sheet and ran a classroom exercise that suggested that these rules were useful in ferreting out some deceptive statements/sources. She then applied the rules to some statements made recently by Hamid Karzai. You can go to her website to see the results…

Finally, I have a student, Nimalan Paul, who is doing a comparative study of HUMINT source evaluation techniques for his master’s thesis.  Nimalan comes to us from the rough and tumble world of Indian primary source (ie HUMINT) intelligence in the business world and will be looking at best practices across all three intelligence communities (national security, law enforcement and business) to see what the common denominators of HUMINT source reliability are. I expect results on that research by this time next year if not sooner.

Using Search Engine Optimization (SEO) Tools To Do Intel Analysis (Original Research)

In my Advanced Analytic Techniques class this term, we have been taking a quick look at a number of different intel analysis methods (with the results posted on our ADVAT blog) but we have also been examining some methods in a bit more detail.

One of the more interesting experiments was Jeff Welgan's attempt to do competitive intelligence analysis using free, online search engine optimization tools. Search Engine Optimization (SEO), for those of you new to the term, is basically about trying to make your website, blog, etc. easier to find.

Jeff noticed that there are many, many tools to help people do this on the cheap. His thought was that you could use these tools not to aid your own efforts but rather to gather data about a competitor company, organization or even a terrorist group through their web presence. This, in turn, might allow an analyst to gain insight into their strategies and, possibly, their next moves.

For his purposes, he focused on two competitors, Starbucks and Caribou Coffee for his case study. His site, however contains a good bit more data, however. He has included basic background material on SEO, a list of operational definitions, a fairly comprehensive list of online tools, and a concise section on how-to use these tools as an intelligence method.

I think Jeff would be the first to admit that relying on SEO analysis exclusively is kind of like relying on HUMINT exclusively when you have SIGINT and IMINT as well. That said, this approach certainly has the potential to add a rich, structured source of data to bounce off the anecdotal and unstructured stuff that makes up most of what is available to the intel analyst.