How To Grade Sources (Link List)

Dax Norman at the National Cryptologic School has done the most authoritative work on general reliability of online sources with intel specifically in mind.  You can find his JMIC/NDIC thesis on the topic and his matrix here.  We teach Dax’s approach in our classes as his research is the best I have seen on the subject and the research was done with the needs of intel analysts in mind.

In the 1997 version of the CIA’s Analytic Tradecraft Notes, the Agency seems to abandon the traditional A1, B2, etc system in favor of a new, less rigorously defined taxonomy.

FBIS used to (and probably still does within the OSC) publish source notes with each of its translated articles.  The BBC does some of this at the end of its Country Reports (See this example).

The website of any library associated with a research university will likely have a section that talks about source reliability.  Here is an example from Cal Berkeley on online sources and here is an example from George Mason on non-web-based sources.

One of my students, Kristina Jakomin, recently completed a study in my Advanced Analytic Techniques class into what law enforcement analysts tend to refer to as “statement analysis”.  She synthesized the “rules’ (such as they are) into a handy cheat sheet and ran a classroom exercise that suggested that these rules were useful in ferreting out some deceptive statements/sources. She then applied the rules to some statements made recently by Hamid Karzai. You can go to her website to see the results…

Finally, I have a student, Nimalan Paul, who is doing a comparative study of HUMINT source evaluation techniques for his master’s thesis.  Nimalan comes to us from the rough and tumble world of Indian primary source (ie HUMINT) intelligence in the business world and will be looking at best practices across all three intelligence communities (national security, law enforcement and business) to see what the common denominators of HUMINT source reliability are. I expect results on that research by this time next year if not sooner.