Conceptual Modeling And Its Role In Intelligence Analysis

I had an opportunity to have an interesting email discussion with one of our grad students regarding the use and value of "conceptual modeling" in intelligence analysis and thought I would capture my part in that exchange in order to encourage some discussion on what I consider an important topic.

I understand that "conceptual modeling" has both a more formal definition as well as a more generic one (in much the same way that "mind-mapping" can be either in reference to Tony Buzan's work or used to describe any information visualization with nodes and lines). Here I am using it in the more generic sense but at a particular place in the analytic process.

I see it as a step that falls somewhere between a full understanding of the intelligence requirement and the beginning of collection. It is, in short, the process of identifying the kinds of information you think you will need to conduct the analysis. For example, if you were analyzing the level of stability in a particular country, you would want to know about its politics, demographics, communications, military, etc. In order to do the conceptual modeling, you would not be looking for specific sources or pieces of information (that comes later), merely the categories of information you might need. In this respect, I think what I am talking about here would resonate with the taxonomists and ontologists in the audience (Note: Some of the insights above came specifically from a conversation I remember having several years ago with Dax Norman at the National Cryptologic School and, to the extent that my ideas have any value at all, I would be remiss not to mention his contribution to them).

This model, then, informs the collection planning process and, at the end, should bear some relation to analytic confidence (if you have very little info from an area that you considered important, you might still be able to make an estimate but I would argue that your confidence in that estimate should be lower).

I would also argue that we all conceptually model before we start to analyze but that there are very strong reasons to make that model explicit. First, just as making an outline helps a writer keep track in a paper, creating a conceptual model (and they seem to work best as mind/concept/knowledge maps -- probably because so many people are visual learners) helps an analyst keep track of the pieces of the puzzle and the relationships between them. We can only keep 5-9 things in our head at a time and we have a bias for the most recent and vivid ones. Making this model explicit not only serves as a storage system but helps us go beyond our cognitive limitations as well.

Second, in group work at least, having each analyst make their model explicit up front allows the individuals in the group to rapidly identify areas of broad agreement and disagreement (We use a combination of individual and group brainstorming to help make these models explicit in our project work). While I am not suggesting that the conceptual modeling process should be used to force consensus, making these models explicit and then comparing them can help narrow the range of the argument by identifying the areas of legitimate disagreement.

Third, making the model explicit provides a useful tool in the oversight process. It helps analysts, their supervisors and other managers more effectively audit the process to see what an analyst did right or wrong.

I also usually like to say that conceptual models should be "written in pencil not pen." As new info comes in, the analyst needs to update the model. As info that is inconsistent with the current view of the target becomes available the analyst(s) should re-consider the model.

One further point worth making: I don't see conceptual modeling as a "method" or "analytic technique" but a tool like brainstorming or devil's advocacy. I think it is most useful when integrated into the broader analytic process that would include techniques (such as Analysis Of Competing Hypotheses or Social Network Analysis) that can more fully support estimative conclusions. I call these types of processes (that support the overall analytic effort but do not, by themselves, support estimative conclusions) "analytic multipliers" (an homage to the military concept of force multipliers, I suppose...).

Related Posts:
Analytic Confidence Defined...Finally!