Part 3 -- The Origins And Scope Of The Data (A Wiki Is Like A Room...)

Part 1 -- Introduction
Part 2 -- What Is A Wiki And Why Is It So Different?

For researchers, one of the enormous advantages of using a wiki is that even the simplest wiki software package (and there are many) captures massive amounts of data about the process itself. The problem, of course, comes in trying to figure out what this data means. Because of the tremendous amount of data accumulated from the 15 wiki-based analytic projects conducted over the last year and because I have only just begun to exploit this data, this series of posts will only skim the surface, drawing, in the process, only broad conclusions about analyst and decisionmaker reactions to wiki-based work.

(Note: The selection of which wiki software to use is not a trivial one. The best known package, produced by Mediawiki and used by Wikipedia, is server based and can be intimidating for first time users. On the recommendation of Fred Hassani, a human factors specialist working with the National Intelligence Council, I decided to use a somewhat more user-friendly, web-based wiki product, produced by Wikispaces.com, for the first wiki and have been using it ever since. I have continued to use Wikispaces because it is easy for the first time wiki user, very reasonably priced (for private wikis; it is free for public wikis), has generous storage limits and a helpful and courteous staff. To be honest, some students have reported problems with the software and students who rapidly adapt to the wiki environment (the “power users”) also occasionally complain about some of the limited features of Wikispaces (sourcing was a particular problem although the work-arounds devised by the students may actually be better than traditional sourcing methods). From my perspective, as a supervisor or instructor in all 15 of the projects, many of the issues tend to repeat themselves and can be safely chalked up to the inevitable problems associated with a first time user on a new piece of software. The positives of using Wikispaces with analysts unfamiliar with wikis, in my opinion, continue to outweigh the negatives.)

The vast majority of the wiki-based analytic projects I will discuss in this article were conducted in the context of my Strategic Intelligence Class at Mercyhurst College (For recent news articles about this class, see here and here). This class is a capstone class for seniors and second year graduate students and is designed to expand the students' knowledge of the fundamental concepts of strategy as well as explore the role of intelligence in the formulation of strategy. To do this, the students, in addition to the wiki-based projects that I will discuss in this article (and in addition to their other classes), are also expected to read a wide variety of basic books and articles on strategy including everything from Sun Tsu’s The Art Of War to selections from the The Strategikon to Samuel Huntington’s The Clash Of Civilizations. We discuss the network theories of Machiavelli, Clausewitz from the perspective of complex systems, the ethics of right vs. right problems as well as the nuances of the US National Security and National Intelligence Strategies. Beyond the classwork, the project, which cuts across the entire course, is designed to give students an additional opportunity to apply the education they have received at Mercyhurst to a problem of strategic interest to a real world decisionmaker.

I use a simplified version of the National Intelligence Estimate process to structure the project part of the course. Students first receive a very brief description of the various projects available. From this description, they select their preferences (Note: Pedagogically, I consider this self-selection a very important part of the process. I believe that students are more willing to work on projects where they have more rather than less control, including control over the selection of the project on which they are working. While I may not be able to give students their first choice, I try very hard to give them one of their top choices for this reason).

Team sizes are typically four to five students, although in the case of the NIC wiki on disease, the entire class of 26 participated in one project (even there, though, the students broke themselves down into geographically oriented teams of four or five students each). Next, the students meet (generally via teleconference) with their decisionmaker to get a more complete description of the intelligence requirement. From this discussion, the students draft a formal Terms of Reference (an example of such a document is here), which the decisionmaker can either adopt as is, edit him or herself or ask the students to edit. This process, which normally comprises the first three weeks of the 10 week class, results in an agreement, almost a contract, between the analysts and the decisionmaker about what is required and what should be produced.

After a conceptual modeling and a short budget exercise (Note: the budget exercise is hypothetical. The Strategic Intelligence projects do not cost the decisionmakers who sponsor them anything), the students begin collecting information and analyzing it in earnest. In about the tenth week of the project, the students go back to the decisionmaker and formally present their findings, typically in the form of an oral brief (via teleconference, again, in most cases) and a wiki-based product (Note: Not all strategic intelligence products in the past have been wiki based. A little later in this series of posts I will explore some of the quantitative and qualitative differences between wiki-based work and "traditional" methods of producing strategic intelligence products in my class). Generally there is very little contact with the decisionmaker from the time the Terms of Reference are complete to when the project is formally presented. This is rarely due to the unwillingness of the decisionmakers but rather due to the constraints on their time that is usually a natural condition of their position. In addition, the hard work to hammer out the Terms of Reference in the first three weeks, to a very great extent, obviates the need for much additional contact prior to the final presentation.

Tomorrow -- Some Broad Metrics