Teaching Strategic Intelligence Through Games (Final Version With Abstract)

Abstract:

Strategic intelligence is considered by intelligence professionals to be the highest form of the analytic art.   There is a tremendous demand for this type of intelligence product and a lack of trained professionals capable of producing it.  Developing effective teaching methods for this challenging subject, therefore, is an area of ongoing concern for the business, law enforcement and national security intelligence communities.

Previous research suggests that a game-based approach to teaching can be successful but no report so far has examined game-based learning in intelligence analysis.  I hypothesized that a game-based approach to teaching strategic intelligence analysis would increase learning and improve performance while also increasing student satisfaction with the course.

This paper reports the initial results and lessons learned from teaching three full courses (2 undergraduate and one graduate) in strategic intelligence using games as a teaching tool.  The paper will begin by examining the unique challenges in teaching about strategy, strategic decisionmaking and the types of intelligence that supports those efforts.  This will be followed by a short discussion concerning games based learning generally before examining in detail the specific approaches used in these three courses.   

This paper will also examine both the learning outcomes and student satisfaction with the courses.  Finally, this paper will discuss appropriate course modifications for undergraduate and graduate students when teaching advanced subjects with games based on the evidence from this study.
 

Part 1 -- Introduction

Part 2 -- What Is Strategy And What Are Strategic Decisions?

Part 3 -- What Is Intelligence And What Is The Role Of Intelligence In The Formulation Of Strategy? 

Part 4 -- Why Games? 

Part 5 -- How, Specifically, Were Games Used in Class? 

Part 6 -- So, How Did It All Work Out? 

Part 7 -- What Did The Students Think About It?

Part 8 -- What Else Did You Learn? 

Teaching Strategic Intelligence Through Games

Part 7 -- What Did The Students Think About It? (Teaching Strategic Intel Through Games)

Part 1 -- Introduction

Part 2 -- What Is Strategy And What Are Strategic Decisions?

Part 3 -- What Is Intelligence And What Is The Role Of Intelligence In The Formulation Of Strategy? 

Part 4 -- Why Games? 

Part 5 -- How, Specifically, Were Games Used in Class? 

Part 6 -- So, How Did It All Work Out?

The SIRs actually measure a number of variables and identifying those that might be most closely associated with the underlying pedagogy of a course are difficult to identify.  Instead, I chose to look at just one of the SIR-generated ratings, the Overall Evaluation of the course.  This is clearly designed to be an overall indicator of effectiveness.  A large change here (in either the positive or the negative direction) would seem to be a clear indication of success or failure from the student's perspective.

Furthermore, my assumption at the beginning of the course was that there would be a large change in one direction or the other.  I assumed that students would either love this approach or hate it and that this would be reflected in the SIR results.  The chart below, which contains the weighted average of the Overall Evaluation score (1-5 with 5 being best) for all classes taught in a particular year, indicates that I was wrong:

Clearly, while students did not love it, they did not hate it either.  The drop in score from recent years could be attributed to a reduction in satisfaction with the class or it could simply be attributed to the fact that the course changed from a fairly well-oiled series of lectures and exercises to something that had the inevitable squeaks and bumps of a new approach.  Feedback from the student surveys given after the course was over, while extremely helpful in providing suggestions for improving the class, gave no real insight into the causes of this modest but obvious drop in student satisfaction.

Comparing this chart with the previous one concerning the quality of the final product yields an even more interesting picture:

This chart seems to be saying that the more a student thinks they are getting out of class (as represented in their Overall Evaluation of the course) the better their final strategic intelligence project is likely to be.  This holds true, it seems, as long as strategic intelligence is taught through more or less traditional methods of lecture, discussion and classroom exercises.  Once the underlying structure of the course is centered on games, however, the students are less satisfied but actually perform better where it matters most – on real-live projects for real-world decisionmakers.

Taken at face value (and ignoring, for the moment, the possibility that this is all a statistical anomaly), a possible explanation is that the students don’t realize what they are getting “for free” from the games-based approach.  Other researchers have noted that information that had to be actively taught, assessed, re-taught and re-assessed in other teaching methods is passively (and painlessly) acquired in a games-based environment. 

I noted this effect myself in my thesis research into modeling and simulating transitions from authoritarian rule.  My goal, in that study, was to develop a predictive model; not to teach students about the target country.  One of my ancillary results, however, was that students routinely claimed that they learned more about the target country in three hours of playing the game than in a semester’s worth of study. 

This “knowledge for free” aspect of the games-based model was nowhere more obvious than in the fairly detailed understanding of the geography of the western part of the Soviet Union acquired by the students in all three classes while playing the boardgame, Defiant Russia.  While this information was available in the form of the game map, learning the geography was not explicitly part of the instructions.  Students rapidly understood, however, that they had to understand the terrain in order to maximize their results within the game.  Furthermore, an understanding of the geography of the western part of the Soviet Union was critical to the formulation of strategic options. 

This raises a broader question regarding games based learning:  If students don't know they are learning, how can they evaluate the learning process?  While I have not had time to dig deeply into the literature regarding implicit learning, I intend to.  Giving students a tangible sense of what they are learning in a game based environment may be one of the biggest challenges to overcome with the approach, at least in higher education.

Next: 

What else did you learn?

Teaching Strategic Intelligence Through Games (Original Research)

(Note:  This is another in a series of posts that I refer to as “experimental scholarship” -- or using the medium of the internet and the vehicle of this blog as a way to put my research online for more or less real-time peer review. Earlier examples of this genre include: A Wiki Is Like A Room..., The Revolution Begins On Page 5, What Is Intelligence? and What Do Words Of Estimative Probability Mean?.  

This series of posts is based off a paper I gave at the Game Education Summit last week.)

Strategic intelligence is considered by intelligence professionals to be the highest form of the analytic art.   There is a tremendous need for this type of intelligence product and a lack of trained professionals capable of producing it.  Developing effective teaching methods for this challenging subject, therefore, is an area of ongoing concern for the business, law enforcement and national security intelligence communities.

Previous research (cited in detail in later posts) suggests that a game-based approach to teaching can be successful but no report so far has examined game-based learning in teaching intelligence analysis.  I hypothesized that a game-based approach to teaching strategic intelligence analysis would increase learning and improve performance while also increasing student satisfaction with the course.

This series of posts reports the initial results and lessons learned from teaching three full courses (2 undergraduate and one graduate) in strategic intelligence using games as a teaching tool.  This series of posts will begin by examining the unique challenges in teaching strategy, strategic decisionmaking and the types of intelligence that supports those efforts.  This will be followed by a short discussion concerning games-based learning generally before examining in detail the specific approaches used in these three courses. 

 

This series of posts will also examine both the learning outcomes and student satisfaction with the courses.  Finally, this series of posts will discuss appropriate course modifications for undergraduate and graduate students when teaching advanced subjects with games based on the evidence from this study.

Next:
What is strategy and what are strategic decisions?

What's In A Name? (Game Education Summit -- Part 2)

Don Marinelli kicked off the Game Education Summit with a rousing "hoo-ah" speech for the academics and game design professionals in the room. He wasn't entirely preaching to the choir but it certainly seemed he knew his audience.

Beyond the high rhetoric (such as video games are the "Lucifer of Liesure" to naysayers and luddites...) there were a number of interesting points he brought up. Rather than focus on the entire speech (which, as I mentioned, was designed as much to inspire as to inform), I intend to simply hit the high points from my perspective:

The intersection between games and education. Marinelli was primarily talking about teaching game design at the University level but he spent a good bit of the speech highlighting the more general importance of games in education. He was quite passionate about it and I think generally correct. Games excel at implicit learning -- learning without knowing that you are learning. Taking advantage of that makes sense.

The inevitability of game-based learning. The shift to game based learning in Marinelli's view is largely generational. As the currnet generation (comfortable with chalkboards, lectures and linearity in general) gives away to the next (comfortable with GPS, SMS and high speed internet) there will inevitably be a shift in the way we educate people. Students want to know that they invested their youth wisely. I couldn't help but think at this point of various theories about why animals play. At least one of the theories is that they play to learn.

Intelligence studies and game design share a similar problem -- perception. Marinelli seemed to be saying that games are considered "fun" and game deigners too often feel like they have to apologize for their profession. Likewise, it occurred to me that intelligence is all too often considered "bad" and intelligence professionals similarly feel a need to apologize for their profession (It actually happened to me here. I asked a question and the presenter asked back, "What is it you teach?" My answer: "You don't want to know." And the cock crowed the first time...).