Why The Most Important Question In Game-based Learning Is "Who Will Fund The Game Genome Project?" (Part 1 of 2)

Way back in 2000, two researchers, Will Glaser and Tim Westergren, began what was then called The Music Genome Project.  It was designed to categorize music by more than 400 different "genes" or characteristics of the music.  The goal was to build a better music recommendation engine.

Today, this project is better known as Pandora.

Glaser and Westergren's fundamental insight was that breaking music down into broad general categories such as Rock or Pop or Country wasn't very useful when it came to making recommendations.  Some people liked music with male vocalists or heavy beats or a fast tempo and no one liked all of country music or everything produced that was labelled "rock".  

In fact most people liked a little bit of everything.  Sure, they had genre preferences, but that didn't keep the Jethro Tull fanatic from liking (and buying) the occasional Mike Oldfield album (ahem...not that I know anyone who would do such a thing...).

Thus the Music Genome Project was born.  By analyzing the genetic makeup of each song, the Project wasn't just able to better dissect individual pieces of music.  It was actually able to make reliable cross genre recommendations.  Oh, you like this driving, 120 beats per minute, sung by a female vocalist with lots of guitar distortion rock anthem?  Then you might also like this hip-hop track with many of the same musical genes!

What Does This Have To Do With Game-based Learning?

This isn't going to sound that earthshaking but it was to me the first time I realized it:  All games teach.  You can design a game that will explicitly (or implicitly) teach something like math or grammar but you don't have to.  With all of the good games, both video and tabletop, that are out there, it is not difficult to find a game that can be used to teach almost any K-12 and many university level subjects.  

How many classrooms routinely use Monopoly, for example, to help teach basic addition and subtraction or units of currency?  Monopoly certainly wasn't designed with this purpose in mind but it serves that purpose nonetheless.  

While I might be bold in my assertion that every subject is covered, I would argue that, if I am wrong, I am not wrong by much.  This is the golden age of gaming.  There are more games being produced (and more good games) than at any other time in human history.  The selection is already immense and growing.  In fact, it might be more accurate for me to say that, while I might be wrong, I won't be for much longer.

So, to put it more formally, you can connect all games to one or more learning objectives (See image to the left).  I am using the term "learning objective" loosely here.  Your learning objectives may come from a formal document, such as the common core, or from a less formal desire "to teach these darn kids something about X".  

Given the prevalence of formal standards in modern education, however, it is pretty easy to imagine (though infinitely less easy to actually do...) professional educators and gamers sitting down together and dissecting every game for the learning objectives that each game addresses (i.e. the things each game teaches).

Eventually - and, of course, you would start with the most popular games and the most important learning objectives - you would have a database that could answer the question, "What game teaches this?"  Almost certainly, multiple games will cover the same learning objectives and some games will cover more relevant learning objectives than others.  It is conceivable that a teacher would be able to query this database and find a single game (See image below) that adequately addressed all of the learning objectives for a particular block of instruction.

Next:  What's Missing From These Pictures?

New Wikipedia Articles Of Interest To Intelligence Professionals

Despite its occasional weaknesses, I really like Wikipedia.  Others (perhaps unnecessarily) worry about an encyclopedia that is editable by anyone.  Whether you like it or not, however, it is undeniably the tertiary source of first resort for most of the planet.  

One of the things that has always bothered me about it, though, is the generally poor coverage of issues related to intelligence.  From intelligence history to intelligence theory, Wikipedia, in my opinion, needs help.

That is why, instead of traditional writing assignments in some of my classes, I like to task students to write Wikipedia articles about intelligence issues that have not already been covered.  

This kind of assignment has a variety of educational benefits.  In addition to adding to the world's body of knowledge, the students have to learn how to use MediaWiki (the same platform that powers Intellipedia and many other wikis in the in the private sector).  

They also have to learn how to write an encyclopedia article complete with Wikipedia's famous "Neutral Point Of View" - a skill that is enormously useful in intel writing as well.  

Finally, they have to expose their work to the varied and critical audience that makes up the ad hoc Wikipedia editorial staff.  This is more important to the learning process than you might think.  Students typically master the skill of gaming their professors pretty quickly.  Writing for an army of discerning, anonymous editors?  Not so much.

So, without further ado, here are a handful of articles recently produced by students in my Collection Operations for Intelligence Analysts class.  The mix is eclectic because I let the students pick their own topics but is, perhaps, more interesting as a result.  

• HUMINT/CI Exploitation Teams
• Michael Soussoudis
• Gang Intelligence Units
• Nimitz Operational Intelligence Center
• Information Board (Estonia)

This handful only represents some of the output from last term.  Some of the articles are still in Wikipedia's increasingly lengthy review process.  I will publish those once they become available.

Intelligence And Ethics: Now You Can Check The Codes!

Some would say that ethics in intelligence is an oxymoron.  

Certainly, the ethical challenges faced by today's intelligence professionals are more severe than any other time in human history.

It is interesting to note, in this regard, that all three of the major sub-disciplines of intelligence (national security, law enforcement and business) now have publicly available codes of ethics for their practitioners.

The most recent of these is from the newly minted National Intelligence Strategy:

As members of the intelligence profession, we conduct ourselves in accordance with certain basic principles. These principles are stated below, and reflect the standard of ethical conduct expected of all Intelligence Community personnel, regardless of individual role or agency affiliation. Many of these principles are also reflected in other documents that we look to for guidance, such as statements of core values, and the Code of Conduct: Principles of Ethical Conduct for Government Officers and Employees; it is nonetheless important for the Intelligence Community to set forth in a single statement the fundamental ethical principles that unite us and distinguish us as intelligence professionals. 

MISSION. We serve the American people, and understand that our mission requires
selfless dedication to the security of our nation. 

TRUTH. We seek the truth; speak truth to power; and obtain, analyze, and provide
intelligence objectively. 

LAWFULNESS. We support and defend the Constitution, and comply with the laws of the United States, ensuring that we carry out our mission in a manner that respects privacy, civil liberties, and human rights obligations. 

INTEGRITY. We demonstrate integrity in our conduct, mindful that all our actions, whether public or not, should reflect positively on the Intelligence Community at large. 

STEWARDSHIP. We are responsible stewards of the public trust; we use intelligence
authorities and resources prudently, protect intelligence sources and methods diligently,
report wrongdoing through appropriate channels; and remain accountable to ourselves,
our oversight institutions, and through those institutions, ultimately to the American people. 

EXCELLENCE. We seek to improve our performance and our craft continuously, share
information responsibly, collaborate with our colleagues, and demonstrate innovation and agility when meeting new challenges. 

DIVERSITY. We embrace the diversity of our nation, promote diversity and inclusion in our workforce, and encourage diversity in our thinking.

The Society of Competitive Intelligence Professionals has long had a code:

• To continually strive to increase the recognition and respect of the profession.
• To comply with all applicable laws, domestic and international.
• To accurately disclose all relevant information, including one's identity and organization, prior to all interviews.
• To avoid conflicts of interest in fulfilling one's duties.
• To provide honest and realistic recommendations and conclusions in the execution of one's duties.
• To promote this code of ethics within one's company, with third-party contractors and within the entire profession.
• To faithfully adhere to and abide by one's company policies, objectives and guidelines.

Finally, the International Association of Crime Analysts offers this as ethical guidelines to its members:

Theoretically today’s professional crime analyst is expected to know the details of every crime in his or her jurisdiction, and often to predict when the next will occur. In reality, the title of crime analyst can mean different things even in the same agency. Generally, a crime analyst is one who monitors crime trends and patterns, researches and analyzes similarities and differences in crime details, and reports those findings to the appropriate personnel that can address those crimes either through deterrence or prevention. Many skills and abilities are necessary to complete the crime analysis process. Necessary skills include logic and critical thinking, research skills, organizational skills to organize facts and findings, written and oral communication skills, and computer skills. Necessary personal traits include a desire to aid in the reduction of crime through the legal and ethical examination of crime facts and data.
The professional crime analyst assists law enforcement managers in decision making, supports street officers and detectives with information helpful to their jobs, and provides service to other crime analysts and to the general public. As professional crime analysts, we commit ourselves to the following principles: 

Personal Integrity
    Maintain an attitude of professionalism and integrity by striving to perform at the highest level of one’s proficiency and competency, in order to achieve the highest level of quality.
    Remain honest and never knowingly misrepresent facts.
    Accurately represent one’s own professional qualifications and abilities, and ensure that others receive due credit for their work and contributions.
    Seek and accept honest criticism for one’s work, and take personal responsibility for one’s errors.
    Treat all persons fairly regardless of age, race, religion, gender, disability, sexual orientation, or nation of origin.
    Practice integrity and do not be unduly swayed by the demands of others. 

Loyalty to One’s Agency
    Safeguard the privacy and confidentiality of restricted information or documents.
    Thoroughly research, analyze, prepare, and disseminate quality work products to the best of one’s ability, ensuring that all reports and documents are accurate, clear, and concise.
    Faithfully adhere to and abide by one’s departmental policies, objectives, and guidelines. Support colleagues in the execution of their lawful duties, and oppose any improper behavior, reporting it where appropriate. 

Commitment to the Crime Analysis Profession
    Continually strive to increase the recognition of and respect for the profession by participating in professional crime analysis associations, contributing time and effort toward their goals and missions.
    Advocate professional, research-based crime analysis functions within the law enforcement environment.
    Seek training and education throughout one’s career; remain current with trends and practices in crime analysis.
    Contribute to the professional education, training, and development of other crime analysts. Share information and the results of research and development by responding to information requests, submitting information to individuals and organizations, participating in meetings, or publishing articles.
    Present methodologies, results and recommendations through fair, honest, conscientious, independent and impartial judgment.
    Exercise objectivity, impartiality, accuracy, validity and consistency in all research conducted. 

If you are looking for an interesting exercise, have your students or colleagues try to apply all three codes of ethics to this situation.

Will This New Game Genre Change Intelligence Training, Education?

Most gamers understand that games fall into genres.  For example, Scrabble, Boggle and my own game, Widget, are all examples of "word games".  There is no standardized list of game genres, of course, but gamers are like Supreme Court Justice Potter Stewart when it comes to genres (Stewart, in trying to define pornography, famously wrote in Jacobellis v. Ohio, "I know it when I see it.").

Most games, then, fit neatly into existing genres and truly new genres come along only rarely.  It is even more rare for a new genre of games to have a large-scale cultural or social impact.  The last such genre that I can think of was the role-playing game, epitomized by the first and still one of the most popular games, Dungeons and Dragons.  Whether you played D and D or not (or liked it if you did play it), there is no denying that it spawned a genre of games that impacted and continue to impact both culture and society.

Today there is a new genre of games - cooperative tabletop games - that I think has a chance to have a similar impact on the way we teach not just intelligence but just about everything.

Cooperative games are labelled as such because players cooperate with each other to defeat the game.  This kind of play style has long been a staple of many video games where players will gather as teams to defeat a common enemy.

While there are a few examples that date back as far as the 1980's, modern cooperative tabletop games typically require much more nuanced gameplay than their video game counterparts.  True cooperation on everything from strategy to resources is usually necessary to defeat these challenging games.  

If you are not familiar with this genre (and most people are not), I strongly recommend you get some of these games and play them.  Two good examples to start with are Pandemic and Forbidden Desert.  Both games pit you and the rest of the players in a race to beat the game.  Either everyone wins or no one wins.

There are many variations on the theme but typically these games throw an escalating series of challenges at the players.  Pandemic, for example, envisions a team of experts working to stop a global disease epidemic.  Forbidden Desert asks players to collect a series of artifacts and escape the desert before sandstorms swallow the players.

Players in these games usually assume a variety of roles, such as Engineer or Medic, each with a particular skill useful in defeating whatever it is the game throws at them.  Players can and do discuss everything from strategy to resource allocation.  This kind of game doesn't just encourage cooperation but demands it from every player.

My recent game, Spymaster (which has proved incredibly popular - I have given out nearly 200 copies to date), was designed as such a game.  Small groups of players have to make collaborative decisions about how and where to place certain collection assets in order to collect various information requirements, all while losing the fewest possible assets.  While the current version of Spymaster allows the players to determine how they will make decisions about asset allocation, I am thinking about an "advanced" version of the game that will assign various roles to the players coupled, of course, with unique capabilities associated with each role.

Whether you have had a chance to play Spymaster or not, once you have played a couple of these kinds of games, the possibilities for their use in class becomes very apparent.  There is a lot of learning going on in these games and not all of it is knowledge-based.  Teamwork, conflict management and collaboration are all essential elements of these games.

More importantly for classroom use, these games can be designed to take a relativity small amount of time to play.  Unlike videogames, tabletop games also tend to expose the underlying system to the players in a bit more detail.  Likewise, tabletop games are vastly less expensive to design and produce than videogames which means that more topics could be covered for the same or less money - clearly a consideration in these budget restricted times.  Finally, bringing a tabletop game into a secure facility is vastly easier than trying to import electrons.

Do I really think that cooperative tabletop games will change intel training and education?  I'm not sure, but I know that they can - and that this is an experiment in games-based learning worth attempting.

Realism, Playability And Games In The Intelligence Classroom

A couple of weeks ago, I made a print-and-play version of my new game about collection management, Spymaster, available to anyone who reads this blog and would drop me an email (The offer is still open, by the way, in case you missed it the first time).

Since then, I have mailed out over 100 copies to everyone from the DNI's office to troops deployed in Afghanistan to academics in Japan to the Norwegian police forces!

Feedback is starting to trickle in and the comments have been largely positive (whew!) even from some very experienced collection managers (Thanks!).  In addition, I have received a number of outstanding suggestions for enhancing or improving the game.  Some of these include:

• Making different collection assets work better or worse against different information requirements.
• Increasing the point value of information requirements collected early.
• Making some of the OSINT cards "Burn - 0" or impossible to burn.
• Giving players a budget and assigning dollar values to each collection asset such that players had to stay within their budget as well.

I recognize that these suggestions may not make much sense if you haven't played the game but all of them (plus many more) are fantastic ideas designed to make the game more real.  And therein lies the rub...

One of the classic problems of games designed to simulate some aspect of the real world is the trade-off between realism and playability.  Playability is really just how easy it is to play the game.  Every time you add a new rule to make the game more realistic, you make the game more difficult to play and therefore less playable.  Its not quite as simple as that but it gives you a good idea of how the problem manifests itself.  Great games designed to simulate reality often give a strong sense of realism while remaining relatively simple but the truth of it is, like the Heisenberg Uncertainty Principle, the more you try to do one, the less, typically, you are able to do the other.

The problem of playability versus realism is analogous to the problem of feature creep in project management.  Most people have been involved in a project that started out simple but, over time, grew incredibly complex as more and more "good ideas" were added.  Each idea, in and of itself, was justifiable but, in the end, led to an unwieldy mess.

Figuring out where to draw the line is just as important in game design as it is in project management.  This constraint is even more strict when considering the modern intelligence classroom.  Here, unless the course is entitled "collection management", there is likely a highly limited amount of time to devote to a game on collection management.  

Consider the case of Spymaster.  I wanted a game which would replace a one-hour lecture on collection management for our intro classes.  To make this work, I would need to be able to set-up the game, explain the rules, play the game and then conduct an outbrief all within an hour.  That's pretty tough to do (at least for me) and still make the game meet your learning objectives.  It becomes a very careful balance of putting good ideas into the game while not running out of time to play the game in class.

The classic solution to this problem is to have a basic version and an advanced version (or several advanced versions).  These can be included in the rules from the outset or added later as expansion packs.  Right now, this is exactly what I am doing with all of the feedback I am receiving - scouring it for good ideas I want to put into more advanced versions of Spymaster!

Thinking in Parallel (Part Three - Testing The Mercyhurst Model Against The Real World)

Part 1 -- Introduction
Part 2 -- The Mercyhurst Model

For the last 11 years, I have been using the model described in Part 2 to structure my Strategic Intelligence class at Mercyhurst University.  This is a capstone class for seniors and 2nd year graduate students within the Intelligence Studies program at Mercyhurst.  This class is centered on a real world project for a real-world decisionmaker, often within the US National Security Community.  To date, I have overseen 133 of these types of projects.

The broad parameters of the projects have remain unchanged since 2003.  Students in the class are divided into teams and are assigned by the instructor to one of 4-5 projects available during that term.  Each project is sponsored by a national security, business, or law enforcement organization that has a strategic intelligence question.  To date, sponsors of these questions have included organizations such as the National Geospatial-intelligence Agency, the Defense Intelligence Agency, the National Intelligence Council, the National Security Agency, 66th Military Intelligence Group, and the Navy’s Criminal Investigative Service to name just a few.  To give readers a sense of the wide variety of questions intelligence studies students are expected to answer in this course, I have listed a few recent examples of them below:

1. What role will non-state actors (NSAs) play and what impact will NSAs have in Sub-Saharan Africa over the next five years?o What is the likely importance of NSAs vs. State Actors, Supra-State Actors and other relevant categories of actors in sub Saharan Africa?o What are the roles of these actors in key countries, such as Niger?o Are there geographic, cultural, economic or other patterns of activity along which the roles of these actors are either very different or strikingly similar?o What analytical processes and methodologies were applied to the questions above and which proved to be effective or ineffective? 

2. What are the most important and most likely impacts on, and threats to, US national interests (including but not limited to political, military, economic and social interests) resulting from infectious and chronic human disease originating outside the US over the next 10-15 years?  

3. What are the likely trends in Brazil’s oil/liquid fuel market and electric power sector in the next ten years?  Where will these trends likely manifest themselves?o What energy capacity and security issues are likely to be the most significant to Brazil’s economy in the next ten years?o How will Brazil likely address current and/or future energy security issues over the next ten years?o Where will Brazil address these energy shortfalls?

In each case, students had only 10 weeks to conduct the research, write the analysis and present the final product to the decisionmaker.  The students had no additional financial resources available to them and, other than the question itself, received no support directly from the decisionmaker.  Students rarely had any subject matter expertise in the area under question and were only allowed to use open sources.  Students were expected to integrate lessons learned from all previous intelligence studies classes and to manage all aspects of the project without significant supervision.  Finally, all the students, in addition to this project, were taking a full academic load at the same time.  

After all of the deliverables had been produced and disseminated, the decisionmakers sponsoring the projects were asked to provide objective feedback directly to the course instructor.  This feedback, in turn, was evaluated on a five point scale correlated with traditional grading practices and professional expectations.  In short, a 3 on this scale is roughly equivalent to a "B" and a “4” on this scale is roughly equal to “A” work in a university setting.  A “5”, on the other hand, is the kind of work that would be expected from a working (albeit junior) intelligence professional.  The chart below indicates how the annual averages have changed over time.

(Note:   While this chart may appear to reflect grade inflation more than any other suggested effect, it should be noted that “A” is essentially “average” among Mercyhurst University Intelligence Studies seniors and 2nd Year graduate students and has been for the entire time frame shown above.  The current dropout rate from the program is approximately 50% and much of that is due to a strict 3.0 minimum GPA in order to stay in the program.  As a result, seniors and second year graduate students (the only students allowed to take the class), typically have GPAs that average 3.6 or above.  For example, two years ago, 18 of the top 20 GPA’s in the entire University belonged to Intelligence Studies students.)

Anecdotally, it is possible to state the exact impact of these reports within national security agencies in only a few cases.  For example, the report that answered the question on global health mentioned earlier earned this praise from the National Intelligence Council: 

“Although the Mercyhurst "NIE" should not be construed as an official U.S. government publication, we consider this product an invaluable contribution to the NIC's global disease project: not only in terms of content, but also for the insights it provides into methodological approaches. The Mercyhurst experience was also an important lesson in how wikis can be successfully deployed to facilitate such a multifaceted and participatory research project.”

Likewise, in David Moore’s book Sensemaking:  A Structure for an Intelligence Revolution (published by the National Defense Intelligence College in 2011), the study on non-state actors in sub-Saharan actors produced in answer to the question mentioned above was judged more rigorous than a similar study conducted by the National Intelligence Council (in cooperation with the Eurasia Group).

Beyond the national security community, however, the impact of these reports on various businesses and other organizations is often easier to determine.  For example, senior managers at Composiflex, a mid-sized composites manufacturer, indicated, “We used this project as a seed for our new marketing plan in 2007 and now an industry that we had not even tapped before is 30% of our business.” 

Likewise, Joel Deuterman, the CEO of Velocity.net, an Internet Service Provider, stated, “The analysts discovered that our approach was actually a cutting-edge, developing standard in our industry…What really substantiated the data for us was to see many of our existing customers on the list. Then we knew we could rely on the validity of the ones they had found for us.”  

Even foreign organizations have seen the benefit of these products including Ben Rawlence, the Advisor for Foreign Affairs and Defense in the Whip’s Office of the Liberal Democrat Party in the UK, stating, “The research carried out by your students was first class, and has been of substantial use to Members of Parliament…  It was comprehensive, well sourced and intelligently put together.  I have had no hesitation recommending it to our MPs and Lords in the same way that I recommend briefings provided for us by professional research organisations…” 

While it is possible to imagine more rigorous testing of this model of the intelligence process, the long term success of the process in generating actionable intelligence for a wide variety of customers on a range of difficult problems in a very short time using limited resources is hard to ignore.  More importantly, not only has the process proven itself successful but this success has trended upwards as improvements have been made over the years in terms of structuring the course and teaching material consistent with this approach to the intelligence process.

*****

Intelligence in the 21st century is best thought of as a series of sub-processes operating interactively and in parallel.  

This conclusion, by itself, has significant implications for the training and education of intelligence professionals.  In the first place, it suggests that it is no longer possible to specialize in one area to the exclusion of another.  Intelligence professionals will have to be trained to think more broadly, to be able to jump more fluidly from modeling to collection to analysis to production and back as the process of creating intelligence moves forward over time.  

Likewise, hardware and software support systems will need to be designed that facilitate this leaping back and forth between the various sub-processes.  Designing products that work sequentially in a parallel world will not only frustrate but will also slow down the process of generating intelligence – a result that is absolutely counter to the intelligence needs of modern decisionmakers.  

Finally, as dramatic as this type of change might appear to be, it is, perhaps, better thought of as merely aligning the training and education of intelligence professionals with what it is they already do.

Thinking In Parallel (Part 2 - The Mercyhurst Model)

Part 1 -- Introduction

While a number of tweaks and modifications to the cycle have been proposed over the years , very few professionals or academics have recommended wholesale abandonment of this vision of the intelligence process.  

This is odd.  

Other fields routinely modify and improve their processes in order to remain more competitive or productive.  The US Army, for example, has gone through several major revisions to its combat doctrine over the last 30 years, from the Active Defense Doctrine of the 1970’s to the AirLand Battle Doctrine of the 80’s and 90’s to Network Centric Operations in the early part of the 21st Century.  The model of the intelligence process, the Intelligence Cycle, however, has largely remained the same throughout this period despite the criticisms leveled against it.  The best answers, then, to the questions, “What is the intelligence process?” and “What should the Intelligence process be?” remain open theoretical questions, ripe for examination.

There are common themes, however, that emerge from this discussion of process.  These themes dictate, in my mind, that a complete understanding of the intelligence process must always include both an understanding of intelligence's role in relationship to both operations and the decisionmaker and an understanding of how intelligence products are created.  Likewise, I believe that the process of creating intelligence is best visualized as a parallel rather than as a sequential process.  I call this the "Mercyhurst Model" and believe it is a better way to do intelligence.  More importantly, I think I have the evidence to back that statement up.  

The first of the common themes referenced above is that the center of the process should be an interactive relationship between operations, the decisionmaker and the intelligence unit.  It is very clear that the intelligence process cannot be viewed in a vacuum.  If it is correct to talk about an “intelligence process” on one side of the coin, it is equally important for intelligence professionals to realize that there is a operational process, just as large if not larger and equally important if not more so, on the other side and a decisionmaking process that includes both.

The operational and intelligence processes overlap in significant ways, particularly with respect to the purpose and the goals of the individual or organization they support.  The intelligence professional is, however, focused externally and attempts to answer questions such as “What is the enemy up to?” and “What are the threats and opportunities in my environment?”  The decisionmaking side of the coin is more focused on questions such as “How will we organize ourselves to take advantage of the opportunity or to mitigate the threat?” and “How do we optimize the use of our own resources to accomplish our objectives?”  In many ways, the fundamental intelligence question is “What are they likely to do?” and the decisionmaker’s question is “What are we going to do?”  The image below suggests this relationship graphically.

The second theme is that it should be from this shared vision of the organization’s purpose and goals that intelligence requirements “emerge”.  With few exceptions, there does not seem to be much concern among the various authors who have written about the intelligence process about where requirements come from.  While most acknowledge that they generally come from the decisionmakers or operators who have questions or need estimates to help them make decisions, it also seems to be appropriate for intelligence professionals to raise issues or provide information that was not specifically requested when relevant to the goals and purpose of the organization.  In short, there seems to be room for both “I need this” coming from a decisionmaker and for “I thought you would want to know this” coming from the intelligence professional as long as it is relevant to the organization’s goals and purposes.

Theoretically, at least, the shared vision of the goals and purpose of the organization should drive decisionmaker feedback as well.  The theoretical possibility of feedback, however, is regularly compared with the common perception of reality, at least within the US national security community, that feedback is ad hoc at best.  There, the intelligence professionals preparing the intelligence are oftentimes so distant from the decisionmakers they are supporting that feedback is a rare occurrence and, if it comes at all, is typically only when there has been a flaw in the analysis or products.  As former Deputy Director Of National Intelligence for Analysis, Thomas Fingar (among others), has noted, “There are only two possibilities: policy success and intelligence failure” suggesting that “bad” intelligence is often a convenient whipping boy for poor decisions while “good” intelligence rarely gets credit for the eventual decisionmaker successes.

It is questionable whether this perception of reality applies throughout the intelligence discipline or even within the broader national security community.  Particularly on a tactical level, where the intelligence professional often shares the same foxhole, as it were, with the decisionmaker, it becomes obvious relatively quickly how accurate and how useful the intelligence provided is to the operators.   While most intelligence professionals subscribe to the poor feedback theory, most intelligence professionals also have a story or two about how they were able to give analysis to decisionmakers and how that analysis made a real difference, a difference willingly acknowledged by that decisionmaker.  The key to this kind of feedback seems less related to the issue or to intelligence writ large and more related to how closely tied are the intelligence and decisionmaking functions.  The more distance between the two, the less feedback, unsurprisingly, there is likely to be.

The third theme, is that from the requirement also emerges a mental model in the mind of the intelligence professional regarding the kinds of information that the he or she needs in order to address the requirement.  This model, whether implicit or explicit, emerges as the intelligence professional thinks about how best to answer the question and is constructed in the mind of the intelligence professional based on previous knowledge and the professional’s understanding of the question.  

This mental model typically contains at least two kinds of information; information already known and information that needs to be gathered.  Analysts rarely start with a completely blank slate.  In fact, Phillip Tetlock has demonstrated that a relatively high level of general knowledge about the world significantly improves forecasting accuracy across any domain of knowledge, even highly specialized ones (Counter-intuitively, he also offers good evidence to suggest that high degrees of specialized knowledge, even within the domain under investigation does not add significantly to forecasting accuracy). 

The mental model is more than just an outline, however.  It is where biases and mental shortcuts are most likely to impact the analysis.  It is where divergent thinking strategies are most likely to benefit and where their opposites, convergent thinking strategies such as grouping, prioritizing and filtering, need to be most carefully applied.  One of the true benefits of this model over the traditional Intelligence Cycle is that it explicitly includes humans in the loop - both what they do well and what they don't.

Almost as soon as the requirement gains enough form to be answerable, however, and even if it continues to be modified as a result of an exchange or series of exchanges between the decisionmakers and the intelligence professionals, four processes, operating in parallel, start to take hold: The modeling process we just discussed, collection (in a broad sense) of additional relevant information, analysis of that information with the requirement in mind and early ideas about production (i.e. how the final product will look, feel and be disseminated in order to facilitate communicating the results to the decisionmaker).

The notional graphic below visualizes the relationship between these four factors over the life of an intelligence product.  Such a product might have a short suspense (or due date) as in the case of a crisis or a lengthier timeline, as in the case of most strategic reports, but the fundamental relationship between the four functions will remain the same.  All four begin almost immediately but, through the course of the project, the amount of time spent focused on each function will change, with each function dominating the overall process at some point.  The key, however, is that these four major functions operate in parallel rather than in sequence, with each factor informing and influencing the other three at any given point in the process.

A good example of how these four functions interrelate is your own internal dialogue when someone asks you a question.  Understanding the question is clearly the first part followed almost immediately by a usually unconscious realization of what it would take to answer the question along with a basic understanding of the form that answer needs to take.  You might recall information from memory but you also realize that there are certain facts you might need to check out before you answer the question.  If the question is more than a simple fact–based question, you would probably have to do at least some type of analysis before framing the answer in a form that would most effectively communicate your thoughts to the person asking the question.  You would likely speak differently to a child than you would to an adult, for example, and, if the question pertained to a sport, you would likely answer the question differently when speaking with a rabid fan than to a foreigner who knew nothing about that particular sport.

This model of the process concludes then where it started, back with the relationship between the decisionmaker, the intelligence professional and the goals and purposes of the organization.  The question here is not requirements, however, but feedback.  The intelligence products the intelligence unit produced were, ultimately, either useful or not.  The feedback that results from the execution of the intelligence process will impact, in many ways, the types of requirements put to the intelligence unit in the future, the methods and processes the unit will use to address those requirements and the way in which the decisionmaker will view future products.

This model envisions the intelligence process as one where everything, to one degree or another, is happening at once.  It starts with the primacy of the relationship between the intelligence professional and the decisionmakers those professionals support.  It broadens and redefines, however, those few generally agreed upon functions of the intelligence cycle but sees them as operating in parallel with each taking precedence in more or less predictable ways throughout the process.  This model, however, explicitly adds the creation and refinement of the mental model of the requirement created by the intelligence unit as an essential part of the process.  This combined approach captures the best of the old and new ways of thinking about the process of intelligence.  Does it, however, test well against the reality of intelligence as it is performed on real-world intelligence problems?

Part Three - Testing The Mercyhurst Model Against The Real World

Thinking In Parallel: A 21st Century Vision Of The Intelligence Process

(Note:  I recently was asked to present a paper on my thoughts about re-defining the intelligence process and the implications of that redefinition on education, training and integration across the community at the US Intelligence Community's Geospatial Training Council's (CGTC) conference in Washington DC.  For those familiar with my earlier work in the intelligence cycle and the damage it is causing, you will find this paper shorter and less about the Cycle and more about the alternative to it I am proposing (and the evidence to support the adoption of that alternative...).  Enjoy!)

Abstract:  Effective integration and information sharing within the intelligence community is not possible until the fundamental process of intelligence is re-imagined for the 21st Century.  The current model, the Intelligence Cycle, developed in World War 2 and widely criticized, has outlived its useful life.  In fact, it has become part of the problem.  This paper abandons this sequential process that was appropriate for a slower and less information rich environment.  Instead, a more streamlined parallel process is proposed.  Accompanying this new vision of the intelligence process will be an analysis of data collected from over 130 real-world intelligence projects conducted using this model of the intelligence process and delivered to decisionmakers in the national security (including GEOINT), law enforcement and business sectors.  Additionally, the training and education implications as well as the kinds of software and hardware systems necessary to support this new understanding of the process are discussed.

Part 1 -- Introduction

"We must begin by redefining the traditional linear intelligence cycle, which is more a manifestation of the bureaucratic structure of the intelligence community than a description of the intelligence exploitation process." -- Eliot Jardines, former head of the Open Source Center, in prepared testimony in front of Congress, 2005  

"When it came time to start writing about intelligence, a practice I began in my later years at the CIA, I realized that there were serious problems with the intelligence cycle.  It is really not a very good description of the ways in which the intelligence process works."  Arthur Hulnick, "What's Wrong With The Intelligence Cycle", Strategic Intelligence, Vol. 1 (Loch Johnson, ed), 2007

"Although meant to be little more than a quick schematic presentation, the CIA diagram [of the intelligence cycle] misrepresents some aspects and misses many others." -- Mark Lowenthal, Intelligence:  From Secrets to Policy (2nd Ed.,2003) 

"Over the years, the intelligence cycle has become somewhat of a theological concept:  No one questions its validity.  Yet, when pressed, many intelligence officers admit that the intelligence process, 'really doesn't work that way.'" -- Robert Clark, Intelligence Analysis:  A Target-centric Approach, 2010

Academics have noted it and professionals have confirmed it:  Our current best depiction of the intelligence process, the so-called "intelligence cycle", is fatally flawed.  Moreover, I believe these flaws have become so severe, so grievous, that continued adherence to and promotion of the cycle is actually counterproductive.  In this paper I intend to briefly outline the main flaws in the intelligence cycle, to discuss how the continued use of the cycle hampers, indeed extinguishes, efforts to effectively integrate and share information and, finally, suggest an alternative process – a parallel process – that, if adopted, would transform intelligence training and education.

*****

Despite its popularity, the history of the cycle is unclear.  US army regulations published during WWI identify collection, collation and dissemination of military intelligence as essential duties of what was then called the Military Intelligence Division but there was no suggestion that these three functions happen in a sequence, much less in a cycle.

By 1926, military intelligence officers were recommending four distinct functions for tactical combat intelligence:  Requirements, collection, "utilization" (i.e. analysis), and dissemination, though, again, there was no explicit mention of an intelligence cycle.

The first direct mention of the intelligence cycle (see image) is from the 1948 book, Intelligence Is For Commanders.  Since that time, the cycle, as a model of how intelligence works, has become pervasive.  A simple Google image search on the term, "Intelligence Cycle" rapidly gives one a sense of the wide variety of agencies, organizations and businesses that use some variant of the cycle.

The Google Image Search above highlights the first major criticism of the Intelligence Cycle:  Which one is correct?  In fact, an analysis of a variety of Intelligence Cycles from both within and from outside the intelligence community reveals significant differences often within a single organization (See chart below gathered from various official websites in 2011).
While there is some consistency (“collection”, for example, is mentioned in every variant of the cycle), these disparities have significant training and education implications that will likely manifest themselves as different agencies attempt to impose their own understanding of the process during joint operations.  Different agencies teaching fundamentally different versions of the process will likewise seriously impact the systems designed to support analysts and operators within agencies.  This, in turn, will likely make cross-agency integration and information sharing more difficult or even impossible.

The image also highlights the second major problem with the cycle:  Where is the decisionmaker?  None of the versions of the intelligence cycle listed above explicitly include or explain the role of the decisionmaker in the process.  Few, in fact, include a specific feedback or evaluation step.  From the standpoint of a junior professional in a training environment (particularly in a large organization such as the US National Security Intelligence Community where intelligence professionals are often both bureaucratically and geographically distant from the decisionmakers they support), this can create the impression that intelligence is a “self-licking ice-cream cone” – existing primarily for its own pleasure rather than as an important component of a decision support system.

Finally, and most damningly (and as virtually all intelligence professionals know):  “It just doesn’t work that way.”  The US military's Joint Staff Publication 2.0, Joint Intelligence (Page 1-5), describes modern intelligence as the antithesis of the sequential process imagined by the Cycle.  Instead, intelligence is clearly described as fast-paced and interactive, with many activities taking place simultaneously (albeit with different levels of emphasis):

"In many situations, various intelligence operations occur almost simultaneously or may be bypassed altogether. For example, a request for imagery requires planning and direction activities but may not involve new collection, processing, or exploitation. In this case, the imagery request could go directly to a production facility where previously collected and exploited imagery is reviewed to determine if it will satisfy the request. Likewise, during processing and exploitation, relevant information may be disseminated directly to the user without first undergoing detailed all-source analysis and intelligence production. Significant unanalyzed operational information and critical intelligence should be simultaneously available to both the commander (for time-sensitive decision-making) and to the all source intelligence analyst (for the production and dissemination of intelligence assessments and estimates). Additionally, the activities within each type of intelligence operation are conducted continuously and in conjunction with activities in each intelligence operation category. For example, intelligence planning (IP) occurs continuously while intelligence collection and production plans are updated as a result of previous requirements being satisfied and new requirements being identified. New requirements are typically identified through analysis and production and prioritized dynamically during the conduct of operations or through joint operation planning.”

The training and education implications of this kind of disconnect between the real-world of intelligence and the process as taught in the classroom, between practice and theory, are both severe and negative.  

At one end of the spectrum it is as simple as a violation of the long-term military principle of “Train as you will fight”.  Indeed it is only questionable as to which approach will be more counterproductive:  Forcing students of intelligence to learn the Cycle only to realize after graduation and on their own that it is unrealistic or throwing a slide of the Cycle up on the projector only to have an experienced instructor indicate that “This is what you have to learn but this isn’t the way it really works.”  Both scenarios regularly take place within the training circles of the intelligence community.

At the other end of the spectrum, the damage is much more nuanced and systemic.  Specifically, intelligence professionals aren’t just undermining their own training, they are miscommunicating to those outside the community as well.  The effects of this may seem manageable, even trivial, to some but imagine a software engineer trying to design a product to support intelligence operations.  This individual will know nothing but the Cycle, will take this as an accurate description of the process, and design products accordingly.  

In fact, it was the failure of these kinds of software projects to gain traction within the Intelligence Community that led Georgia Tech visual analytics researcher Youn-ah Kang and her advisor, Dr. John Stasko, to undertake an in-depth, longitudinal field study to determine how, exactly, intelligence professionals did what they did.  While all of the results of their study are both interesting and relevant, the key misconception they identified is that “Intelligence analysis is about finding an answer to a problem via a sequential process.”  In turn, the failure to recognize this misconception earlier resulted in a failure of many of the tools they and others had created.  In short, as Kang and Stasko noted, “Many visual analytics tools thus support specific states only (e.g., shoebox and evidence file, evidence marshalling, foraging), and often they do not blend into the entire process of intelligence analysis.

Next:  Part 2 -- The Mercyhurst Model

Myth #3a: I Want To Make A Game That Teaches... (The 5 Myths Of Game-based Learning)

Part 1:  Introduction
Part 2:  Myth #1:  Game-based Learning Is New 
Part 3:  Myth #2:  Games Work Because They Capture Attention 
Part 4:  Myth #3:  I Need A Game That Teaches...

(Needless to say, it has been a strange August.  Thanks for the well wishes and notes of concern.  Hopefully, I am back at it...)

You have a PhD (or you have just been teaching a subject for quite some time) and you like games.  If no one has bothered to make a game that happens to teach anything remotely related to your subject matter, why not just make your own game?  

I have already made the point that good game design is hard (if you want to get an idea of how hard, check out Ian Schreiber's excellent 20 part series:  Game Design Concepts).  Teaching is also hard which makes designing a game that teaches a real...well, you get the point.

None of that is going to deter some of you, though.  If you are still bound and determined to design a game that teaches, whatever you do, don't try to make it a video game.  I have nothing against video games, but they have three strikes against them when it comes to teaching.

Strike One:  Even inexpensive video games cost a ton to make. According to the Casual Games Association, the least expensive games to develop (such as the ones on Facebook) still cost between $50,000 and $400,000.  Large scale games (such as Call of Duty or Mass Effect) can exceed $30 million. No educator has that kind of money laying around for course development. 

Strike Two:  Video games have a very short shelf life.  The technology is advancing so quickly that very few video games hold up well over time.  Most start to look their age within a year or two and many feel old and clunky within 3-4 years.  To get a sense of this drop off, take a look at the steep discounting that typically takes place on video games within the first few years of life:

http://blog.pricecharting.com/2012/03/lifecycle-of-video-games-price-30-years.html

Even if you can design a great game that teaches, if it is a video game, you will have to work pretty hard to keep the game looking fresh and up to date.

Strike Three (A):  A single video game will typically not have enough content to fill a course.  Two of my favorite games of the last year were Portal 2 and Kingdoms of Amalur.  I play both of these games through Steam (for those of you not familiar with Steam, it is like an iTunes for games.  Just like iTunes, it lets you download content directly to your PC and just like iTunes it keeps track of your statistics for you -- how long you play, what you play, how much you like a game, etc).  Steam says I logged 17 hours playing Portal 2 and 101 hours playing Kingdoms of Amalur.  

Both games (which I purchased on sale) provided excellent value for money in my opinion.  Portal 2 is one of the highest ranked games ever and was immensely fun.  Kingdoms of Amalur was designed to be a much lengthier game and was equally fun to play (though many reviewers did not think so...). With an average university course requiring approximately 45 classroom hours and, depending on who you talk to, 2:1 to 4:1 hours outside studying to inside of class, it is arguable (in a rough order of magnitude sort of way) that only video games on the scale of Kingdoms of Amalur could hope to fully replace even a single university course.

Strike 3 (B):  Even if the content is there, relatively few players actually finish video games.  Consider the two games I mentioned above.  Portal 2 is one of the highest rated games of all time.  Players and reviewers loved it.  Heck, I loved it.  I played every level and received every "Achievement" - little electronic tokens of accomplishment that players collect throughout the game.  Steam, of course, keeps track of "Achievements".   Typically, there is at least one achievement associated with completing the main part of the game.  In the case of Portal 2, that achievement is called "Lunacy" (play the game and you will understand why).  I have received this achievement and truly enjoyed the process of getting there.

What is really interesting, though, is that Steam allows me to compare my achievements with the millions of other players who have also played the game.  Only about 56.4% of those who have played the game through Steam have received the Lunacy Achievement.  That is actually a pretty stunning statistic when you consider this is one of the best rated games ever, players presumably volunteered/wanted to play the game and they had to pay between $30 and $60 for the privilege.  It is even harder to imagine a successful class where only 56% of those who start it, finish it.  Kingdoms of Amalur is in an even worse position.  Here only 18.1% of those who started the game played through to the final achievement, "Destiny Defiant". 

**********

OK, so its not as bad as I make it look.  I will readily acknowledge that many of the arguments I make are not as strong as they appear to be.  Indie game designers are bringing extraordinary labors of love to the attention of the masses every day.  The overwhelming success of video games like Minecraft, Braid and Bastion are testaments to what creative people can do on a shoestring.  Likewise, even if one of today's games can't fill a course or routinely get played to completion, you, Kris Wheaton, are the one who said we would have to have multiple games for our courses anyway.  Besides, just because the games aren't here today, does not mean that we shouldn't keep trying.

Exactly.  My point is not to deter game-based learning approaches -- I believe in them wholeheartedly!  My goal is to let teachers know that the process is not as easy and straightforward as it appears.  This is truly a "hard problem" and hard in two fields, game design and education.

I believe the problem will be solved but what are we to do in the meantime?  I recommend two strategies for teachers.  First (and this is the one I use in my Strategic Intelligence class), look for great games that already exist that can teach, reinforce or supplement one or more of your learning objectives.  Second, if you must design your own game, make it a board or card game.  These cost significantly less to design and produce and require much less equipment to play.  They are easier to fit into the constraints associated with a normal 1-2 hour class and, for intelligence professionals, at least, are simply easier to get into the building!

Next:  Myth #4:  The Learning Objectives Come First

Myth #3: I Need A Game That Teaches... (The 5 Myths Of Game-based Learning)

Part 1:  Introduction
Part 2:  Myth #1:  Game-based Learning Is New 
Part 3:  Myth #2:  Games Work Because They Capture Attention

"I'd love to use game-based learning in my classes but I need a game that teaches..." organic chemistry, quantum physics, SIGINT, whatever.

I hear this quite often and it is a legitimate concern.  So many things to teach and so few game designers and publishers willing to take them on. Before I answer why this is, let's assume, for the sake of the argument, that all of the administrative and regulatory hassles involved in designing a game that teaches could be overcome (These are not trivial.  On the contrary, I suspect that these kinds of issues are a big part of the reason that game-based learning strategies have not been more widely tested and applied).  Let's also assume that there is a business model that makes these kinds of games profitable to produce and distribute (another non-trivial assumption).

What's left?  Just building a great game and, at the same time, making sure the course content is integrated into it.   If this sounds really hard, it is.

And its just the beginning.

Because the reality is that you don't need a single great game that teaches these concepts, you really need multiple games that teach.  It turns out that game-based learning is plural.

If, to be successful, game-based learning needs to be, at least to some extent, voluntary (and particularly if you accept the premise, as I do, that the more voluntary the game play is, the more learning will occur), then it makes sense that you will need more than one game covering the same topic to fully engage a diverse classroom full of learners.

To explain this as simply as I can, I often ask people to imagine a typical elementary classroom.  If I only have one great game, let's call it "Barbie Math", I suspect that I may only engage approximately one-half of the students.  I probably need another great game, let's call it "GI Joe Math", to get the other half.  This grade school example is about as simple as I can make the problem but it is potentially much, much worse because of "fun". 

Most game designers I know hate the word "fun".  They hate this word because it is so indistinct and overused that it has virtually lost its meaning.  To say a game is fun (or not fun) is, in short, not very useful criticism.  There are lots of ways games can succeed or fail to produce fun generally and, more relevant to games that teach, specifically for individual students. 

The best place to start to get a sense of this problem from a game design perspective is Raph Koster's A Theory of Fun.  Koster lays out the problem pretty clearly and his book is widely used as a text and cited by professionals. 

To get an even more practical view of the problem, I like Pierre-Alexandrre Garneau's 14 Forms Of Fun article for the online magazine, Gamasutra.  Here Garneau outlines 14 different ways that a game can be fun along with a number of examples of how each element worked in a game (see list to right).  This list has not been scientifically validated and I am sure that, if we got 10 game designers or gamers in a room, there would be lots of disagreements about this list.

I like it, however, because it makes a good case for thinking about fun, and, by extension, about what makes a great game more broadly.  If I think about what I like in a game, I can better see it in this list.  I don't just like the game Portal 2 because it is fun, I like it because it is a witty, immersive game that focuses on intellectual problem solving, advancement and completion (If you are not familiar with the Portal franchise, watch the video below.  It doesn't give much sense of the gameplay but it does give a good sense of the humor in the series).  Moreover, once I know why I like what I like, I can use this system, in much the same way the Music Genome Project worked for music, to help me think about other games I might like to play.

My preferences might not be my students' preferences, however.  It is easy to imagine a student or students that prefer the exact opposite -- I may like cooperative games; they prefer competitive games.  I may like beautiful, discovery games like Myst but they like beautiful, thrill of danger games like Batman:  Arkham City.

We are still just scratching the surface.  What about genres of games?  Some will only like sports games while others will prefer action titles.  What about themes?  Some like high fantasy (like Lord of the Rings Online) while some prefer space based games (Like Eve Online). And what about students who cannot define what they like ("I hate math and statistics and besides I have to spend this entire weekend preparing for my fantasy football draft...")?

These differences have focused on gaming style but even more important are  teaching concerns.  Different students are known to learn differently -- sometimes dramatically.  Text based games, for example, no matter how compelling, may be inaccessible to dyslexic students. 

I know it may sound like I am trying to paint a picture that game-based learning is a herculean, almost impossible task.  That is just because I am a lawyer and creating a "parade of horribles" is what we do.  Many of these distinctions probably matter far less than the discussion so far might lead you to believe.  Some might not matter at all.  Gamers tend to have broader rather than narrower tastes in games.  For every student who only plays sports games, for example, there are likely many more who play both sports games and high fantasy games.  Likewise there are a number of strategies for overcoming almost all learning differences and many could likely be applied to games.

I recognize and accept these objections.  My goal here is simply to paint a more nuanced picture of the challenges teachers and game designers face when they try to take games into the classroom.  There is a naivete in the statement "I need a game that teaches..." that nothing in my experience justifies.

I hope my observations will resonate with the comments made by James Shelton at the Games For Change conference last year (see the video in Part 1 of this series):  In order for game-based learning to go mainstream, it has to scale.  It can't just work with a self-selected population; it has to work across demographic lines and socioeconomic lines and learning differences lines.  This likely means that whatever course or subject you are teaching, you will need multiple games to fully engage your entire class.  A single game is unlikely to do it all.

Next:  Myth 3a:  I Want To Make A Game That Teaches...