Answers For Pennies, Insights For Dollars: Generative AI And The Question Economy

No one seems to know exactly where the boom in Generative AIs (like ChatGPT and Claude) will lead us, but one thing is for certain:  These tools are rapidly driving down the cost of getting a good (or, at least, good enough) answer very quickly.  Moreover, they are likely to continue to do so for quite some time.  

The data is notional
but the trend is unquestionable, I think.

To be honest, this has been a trend since at least the mid-1800's with the widespread establishment of public libraries in the US and UK.  Since then, improvements in cataloging, the professionalization of the workforce, and technology, among other things, worked to drive down the cost of getting a good answer (See chart to the right).

The quest for a less expensive but still good answer accelerated, of course, with the introduction of the World Wide Web in the mid-1990's, driving down the cost of answering even tough questions.  While misinformation, disinformation, and the unspeakable horror that social media has become will continue to lead many people astray, savvy users are better able to find consistently good answers to harder and more obscure questions than ever before.  

If the internet accelerated this historical trend of driving down the cost of getting a good answer, the roll-out of generative AI to the public in late 2022 tied a rocket to its backside and pushed it off a cliff.  Hallucinations and bias to the side, the simple truth is that generative AI is, more often than not, able to give pretty good answers to an awful lot of questions and it is free or cheap to use.  

How good is it?  Check out the chart below (Courtesy Visual Capitalist).  GPT-4, OpenAI's best, publicly available, large language model, blows away most standardized tests.  

It is important to note that this chart was made in April, 2023 and represent results from GPT-4.  OpenAI is working on GPT 5 and five months in this field is like a dozen years in any other (Truly.  I have been watching tech evolve for 50 years.  Nothing in my lifetime has ever improved as quickly as generative AIs have).  Eventually, the forces driving these improvements will reach a point of diminishing returns and growth will slow down and maybe even flatline, but that is not the trajectory today.

All this sort of begs a question, though: If answers are getting better, cheaper, and more widely available at an accelerating rate, what's left?  In other words, if no one needs to pay for my answers anymore, what can I offer?  How can I make a living?  Where is the value-added?  This is precisely the sort of thinking that led Goldman-Sachs to predict the loss of 300 million jobs worldwide due to AI.  

My take on it is a little different.  I think that as the cost of a good answer goes down, the value of a good question goes up.  

In short, the winners in the coming AI wars are going to be the ones who can ask the best questions at the most opportune times.  

There is evidence, in fact, that this is already becoming the case.  Go to Google and look for jobs for "prompt engineers."  This term barely existed a year ago.  Today, it is one of the hottest growing fields in AI.  Prompts are just a fancy name for the questions that we ask of generative AI, and a prompt engineer is someone who knows the right questions to ask to get the best possible answers.  There is even a marketplace for these "good questions" called Promptbase where you can, for aa small fee, buy a customizable prompt from someone who has already done the hard work of optimizing the question for you.

Today, earning the qualifications to become a prompt engineer is a combination of on-the-job training and art.  There are some approaches, some magical combination of words, phrases, and techniques, that can be used to get the damn machines to do what you want.  Beyond that, though, much of what works seems to have been discovered by power users who are just messing around with the various generative AIs available for public use.

None of this is a bad thing, of course.  The list of discoveries that have come about from people just messing around or mashing two things together that have not been messed with/mashed together before is both long and honorable.  At some point, though, we are going to have to do more than that.  At some point, we are going to have to start teaching people how to ask better questions of AI.

The idea that asking the right question is not only smart but essential is a old one:

“A prudent question is one-half of wisdom.” – Francis Bacon

"The art of proposing a question must be held of higher value than solving it.” – Georg Cantor

“If you do not know how to ask the right question, you discover nothing.” – W. Edwards Deming

And we often think that at least one purpose of education, certainly of higher education, is to teach students how to think critically; how, in essence to ask better questions.  

But is that really true?  Virtually our whole education system is structured around evaluating the quality of student answers.  We may think that we educate children and adults to ask probing, insightful questions but we grade, promote, and celebrate students for the number of answers they get right.  

What would a test based not on the quality of the answers given but on the quality of the questions asked even look like?  What criteria would you use to evaluate a question?  How would you create a question rubric?  

Let me give you an example.  Imagine you have told a group of students that they are going to pretend that they are about to go into a job interview.  They know, as with most interviews, that once the interview is over, they will get asked, "Do you have any questions for us?"  You task the students to come up with interesting questions to ask the interviewer.

Here is what you get from the students:

1. What are the biggest challenges that I might face in this position?
2. What are the next steps in the hiring process?
3. What’s different about working here than anywhere else you’ve ever worked?

What do you think?  Which question is the most interesting?  Which question gets the highest grade?  If you are like the vast majority of the people I have asked, you say #3.  But why?  Sure, you can come up with reasons after the fact (humans are good at that), but where is the research that indicates why an interesting question is...well, interesting?  It doesn't exist (to my knowledge anyway).  We are left, like Justice Stewart and the definition of pornography, with "I know it when I see it."

What about "hard" questions?  Or "insightful" questions?  Knowing the criteria for each of these and teaching those criteria such that students can reliably ask better questions under a variety of circumstances seems like the key to getting the most out of AI.  There is very little research, however, on what these criteria are.  There are some hypotheses to be sure, but statistically significant, peer-reviewed research is thin on the ground.

This represents an opportunity, of course, for intellectual overmatch.  If there is very little real research in this space, then any meaningful contribution is likely to move the discipline forward significantly.  If what you ask in the AI-enabled future really is going to be more important than what you know, then such an investment seems not just prudent, but an absolute no-brainer.

What's The Most Difficult Language For English Speakers To Learn?

Once you start looking for them, there are lots of places on the internet that talk about the relative difficulty of learning to speak different languages.  This infographic (courtesy of Voxy.com) sums up what many of these sites are saying:

While I like the infographic, it is impossible to ignore some of the more robust efforts to categorize languages by difficulty. For example, anyone who has been to the Defense Language Institute or the Foreign Service Institute knows that the US government has its own scale that it uses to categorize languages by difficulty.

Even more interesting is the fact that the relative difficulty of the language may or may not correlate with how much extra pay you will receive for speaking a particular language.  For example, compare how difficult it is to learn French with how much extra the Army will pay you each month if you speak French on Page 20 of Army Regulation 11-6...

Top 11 Online Language Learning Resources (Or: How To Make Yourself More Attractive For US Government Intelligence Positions)

In fiscal year 2011, the National Intelligence Program (NIP) made payments to 7,507 government employees within the CIA, DIA, FBI, NGA and NSA for foreign language proficiency (See the Washington Post's breakdown of the so-called "Black Budget"). This is not surprising given the Director of National Intelligence's emphasis on foreign language capability within the US national security intelligence community.

Among the top foreign languages are all the usual suspects - Spanish, Arabic, and Chinese -whereas less common languages populate the "special interest languages" list - Tagalog, Punjabi, Somali and Urdu. 

Figure 1: Snippet of the US Army's Language Payment Incentives

Similarly, the scores on the Department of Defense's DLPTs (Defense Language Proficiency Tests) can earn you pay incentives within government positions and the US Army for each of three language lists - A, B and C - ranked by importance (See Figure 1).

What does this mean for entry-level intelligence analysts? 

It means that being bi- or multi-lingual is practically a pre-requisite for an intelligence analyst position.

Can't afford Rosetta Stone or other expensive programs? No time in a busy class schedule (or even busier work schedule) for a foreign language course?

Don't worry - You don't have to have a BA in Balkan Studies to pass a Serbo-Croatian language exam. There are plenty of (free!) online resources to help you achieve language proficiency levels all on your own. 

Below are the top language learning resources on the web according to the blogosphere:

• DuoLingo. The absolute best thing about DuoLingo is that it is free! The only downside is that it currently only offers instruction in six languages, all of them European (and four of them romance languages). Of the free resources, DuoLingo's classes are arguably the most structured (and they also have an app!)
• LiveMocha. A DuoLingo rival is LiveMocha. Though the premium version is not free, the unpaid offering is significantly more robust than a standard trial version (and with 35 languages, their selection trumps that of DuoLingo). An alternative to paying for the premium upgrade is engaging in the LiveMocha social network and tutoring or offering rudimentary language instruction as a native speaker of your mother tongue in exchange for credit, which can then be applied to your language program of choice.
• OpenCulture, the cultural education blog, features a compendium of resources for language learning in 46 languages. The first suggestion in the majority of the language lists points you towards free iTunes podcasts and downloads for language learning, largely posted by the Radio Lingua Network (entitled Coffee Break) and The Peace Corps. (Also, see OpenCulture's full list of free online courses from top universities).
• Busuu. Busuu is of the same ilk as LiveMocha - It is interactive with a robust social media component, but also provides limited content with expanded content provided for premium members. LiveMocha, however, provides many more languages than Busuu's limited 12 language offering. Despite these limitations, it remains one of the more highly ranked language learning resources on the Internet. 
• The US Department of State is also a good place to find language learning resources, though the Foreign Service Institute is likely for the more linguistically-inclined. The FSI posts all its language materials for 45 different languages online in an open source format including student text to audio recordings. Though the content is not aesthetically appealing and no instruction is provided, the resources are professionally-developed and highly useful, especially for those already fluent in the Linguistics vernacular.
• The Defense Language Institute/Foreign Language Center (DLI/FLC).  DLI is one of the very best language schools in the world.  In addition, it also provides a huge amount of its material for free.  Need a language survival guide in Bulgarian?  Try DLI.  Need a quick series of lesson in Afghan handwriting?  Try DLI. Need to know how to say "Goodbye!" in Balochi?  You guessed it - DLI.  There is a just an immense amount here and in a variety of hard-to-find foreign languages.
• Lang-8. Lang-8 is an interesting resource that provides a network of native speakers to correct foreign language writing submissions. It's pretty simple - You correct the submissions of others in your native language, and submit your writing to be corrected by others. Lang-8 boasts a network of native speakers from 180 different countries.
• RhinoSpike. RhinoSpike is a network similar to that of Lang-8 that facilitates the exchange of audio files alongside the text documents.

So now that you've got a collection of language learning resources on the Internet, there are a handful of what can only be considered recreational practice resources also available:

• iSketch. iSketch is my personal favorite.  I used to play this all the time when I was learning Spanish! It is essentially an online form of Pictionary that can be played in English, French, Spanish and Portuguese with variations in languages such as Turkish and Swedish. It is easy, informal, extremely entertaining and useful for practicing foreign language vocabulary.
• italki. italki is an online community of Skype users aimed at language learning. Pay for individual Skype lessons, or arrange Skype lessons in your target language in exchange for giving lessons in your native language.
• Verbling. Verbling operates around much the same concept as italki, but is a bit more structured. You can sign up for an attend group lessons (the schedule for which is broadcasted on the home page) or you can pay for private tutoring sessions (the going rate is typically $25/hr).

When it comes to language, nothing teaches better than in-person interaction. My recommendation is to pick up the basics of whatever language you seek to learn from list one, then do the majority of your language learning on list two connecting with and talking to native speakers of your target language.

My final caveat to this resource repository is that nothing (I mean, nothing) compares to a passport when it comes to learning a foreign language. Though a round trip ticket to Argentina is a lot more expensive than the one-time purchase of Rosetta Stone Spanish (though, probably not by much), the experience (and the language) will likely stay with you a lot longer. 

Do you have a good resource for online language learning?  Leave it in the comments!

Sources And Methods Games? Yes!

Well, I went and did it!  Started my own company - Sources And Methods Games!

No, I am not quitting my day job.

But ... I do want to explore the intersection between games, intelligence, learning and entrepreneurship (4 big passions of mine).  Doing so from the safety of the University and with someone else's money, however, just didn't seem right somehow.  I suppose I just like testing theories more than talking about them.

What specifically, do I hope to learn?

• What makes a game good?  
• How do games teach?  
• Is it possible to make good games that teach intelligence concepts or methods?  
• Does it make more sense, from the standpoint of teaching intelligence, to focus on tabletop or video games?  
• How is crowdfunding (KickStarter, IndieGoGo, etc.) changing the gaming industry (particularly the tabletop games industry)?  
• What are the intelligence requirements of a globalized, crowdfunded, social media driven business environment?
• Given the research into effectual reasoning and entrepreneurship, what is intelligence's role in a start-up? 
• What are an entrepreneur's Essential Elements of Information (EEIs)?  
• How can they best be met?  

There's more, but this list is a good start.  The bottomline is that this effort -- which is as much a research project as a company -- gives me a chance to explore these questions in a more realistic way.  I recognize that much of the evidence I gather will be anecdotal but I think I will be better able to orient myself on important issues as a result.

This is not the first time I have owned a games company, though.  I ran a small company, Wheaton Publications, back in the early 80's while I was still in law school - ran it right into the ground.  Despite having two pretty good, well-reviewed games, the company was massively under-capitalized and had to be shuttered within 18 months.  

While it was painful at the time, I remember how much I learned from the experience.  While I am no Rockefeller, my financial position has improved a bit since I was a poor law student living in a basement and eating mac and cheese 4 times a week.  I still have good games (including one I which I am about to launch in the next couple of weeks -- if you are interested you can get more information here).  So, who knows?  Maybe I'll make it 24 months this time!

Teaching Strategic Intelligence Through Games (Final Version With Abstract)

Abstract:

Strategic intelligence is considered by intelligence professionals to be the highest form of the analytic art.   There is a tremendous demand for this type of intelligence product and a lack of trained professionals capable of producing it.  Developing effective teaching methods for this challenging subject, therefore, is an area of ongoing concern for the business, law enforcement and national security intelligence communities.

Previous research suggests that a game-based approach to teaching can be successful but no report so far has examined game-based learning in intelligence analysis.  I hypothesized that a game-based approach to teaching strategic intelligence analysis would increase learning and improve performance while also increasing student satisfaction with the course.

This paper reports the initial results and lessons learned from teaching three full courses (2 undergraduate and one graduate) in strategic intelligence using games as a teaching tool.  The paper will begin by examining the unique challenges in teaching about strategy, strategic decisionmaking and the types of intelligence that supports those efforts.  This will be followed by a short discussion concerning games based learning generally before examining in detail the specific approaches used in these three courses.   

This paper will also examine both the learning outcomes and student satisfaction with the courses.  Finally, this paper will discuss appropriate course modifications for undergraduate and graduate students when teaching advanced subjects with games based on the evidence from this study.
 

Part 1 -- Introduction

Part 2 -- What Is Strategy And What Are Strategic Decisions?

Part 3 -- What Is Intelligence And What Is The Role Of Intelligence In The Formulation Of Strategy? 

Part 4 -- Why Games? 

Part 5 -- How, Specifically, Were Games Used in Class? 

Part 6 -- So, How Did It All Work Out? 

Part 7 -- What Did The Students Think About It?

Part 8 -- What Else Did You Learn? 

Teaching Strategic Intelligence Through Games

Part 7 -- What Did The Students Think About It? (Teaching Strategic Intel Through Games)

Part 1 -- Introduction

Part 2 -- What Is Strategy And What Are Strategic Decisions?

Part 3 -- What Is Intelligence And What Is The Role Of Intelligence In The Formulation Of Strategy? 

Part 4 -- Why Games? 

Part 5 -- How, Specifically, Were Games Used in Class? 

Part 6 -- So, How Did It All Work Out?

The SIRs actually measure a number of variables and identifying those that might be most closely associated with the underlying pedagogy of a course are difficult to identify.  Instead, I chose to look at just one of the SIR-generated ratings, the Overall Evaluation of the course.  This is clearly designed to be an overall indicator of effectiveness.  A large change here (in either the positive or the negative direction) would seem to be a clear indication of success or failure from the student's perspective.

Furthermore, my assumption at the beginning of the course was that there would be a large change in one direction or the other.  I assumed that students would either love this approach or hate it and that this would be reflected in the SIR results.  The chart below, which contains the weighted average of the Overall Evaluation score (1-5 with 5 being best) for all classes taught in a particular year, indicates that I was wrong:

Clearly, while students did not love it, they did not hate it either.  The drop in score from recent years could be attributed to a reduction in satisfaction with the class or it could simply be attributed to the fact that the course changed from a fairly well-oiled series of lectures and exercises to something that had the inevitable squeaks and bumps of a new approach.  Feedback from the student surveys given after the course was over, while extremely helpful in providing suggestions for improving the class, gave no real insight into the causes of this modest but obvious drop in student satisfaction.

Comparing this chart with the previous one concerning the quality of the final product yields an even more interesting picture:

This chart seems to be saying that the more a student thinks they are getting out of class (as represented in their Overall Evaluation of the course) the better their final strategic intelligence project is likely to be.  This holds true, it seems, as long as strategic intelligence is taught through more or less traditional methods of lecture, discussion and classroom exercises.  Once the underlying structure of the course is centered on games, however, the students are less satisfied but actually perform better where it matters most – on real-live projects for real-world decisionmakers.

Taken at face value (and ignoring, for the moment, the possibility that this is all a statistical anomaly), a possible explanation is that the students don’t realize what they are getting “for free” from the games-based approach.  Other researchers have noted that information that had to be actively taught, assessed, re-taught and re-assessed in other teaching methods is passively (and painlessly) acquired in a games-based environment. 

I noted this effect myself in my thesis research into modeling and simulating transitions from authoritarian rule.  My goal, in that study, was to develop a predictive model; not to teach students about the target country.  One of my ancillary results, however, was that students routinely claimed that they learned more about the target country in three hours of playing the game than in a semester’s worth of study. 

This “knowledge for free” aspect of the games-based model was nowhere more obvious than in the fairly detailed understanding of the geography of the western part of the Soviet Union acquired by the students in all three classes while playing the boardgame, Defiant Russia.  While this information was available in the form of the game map, learning the geography was not explicitly part of the instructions.  Students rapidly understood, however, that they had to understand the terrain in order to maximize their results within the game.  Furthermore, an understanding of the geography of the western part of the Soviet Union was critical to the formulation of strategic options. 

This raises a broader question regarding games based learning:  If students don't know they are learning, how can they evaluate the learning process?  While I have not had time to dig deeply into the literature regarding implicit learning, I intend to.  Giving students a tangible sense of what they are learning in a game based environment may be one of the biggest challenges to overcome with the approach, at least in higher education.

Next: 

What else did you learn?

Part 5 -- How, Specifically, Were Games Used In Class? (Teaching Strategic Intel Through Games)

Part 1 -- Introduction

Part 2 -- What Is Strategy And What Are Strategic Decisions?

Part 3 -- What Is Intelligence And What Is The Role Of Intelligence In The Formulation Of Strategy? 

Part 4 -- Why Games?

“Indeed, experiments have shown that the more mental work readers have to do to infer a cause from a set of facts, the more memorable the causal inference will be.” – The Trouble With Intuition, Daniel J. Simons and Christopher F. Chabris

While I wanted games to be a central tool within the context of the class, I established some fairly rigorous pre-conditions for myself before deciding which games and how exactly they would be used in class.

First, I did not want the games to detract from the project.  The experience gained from working on a real world project for a real world decisionmaker trumped, in my mind, any value games might bring to learning the essential lessons of strategic intelligence.  In addition, these projects have significant tangible value to the students beyond the knowledge they gain from them.  In many cases over the years, these projects have led to jobs (or at least job offers) either directly or indirectly.

I was worried that the games might detract in two specific ways.  First, the games were to add value, if at all, to only that 20% of the class that was indirectly involved with the project.  I did not want to create an environment where a student felt they had to choose between playing a game and getting work done on the project.

Second, I did not want the course to become about the game or games.  I know from experience that games can be genuinely compelling.  There are many good strategic simulations (Diplomacy, The Total War series) around which a strategic course could easily be built.  In these cases, I saw the games not just competing for time with the project but actually overwhelming the project completely.

Beyond the potential for distraction from the project, the games I did select needed to resonate in a meaningful way with the core concepts of the course.  I either needed a single game through which I could articulate all of the core concepts of the course more or less in the order they needed to be presented (which proved impossible to find) or find a number of different games which could help me accomplish the same goals.  Of course, if I were to use a number of games, the time to learn the new rules of each game would become a factor as well.

If I were to use a number of games, then I also believed I needed to include a variety of game types and genres.  I know from experience that not all game types appeal to all game players.  In fact, I believe that one of the major hurdles to overcome with regard to game-based learning will be the re-packaging of core concepts in any given subject into a variety of game genres such that at least one approach will work for every student. 

Finally, I was very conscious of the cost.  Textbooks are expensive and I believe that, in many cases, they are unreasonably so.  I could not see adding to that burden.

In the end, I settled on using casual online or downloadable games or games, such as World Of Warcraft, that came with free trials (for a complete list of the games and the core concepts, I will include the syllabus to the course in the last post in this series).  The exception to this general rule was the addition of one "old-school", paper-pencil wargame, Defiant Russia (pictured above).   

Students were required to play the game before each class.  In addition, students were required to come to some defensible conclusion about how the game related to the topic of that particular class and to be prepared to discuss it when they came to class. I indicated to the students that the relationships between the games and the topics were rarely obvious and that in some cases there were many possible defensible conclusions.  In many cases, I informed them, there might appear, on the surface, to be no real connection between the game and the topic.  I wanted them to have to think hard about the possible connections, to evaluate them and to come to a conclusion that they were prepared to defend.

Classroom time was devoted in part to examining what the students saw compared with what I saw as the essential connections.  A careful matching of games and topics yielded a fairly high overlap between what the students saw and what I hoped they would see.  In addition, I had the genuine pleasure of having students come up with unique and deep interpretations far beyond my expectations (I will discuss this in greater detail in the subsequent posts).

To give a sense of how this worked in practice, in the class where we discussed strategic intelligence requirements (i.e. the questions that intelligence professionals are asked at the strategic level), students had to play World of Warcraft (WOW) or some other quest-based game.  While there are many defensible answers to the question regarding the connection between WOW and intelligence requirements, I was able to leverage the student's experience with a well-formed "quest"(typical of the high-end MMORPGs) and contrast it with the consequences of poorly formed intelligence requirements.  This, in turn, gave the students a unique perspective on their upcoming meeting with their decisionmaker where they would be receiving the intelligence requirement relevant to their particular project. 

As a result of my experience with the first two classes, I required the third class in which I used this approach to write down their conclusions and post them to a discussion board on Mercyhurst’s Blackboard course management software prior to class.  This writing assignment was modest (100-150 words) but it allowed me to better prepare for the class itself.

While students were required to play the game and to come to a conclusion, I also made a wide variety of supplemental readings available.  These provided "hints' to the connections between the game and the topic that I saw.  I believed that, in some cases (particularly where the material was more in the way of a review), students would be able to come to reasonable conclusions without any additional reading.  I also believed that students that were not overly familiar with the topic under consideration would be more engaged in the reading if they were working to answer a question, even one as difficult as the one I posed.

Next:
So, how did it all work out?

Additional Readings:
School Uses Video Games To Teach Thinking Skills

Teaching Strategic Intelligence Through Games (Original Research)

(Note:  This is another in a series of posts that I refer to as “experimental scholarship” -- or using the medium of the internet and the vehicle of this blog as a way to put my research online for more or less real-time peer review. Earlier examples of this genre include: A Wiki Is Like A Room..., The Revolution Begins On Page 5, What Is Intelligence? and What Do Words Of Estimative Probability Mean?.  

This series of posts is based off a paper I gave at the Game Education Summit last week.)

Strategic intelligence is considered by intelligence professionals to be the highest form of the analytic art.   There is a tremendous need for this type of intelligence product and a lack of trained professionals capable of producing it.  Developing effective teaching methods for this challenging subject, therefore, is an area of ongoing concern for the business, law enforcement and national security intelligence communities.

Previous research (cited in detail in later posts) suggests that a game-based approach to teaching can be successful but no report so far has examined game-based learning in teaching intelligence analysis.  I hypothesized that a game-based approach to teaching strategic intelligence analysis would increase learning and improve performance while also increasing student satisfaction with the course.

This series of posts reports the initial results and lessons learned from teaching three full courses (2 undergraduate and one graduate) in strategic intelligence using games as a teaching tool.  This series of posts will begin by examining the unique challenges in teaching strategy, strategic decisionmaking and the types of intelligence that supports those efforts.  This will be followed by a short discussion concerning games-based learning generally before examining in detail the specific approaches used in these three courses. 

 

This series of posts will also examine both the learning outcomes and student satisfaction with the courses.  Finally, this series of posts will discuss appropriate course modifications for undergraduate and graduate students when teaching advanced subjects with games based on the evidence from this study.

Next:
What is strategy and what are strategic decisions?

What's In A Name? (Game Education Summit -- Part 2)

Don Marinelli kicked off the Game Education Summit with a rousing "hoo-ah" speech for the academics and game design professionals in the room. He wasn't entirely preaching to the choir but it certainly seemed he knew his audience.

Beyond the high rhetoric (such as video games are the "Lucifer of Liesure" to naysayers and luddites...) there were a number of interesting points he brought up. Rather than focus on the entire speech (which, as I mentioned, was designed as much to inspire as to inform), I intend to simply hit the high points from my perspective:

The intersection between games and education. Marinelli was primarily talking about teaching game design at the University level but he spent a good bit of the speech highlighting the more general importance of games in education. He was quite passionate about it and I think generally correct. Games excel at implicit learning -- learning without knowing that you are learning. Taking advantage of that makes sense.

The inevitability of game-based learning. The shift to game based learning in Marinelli's view is largely generational. As the currnet generation (comfortable with chalkboards, lectures and linearity in general) gives away to the next (comfortable with GPS, SMS and high speed internet) there will inevitably be a shift in the way we educate people. Students want to know that they invested their youth wisely. I couldn't help but think at this point of various theories about why animals play. At least one of the theories is that they play to learn.

Intelligence studies and game design share a similar problem -- perception. Marinelli seemed to be saying that games are considered "fun" and game deigners too often feel like they have to apologize for their profession. Likewise, it occurred to me that intelligence is all too often considered "bad" and intelligence professionals similarly feel a need to apologize for their profession (It actually happened to me here. I asked a question and the presenter asked back, "What is it you teach?" My answer: "You don't want to know." And the cock crowed the first time...).