Beyond Gold Farming: The Economics Of Virtual Worlds (The Speaker And The Setting)

Part 1 -- Introduction

The Speaker

As I mentioned in the last post, finding someone who could speak intelligently yet engagingly on the subject of the economics of virtual worlds was surprisingly easy -- for me. I had, serendipitously, met Lenny Raymond while in California last summer and he was the perfect person for the task.

Lenny is the starter, strategist and fixer for V0lv0x Associates, a "band of experienced high tech marketing, engineering, strategy, and management professionals" based, of course, in San Francisco. Lenny's bio is pretty impressive:

• "Mr. Raymond was previously Entrepreneur in Residence at Alsop Louie Partners, an early stage venture capital firm, where he helped start what is going to be the next big high tech toy company. "
  
• "Previously Lenny was co-founder & COO of FingerTwitch, Inc., the leading provider of deployment technology and services for mobile game publishers & developers; FingerTwitch was acquired by Hands-On Mobile, Inc., the world’s largest mobile application publisher, one year after its inception. "
  
• "Before FingerTwitch, Mr. Raymond was founding CEO of Xigo, Inc., a venture-funded provider of real-time unstructured data analytical software and services for the brokerage industry. "
  
• "Earlier in his career, Mr. Raymond was general manager of Wizards of the Coast (Hasbro)’s online and electronic media business unit. "
  
• "Lenny has also worked at Mercer Management Consulting, in Apple’s Advanced Product Group, and in global corporate finance at Citibank."
• "Mr. Raymond’s education includes a BA in Economics from the University of Pennsylvania and an M.B.A. from The Wharton School."

More importantly, though, was his combination of a deep understanding of virtual worlds and their economies and his ability to translate that knowledge into something comprehensible for an audience of relative neophytes. Oh, and the fact that he has enormous "gamer cred" (from being a long-time player of the games themselves) didn't hurt.

The Setting

Getting an excellent speaker on an interesting topic wasn't difficult enough, though. We needed to make it more complicated. Our budget ($0) did not permit us to fly Lenny from SF to Erie so we decided to hold the lecture about virtual worlds IN a virtual world.

This is not the first time this has happened, obviously (there are lectures and presentations in Second Life all the time, for example), but it was the first time we had done it and none of us were exactly sure how to proceed.

First, we decided to hold the lecture in the online game, World Of Warcraft (WoW). We were all more familiar with WoW than with Second Life and we all knew that the economy is a vibrant and integral part of WoW (potentially, we thought, providing Lenny with the opportunity to make some of his points directly. Boy, were we right about that...).

WoW also has very good in-game communications. There is a text chat function with multiple channels (including private and semi-private channels) and the possibility of in-game voice chat as well. All of us (including Lenny) had experience with these communications modes but none of us had ever tried to pull off a lecture in WoW before. A couple of students volunteered to figure out the details and, before I knew it, we had formed a "Guild" and had developed a comms plan (Thanks Henry, Devin, Kirk and all the other Initiative members who helped make it happen!).

• Note: For the national security types who are concerned about the implications of these robust communications capabilities, it should be comforting to know that Blizzard, who makes WoW, has the ability to monitor all of these methods of communication.

The concept of a Guild is a partiuclarly interesting one within WoW. Nine players have to "sign a charter" in the game to form a guild. There are many reasons to form a guild and they often form around groups of players who have a common interest outside the game (the MCIIS Guild -- see picture below of a "meeting" -- , for example, is an extremely small guild currently comprised mostly of intel studies students or their friends and family members) though that is certainly not always the case.


Some Guilds have 100's of members or have been in existence since the game began. In certain circles, getting into the "right" guild is seen as a good career move and WoW has been called "the new golf" by some (though others would clearly dispute this).

For our purposes, forming a Guild gave us two advantages. The first was Guild Chat (a private means of communication among Guild members) and the second was the potential to access the Guild "Bank", clearly part of the economic infrastructure of the game.

We also decided that we wanted non-players to be able to view the lecture as well. To do this, we loaded up the game on one of the computers in one of our classrooms and broadcast it on the big screen through a projector. In the end, about half of the 25 person audience was in the game and the other half was in the classroom watching the action and listening through speakers.

We had the inevitable technical difficulties -- Blizzard, which makes WoW, had released a massive update/patch to the system a few days before the event -- but things worked out well. Lenny gave an excellent presentation (more on that in the New Year...) and we all learned something from both constructing and particpating in the experience.

As a side note (for the educators in the audience), I was and continue to be very impressed with the learning potential of virtual worlds and WoW in particular. It is a fully realized virtual world, massive in its geographic size and scope. It is a stable, almost always available platform with a wide variety of useful communication/coordination/collaboration tools. You cannot do anything you want, but the variety of things you CAN do gives the world a number of possibilities for interactive learning.

Yes, yes, I know it is a "game" and its intent is to provide entertainment (and make money for Blizzard), not to serve as a platform for a learning experience. It has the tools necessary, however, for a creative educator to take advantage of the high interest, immersive environment the game provides. Whether we build these experiences in games like WoW or simply use the lessons learned from these type of games in the next generation of curricula, my initial impression is that there is something here worth exploring further.

After New Years: Lecture Notes!

Beyond Gold Farming: The Economics Of Virtual Worlds (Introduction)

(Several months (!) ago, we had the good fortune here at Mercyhurst to have a truly gifted speaker give a great lecture about the economies and economics of virtual worlds in a genuinely unique manner. I am just now getting around to writing about it...)

First, A Little History

Beginning with last year's student project into possible Islamic extremist use of Second Life and YouTube, I have begun to become increasingly interested in how virtual worlds and augmented reality will impact our notions of national security -- what are the threats and opportunities (if any) presented by these new technologies?

Over the summer I had the occasion to further explore this topic with a large, diverse and fascinating group of people from a variety of different disciplines. I came away from the experience convinced that this was not only a topic worth exploring but also one that would likely facilitate learning. Many of my students, after all, were already into this stuff -- a trend that I think is highly likely to continue. Figuring out how best to teach (and what can one usefully learn) in these high interest environments seemed to make some sense.

With this in mind, we formed a small, all volunteer research group (AKA "a bunch of people talking about the same thing (more or less)") to explore the possibilities of these new technologies. We gave it a grand name (the Mercyhurst Virtual Worlds Initiative) and set about trying to figure out what that might mean.

We decided to focus on two virtual environments to begin with -- one open and one closed. For the open environment we chose Second Life. It is arguably the best-known virtual world (though far from the largest) and its low cost of entry (free) and well-developed environment made it particularly attractive.

For the closed environment we chose the massively multiplayer online role-playing game, World Of Warcraft or "WoW" as it is often called. We chose WoW becaue it IS the largest such game (with 11.5 million players and growing) and because a number of the Intiative's members were already very familiar with the game and its mechanics.

• The students also established a wiki for the Intiative. It serves mostly as a way to organize events, collectively compile notes and other info of interest to the group. It is not designed to be a Wikipedia-like product or the basis for a finished analytic report. It is really just a "place for our stuff" (You can take a look at it if you like but the organization is fairly idiosyncratic and its utility to anyone outside the Initiative is questionable...).

Picking A Topic

Economics is one of the places where the virtual is already blurring with the real. Real currencies today have a strong virtual element to them. Most transactions in dollars or euros or yen are little more, in reality, than the exchange of a torrent of 1's and 0's across the internet.

Likewise, many virtual currencies, such as Linden Dollars or WoW Gold, are actually or effectively exchangable for harder currencies. The size of these virtual economies is still fairly modest but the possibilities for large-scale money laundering are there and these virtual currencies are attracting the attention of regulators in places like China and Iceland.

Despite these pressing and very real issues involving virtual economies, one aspect -- so-called "gold farming", or the practice of using inexpensive labor in under-developed countries to harvest virtual currencies for sale to players in wealthier countries for real money -- has, to a certain extent, captured the public imagination and dominated the discussion until fairly recently (See the video below for an introduction to gold farming).

Fortunately, one of the people I happened to meet over the summer was able to address, in a sophisticated yet accessible manner, not only the issue of gold farming but also the broader issues tied up with virtual economies. It was a simple thing to determine that economics should be the Intiative's first topic and that getting this guy as a speaker should be our first project. Simple in concept, that is...

Tomorrow: The Speaker And The Setting

Top Eleven "Top Ten" List (Extended Version)

Embed-O-Matic

• m=

Embed-O-Matic

• m=

This time of year it seems that everyone is doing a Top Ten (or "Top Eight" or "Top Five" or "Top Something") list. Rather than report every single one of them separately, I decided to list them all and let you sort out the ones of interest...

• Top 10 Forecasts for 2009 and Beyond
• Top 10 Alternative Search Engines + 8 Other Top Ten Web Related Lists
• Top 10 Worst Predictions of 2008 (Thanks, Rex!)
• Top Online Security Threats for 2009
• Top 10 Science Breakthroughs
• Top 10 Treehouses
• Top 10 Technology Breakthroughs
• Top Weird Gadget Lists of 2008
• Top Influencers on Social Media in 2009
• 10+ Web Tools to Save Your Butt in School
• 5 Best Data Visualization Projects of the Year

And here is the extended version...

• 10 USB Gadgets to Keep You Warm
• 5 Methods to Bypass Blocked Sites
• 8 Machines That Can Assemble Themselves
• 10 Ways to Enhance Brain Fitness Online
• Most Trusted Companies for Privacy
• Top 3 Serious Games
• Top 5 Indie Games (last week!)
• Top 10 Ways to Repurpose Your Old iPod
• Megatrends of Game Design
• Identity Events of the Year
• 3 Websites to Learn Cool Tricks and Impress Friends
• Top 21 USB Flash Drives
• 55 Great Websites to Download Free Sound Effects
• The Essential Guide to the Internet and Software for First-time Computer Users

Finally, if you were wondering why this list went to 11 (and beyond) let's just say that it is an homage to the mockumentary, This Is Spinal Tap (See relevant clip below), which some of us may remember as one of the best Christmas gifts ever.

Spinal tap amp

Day After Christmas Videos

For readers of SAM outside of North America (South America, most of Europe, Australia, New Zealand, etc.) this may seem like just another Friday but for us, it is the day after Christmas and no one is doing anything but recuperating. Even if they are at work, believe me...they are still recuperating.

With this in mind, I offer a bit of light entertainment. Happy Holidays to everyone!

From Improbable Research, "Violent Chess":



From MakeUseOf, "Beaker's Ode To Joy":



and "Chad Vader: Day Shift Manager":



and finally, from Gizmodo, "Nerf Vulcan":

Top Ten Uncracked Codes (Funnbee.info via INTELST)

INTELST, the US Army's listserve for intelligence professionals, pointed me this morning towards a Funnbee.info post that catalogs the 10 most famous uncracked codes (in case you are bored over the holidays...).

The list includes:

• Kryptos
• Linear A
• The Phaistos Disk
• The Shugborough Hall Inscription
• The Chinese Gold Bar Cipher
• The Beale Ciphers
• The Voynich Manuscript
• The Dorabella Cipher
• The Chaocipher
• The D'agapeyeff Cipher

For more info on any of these, click on the link above!

Where The Oil Comes From (RMI via Google Maps Mania)

Google Maps Mania recently featured this interactive map/data mash-up from the Rocky Mountain Institute that shows where US oil imports have come from over the last 30+ years. It is an interesting way to look at this data. Sorry, no embed feature (again!) so you will have to click on this link to see the interactive version. The YouTube video below gives some idea of what the site delivers.



Related Posts:
Good Resource On International Energy

Top 5 Intelligence Analysis Methods: Analysis Of Competing Hypotheses (#1)

(Note: I apologize for how long it has taken me to get here. Conferences, classes and life in general conspired to get in the way this week. For the patient, here is the last in this series of posts...)

Part 1: Introduction
Part 2: What Makes A Good Method?
Part 3: Bayesian Analysis (#5)
Part 4: Intelligence Preparation Of The Battlefield/Environment (#4)
Part 5: Social Network Analysis (#3)
Part 6: Multi-Criteria Decision Making Matrices/Multi-Criteria Intelligence Matrices (#2)

Analysis Of Competing Hypotheses (ACH) is probably the best-known intelligence analysis method today. Invented by Richards Heuer over 30 years ago and made famous in his intelligence classic, the Psychology of Intelligence Analysis, ACH is widely taught and conceptually easy even for entry-level analysts.

In addition, it was specifically designed to work in all kinds of situations with any kind and quality of data. What is less clear is that the method produces unequivocally better estimative results. While the method is rooted fundamentally in the scientific method, studies testing the value of the method as a way to improve forecasting have been few and the results have been mixed (In no study have the results been worse than without the method but some studies have shown that the method only helps certain subsets of analysts. For a good list of these studies see the Notes List at the end of the ACH article on Wikipedia).

I am not sure why this is so. My own impression is that a well done ACH provides a better estimate in less time and with more nuance than virtually any other method available.

We teach ACH here in our freshman classes. I see many, many students struggle not with the basic concepts of ACH but with the details. I see countless examples each year of student projects where they have improperly executed the method (in much the same way a student gets their first attempts at a calculus or chemistry problem incorrect).

In most cases, it is fairly easy to correct the mistakes and the students rarely have a problem seeing what they did wrong or in making the appropriate adjustments. It is less clear to me that, at this early stage in their education, they are able to transfer this knowledge from one type of problem to another, however. We try to reinforce all our methods in upper level classes but the opportunities for reinforcement in the real world are slim (we rarely find, for example, that students are required to use structured methods in their internships).

My own instincts tell me that ACH (and many of the experiments involving it -- including our own) is a powerful method but won't get a fair test until such a test is done with analysts who have worked with the method on multiple problems and in multiple circumstances. To be honest, I suspect that this is true with all of the methods I have discussed in this series. Deliberate practice seems to be a key component of expertise in multiple other fields and I imagine this is true when it comes to intelligence analysis methods as well.

Improving the quality of the final estimate is only one (albeit an important) way that a method should contribute to a quality intelligence product, however. ACH brings much more to the table in my estimation and it does this immediately, in even the earliest projects.

ACH can help the analyst at every stage of the problem, including modelling, collection and collection planning, and preparing a document for dissemination. It is a wholly transparent method and can very easily be used collaboratively. Its transparency is also crucial in helping instructors or managers identify problems in the analysis of the data. The transparency is also of enormous benefit in understanding and improving the analytic process after the fact as well. It integrates extremely well with various data resources and is very suitable for automation. We find that it is actually faster to use, particularly in a group setting, than most other methods (including intuitive analysis).

The way ahead is a little different here than with the other methods. We think we have a pretty good handle on how to teach ACH. The key, in my estimation, is to create opportunties to reinforce that teaching in and outside the confines of the classroom.

Jigsaw Demo: A Powerful, New Visual Analytics Tool (YouTube)

Yesterday, I posted my initial reaction to an extraordinary piece of software called Jigsaw I saw demo-ed during my visit to GA Tech for the Workshop on Visual Analytics Education sponsored by NVAC. Today, John Stasko and his team uploaded a video of the software in action that is a must-see.

I understand the video is a little old so some of the functions I saw are not in this video. The good thing about John and his band of software wizards, though, is that they are constantly improving the product.

Amazing Data Visualization Tool From The Geniuses At GA Tech (Jigsaw)

(Note: I am at a visual analysis conference sponsored by the National Visualization And Analytics Center so the last of the Top 5 Methods list will have to wait a day or two. In the meantime, I have found something else that is pretty cool...)

John Stasko and the computer scientists at the Information Interfaces Lab at Georgia Tech may not have found the Holy Grail of visual analysis but they have come pretty darn close with their Jigsaw product.

This extraordinary visualization tool automatically extracts entities (names, places, dates, etc.) from plain text documents. Then, it automatically creates a visualization of the relationships between those entities and the documents containing them. The screenshots below do not do it justice (I hope to have a video of the product in action within a couple of days, though).


The program is fully customizable so you can add or delete data, designate entities or create relationships to modify what the automatic entity extractors come up with.

The real power of the tool comes into play after the data is in the program. You can play with it in a variety of powerful and interesting ways all accessible through a drop dead easy user interface.

The software is continuously improving. On the horizon is the ability to use web input and there is a long analyst generated to-do list that the grad students at GA Tech are cranking through one at a time.

The software runs on a desktop and was developed with a DHS grant so government and academics should reach out to John for a test copy. GA Tech is also home of the Visual Analytics Digital Library and well worth checking out.

Top 5 Intelligence Analysis Methods: Multi-Criteria Decision Making Matrices/Multi-Criteria Intelligence Matrices (#2)

Part 1 -- Introduction
Part 2 -- What Makes A Good Method?
Part 3 -- Bayesian Analysis (#5)
Part 4 -- Intelligence Preparaton Of The Battlefield/Environment (#4)
Part 5 -- Social Network Analysis (#3)

Multi Criteria Decision Making (MCDM) is a well-known and widely studied decision support method within the business and military communities. Some of the most popular variants of this method include the analytic heirarchy process, multi-attribute utility analysis and, in the US Army, at least, the staff study (see Annex D). There is even an International Society on Multiple Criteria Decision Making.

At its most basic level, MCDM is used to evaluate a variety of courses of action against a set of established criteria (see the image below for what a simple matrix might look like). One can imagine, for example, considering three different types of car and evaluating them for criteria such as speed, cost and fuel efficiency. MCDM would suggest that the car that has the highest total rating across those three categories would be the best car to buy. In fact, Consumer Reports uses exactly this type of method in its famous "circle charts" of everything from cars to hair care products.


MCDM is flexible and works with a wide variety of data but there are numerous devils in the details of its implementation. The simple example above gets increasingly complicated when we start to examine the many other criteria that someone might use to evaluate a car. Likewise, there are problems with rating each of these criteria (Is a car that gets 27 miles to the gallon really worse than a car that gets 27.1 miles to the gallon?). Even worse is when the analyst starts to think about abstract evaluation criteria such as which of the three cars is "coolest"? You start thinking like this and you begin to understand why they have an international society dedicated to this method...

MCDM is, at its heart, an operational methodology, not an intelligence method, however. That said, we have had very good luck “translating” it into an intelligence analysis method (i.e. MCIM) in our strategic intelligence projects. These projects have covered the gamut from large-scale national security studies to small-scale business studies. The matrices can be simple (I actually used such a matrix to evaluate the 5 methods mentioned in this series) or enormously complex (The MCIM matrix on the likely impact of chronic and infectious disease on US national security interests clocks in at 15 feet long when printed).

The key difference between the operational variant and the intelligence variant is perspective. In the operatonal variant, the analyst is trying to figure out his or her organization's best course of action. In the intelligence variant, the analyst puts him or herself in the shoes of the adversary and attempts to envision how the other side might see both the courses of action available and the criteria with which the adversary will evaluate them. The intelligence variant has not, to the best of my knowledge, been validated but we have a grad student working on it.

The research agenda for this method (as with many of the other methods discussed so far) is straightforward. First, it has to be validated as an intelligence specific methodology. The anecdotal evidence and the evidence in the operational literature is good but further testing needs to be done. Second, analysts need to figure out which variants of MCIM work best in which types of intelligence situations. Finally, we need to get the method out of the school house and into the field.

Next Week: #1!

Top 5 Intelligence Analysis Methods: Social Network Analysis (#3)

Part 1 -- Introduction
Part 2 -- What Makes A Good Method?
Part 3 -- Bayesian Analysis (#5)
Part 4 -- Intelligence Preparation Of The Battlefield/Environment (#4)

Social Network Analysis (SNA) is fundamentally about entities and the relationships between them. As a result, this method has a number of variations within the intelligence community ranging from techniques such as association matrices through link analysis charts right up to the validated mathematical models. It is most commonly used as a way to picture a network, however, and is rarely used in the more formal way envisioned by the sociologists who created the method. In other words, while SNA is a very powerful method, intelligence professionals rarely take advantage of its full potential.

Because it is primarily a visual method, most analysts (and the decisionmakers they support) immediately grasp the value of the method. Likewise, some variation of SNA will likely work with any data set where the entities and the relationships among those entities are important (in other words, almost every problem). Parsing all the relevant attributes can be difficult, however, and there are few automated solutions that work well with unstructured data sets. Likewise, at the higher levels of analysis, where the analyst is trying to do more than merely visualize a network, a good bit of special knowledge is required to understand the results.


Talking about SNA doesn't make a lot of sense though. It is much easier to grasp its value as an intel method by looking at some examples. I2's Analyst Notebook is widely available and examining their case studies is helpful in seeing what that tool can do. Another widely used tool, particularly for more formal analyses, is UCINET. For example, some of our students used this tool last year to examine the "social network" of government and non-governmental organizations engaged in security sector reform in sub-Saharan Africa (the image above comes from their study). Recently, my students and I have been playing around with a new tool from Carnegie Mellon University called ORA. It is very easy to use and very powerful.

Of the 5 methods I intend to discuss, SNA is the one with the widest visibility in all three major intelligence communities (national security, business and law enforcement). As such, there is already a good bit of research activity into how to better use this method in the intelligence analysis process. The big challenge, as I see it, is to design educational programs and tools that help analysts move away from the "pretty picture" possibilities presented by this method and toward the more rigorous results generated by the more formal application of the method.

Tomorrow: #2...

And Now For Something Completely Different... (YouTube)

Embed-O-Matic

• 47LCLoidJh4&hl=

Embed-O-Matic

• 47LCLoidJh4&hl=

I was pretty seriously under the weather yesterday so I decided to take a break from the top five list today to show this video (Thanks, Andrew!). I recommend you watch the video before reading the rest of the post...



This video draws on an interesting line of research at the Visual Cognition Lab at the University of Illinois, specifically on their 2004 Ig Nobel prize winning paper, "Gorillas in Our Midst", published in Perception. The video used in this study (and other videos from the lab) are available for purchase. We use the "basketball video" as an introduction to a series of classes on cognitive biases.

Top 5 Intelligence Analysis Methods: Intelligence Preparation Of The Battlefield/Environment (#4)

Part 1 -- Introduction
Part 2 -- What Makes A Good Method?
Part 3 -- Bayesian Analysis (#5)

Intelligence Preparation of the Battlefield (IPB) is a time and battle tested method used by military intelligence professionals. Since its development over 30 years ago by the US Army, it has evolved into an increasingly useful and sophisticated analytic method. (Note: If you are interested in the Army's Field Manual on IPB, it is available through the FAS and many other places. We teach a very simplified version of IPB to our freshmen and you can download examples of their work on Algeria and Ethiopia (They are .kmz files, so you will need Google Earth to view the files).)

IPB is noteworthy for its flexibility. Its success in the field led to a variety of modifications and extensions of its basic concepts. The Air Force, for example, expanded IPB to what it has called Intelligence Preparation of the Battlespace and I know that, several years ago, the National Drug Intelligence Center developed a similar method for use in counter-narcotic operations. Today, the broadest variation on the IPB theme seems to be what NGA calls Intelligence Preparation of the Environment (IPE or sometimes Intelligence Preparation of the Operational Environment -- IPOE).

IPB/IPE/IPOE reminds me a bit of Sherlock Holmes in the The Sign of the Four: "Eliminate all other factors and the one which remains must be the truth." The fundamental concept of IPB – using overlapping templates to define a physical or conceptual space into go, slow-go and no-go areas – can clearly be applied to a variety of situations. It is obvious that this method is particularly useful in situations where geography is important. Mountains, rivers, etc. restrict movement options while roads and the like facilitiate movement. Overlaying weather effects and opposing forces' doctrine on top of this geography (combined with several other factors) can give a commander a good idea of what is possible, impossible and likely.

Simplify the concept even more (and remove it from its traditional military environment) and it begins to look like a Venn diagram with intersecting circles useful in any situation that can be thought of, either concretely or abstractly, as a landscape. Imagine, for example, a business competitor in which we are interested. We suspect that it is preparing to launch a new product. How would we translate IPB into this environment? Perhaps we could see the various product lines where our competitor operates as "avenues of approach". The competitor's capabilities could be defined by its patent portfolio and financial situation. The competitor's "doctrine" could be extrapolated, perhaps, from its historical approach to new product launches. The validity of this and other similar approaches in other fields is, however, largely untested.

By defining, in advance, the relevant ways to group the data available for analysis, IPB is able to effectively deal with large quantities of both strutured and unstructured data. While these groupings are typically quite general, they are finite and it is possible for relevant data to fall through the cracks between the groups. Likewise, as the relevant groupings of data begin to proliferate, the method quickly moves from one which is simple in concept to one which is complex in applicaton (The US Army's IPB manual is 270+ pages...).

For me, the research challenges here are straightforward. The military has a clear lock on developing this method within its environment; there is little value added for academia here. Beyond the military confines, however, the research possibilities are wide open. Does this method or some variation of it work in business? How best to define it in law enforcement situations? Could it work against gangs? In hostage situations? Crime mappers, in particular, might be able to utilize some of these concepts to further refine their art.

Tomorrow: Method #3...

Top 5 Intelligence Analysis Methods: Bayesian Analysis (# 5)

Part 1 -- Introduction
Part 2 -- What Makes A Good Method

(Note: Bayesian statistical analysis is virtually unknown to most intelligence analysts. This is unfortunate but true. At its core, Bayes is simply a way to rationally update previous beliefs when new information becomes available. That sounds like what intelligence analysts do all the time but it has that word "statistics" associated with it, so, even analysts who have heard of Bayes often decide to give it a miss. If you are interested in finding out more about Bayes, you can always check out Wikipedia but I find even that article a bit dense. I prefer Bayes For Beginners -- which is what I am.)

Bayes is the “Gold Standard” for analytic conclusions under conditions of uncertainty and probably ought to be closer to -- if not at the -- top of this list. It provides a rigorous, logical and thoroughly validated process for generating a very specific estimative judgment. It is also enormously flexible and can, theoretically, be applied to virtually any type of problem.

Theoretically. Ahhh... There, of course, is the rub. The problem with Bayes lies in its perceived complexity and, to a lesser degree, the difficulty in using Bayes with large sets of unstructured, dynamic data.

• Bayes, for many people, is difficult to learn. While the equation is relatively simple, its results are often counterintuitive. This is true, unfortunately, for both the analysts and the decisionmakers that intelligence analysts support. It doesn't really matter how good the intelligence analyst is at using Bayes if the decisionmaker will not trust the results at the end of the process because they come across as a lot of statistical hocus-pocus or, even worse, simply "seem" wrong.

• While Sedlmeier and Gigarenzer have had some luck teaching Bayes using so-called natural frequencies (and we, at Mercyhurst, have had some luck replicating his experiments using intelligence analysts instead of doctors), the seeming complexity of Bayes is one of the major hurdles to overcome in using this method effectively.

• In addition to the complexities of Bayes, it appears that this method, which works well with small, well-defined sets of any kind of data, does not handle large volumes of dynamic, unstructured data very well.
  
  
  • Bayes seems to me to work best as an intelligence analysis method when an analyst is confronted with a situation where a new piece of information appears to significantly alter the perceived probabilities of an event occurring. For example, an analyst thinks that the odds of a war breaking out between two rival countries are quite low. Suddenly, a piece of information comes in that suggests that, contrary to the analyst’s belief, the countries are, in fact, at the brink of war. A Bayesian mindset helps to ratchet back those fears (which are actually best described by the recency and vividness cognitive biases).
  
  
• The real world doesn't present its data in ones, however, and not all data should be weighted the same. When analysts try to go beyond having a "Bayesian mindset" and apply Bayes literally to real world problems (as we have on several occasions), they run into problems. Think about the recent terrorist attacks in Mumbai. Arguably, the odds of war between India and Pakistan were quite low before the attacks. As each new piece of data rolled in, how did it change the odds? More importantly, how much weight should the analyst give that piece of data (particularly given that the analyst does not know how much data, ultimately, will come in before the event is "over")? Bayes is easier to apply if we treat "Mumbai Attack" as a single data point but does that make sense given that new data on the attack continues to come in even now?

• Bayes, in essence, is digital but life is analog. Figuring out how to "bin" or group the data rationally with respect to real-world intelligence problems is one of the biggest hurdles to overcome, in my estimation, with using Bayes routinely in intelligence analysis
  

Bayesian statistical analysis has enormous potential for the intelligence community. 20 years from now we will all likely have Bayesian widgets on our desktops that help us figure out the odds -- the real odds, not some subjective gobbledy-gook -- of specific outcomes in complex problems (In much the same way that Bayes powers some of the most effective and powerful spam filters today). The research agenda to get closer to this "Golden Age Of Bayesian Reasoning" is straightforward (but difficult):

• Figure out how to effectively and efficiently teach the basics of Bayes to a non-technical audience.
• Actually teach those basics to both analysts and decisionmakers so that both will have an appropriate "comfort level" with the fundamental concepts.
• Develop Bayesian-based tools (that are reasonably simple to use and in which analysts can have confidence) that deal with large amounts of unstructured information in a dynamic environment.

Anyone got any extra grant money lying around?


Tomorrow -- Method #4...

Surreal Saturday: Auditorium (PlayAuditorium.com)

If you are looking for something different and profoundly odd and wonderful all at the same time, check out the new flash-based game Auditorium (Many thanks to Raph Koster's Website for blogging about Auditorium and to Lenny Raymond's Blog for pointing me to Raph's site...). Don't worry if you don't get it at first, you will (That moment of discovery is sort of what the game is about anyway). If you really get lost, scroll down the home page for a guide but I recommend you try it without the guide first.

Top 5 Intelligence Analysis Methods: What Makes A Good Method? (List, Part 2)

Part 1 -- Introduction

There are a number of good analytic methods available. If you are ever at a loss for a method (or just want to see a really good taxonomy of methods) check out the Principles of Forecasting site. Specifically, look at the methodology tree and the selection tree (You can see a screen shot below but you really owe it to yourself to look at the interactive site or, at least, download the PDF).

While I strongly support the International Institute of Forecasters and all of their good work, I have rarely had the kind of data in the real-world intelligence problems on which I have worked that would allow me to be comfortable using many of the methods that they have listed. I'll be honest; these guys have spent a lifetime thinking about forecasting and deriving a taxonomy of methods so I am probably the one who is wrong but the methods I find most useful -- over and over again -- are simply not on their list.

What makes for a useful intelligence analysis method? Based primarily on my experience with real-world intelligence problems and with teaching entry-level analysts a wide variety of methods, I think there are four primary factors: Validity, simplicity, flexibility and the method's ability to work with unstructured data.

• Validity. There needs to be at least some evidence to suggest that the method actually improves the intelligence estimate and there should not be strong evidence suggesting that the method does not work. Many of today's "generally accepted" methods and multipliers fail to meet this test. Developing and analyzing scenarios and devil's advocacy are two examples. Tetlock took a hard look at one kind of common scenario development method and found it wanting yet this research is almost universally unknown to intelligence analysts. As Steve Rieber has pointed out, there is no real research to support the use of Devil's Advocacy despite its support by the Senate Select Committee on Intelligence. It is surprising to find that many of today's commonly used intelligence analysis "methods" are, in reality, little more than tribal lore passed down from one generation to another.
  
• Simplicity. All successful intelligence analysts are smart but even when they have PHDs, you find a reluctance to use complex and, more importantly, time consuming methods. Due to the error inherent in the data available to most intelligence professionals, the benefit derived from using these methods simply doesn't appear to most analysts to outweigh their costs. To be "simple" by my definition, a method should be able to be taught in a reasonable amount of time and the analyst should be able to see themselves using the method in real-world situations. Analytic methods that actually help communicate the analysis to the decisionmaker or that help evaluate the intelligence process after the fact get extra credit.
  
• Flexibility. Analysts consistently find themsleves in a wide variety of situations. Sometimes these sistuations are tactical and sometimes they are strategic; sometimes the analyst is a subject matter expert and sometimes they are not. In this post Cold War world, it seems to me that national security analysts are getting dragged from one portfolio to another at an accelerating pace. I remember, for example, when all sorts of Russian analysts were re-branded as newly minted Balkans analysts in the 90's and I suspect that several months ago a number of African or Korean analysts suddenly found themselves on a Georgia-Russia Analytic Team trying to figure out what was likely to happen next in South Ossetia. A really good method should work in all these types of situations and across all the disciplines of intelligence as well.
  
• Works With Unstructured Data. One of the things that distinguishes, in my mind, intelligence work from other analytic work is that intelligence deals primarily in unstructured data. Intelligence data does not come in neat columns and rows on Excel spreadsheets. It comes in a variety of forms and is often wrong, incomplete or even deliberately deceptive. An intelligence method that fails to acknowledge this, that needs "clean" data to get good results, is a less useful method in my mind.

I am sure that there are other factors that one should consider when selecting an analytic method (and, please, put yours in the comments!) but these are the ones that seem most important to me.

Monday: Method #5...

Top 5 Intelligence Analysis Methods (List)

(Note: I was recently asked to name and describe my top 5 intelligence analysis methods. As I began to think about it, what seemed like a fairly straightforward question morphed into what I could only think of a series of blog posts. So, here they are...)

Considerable emphasis has been put on improving the methods of intelligence analysis over the last six years. The 9/11 Report alluded to the need for it, the WMD Commission addressed it more directly and the DNI recently highlighted the continued requirement for advanced analytic techniques in its Vision: 2015 document.

Still, the intuitive method (also known as "read a bunch of stuff, think about it for a bit and then write something") remains the most popular method for producing intelligence analysis despite this method's well known tendency to permit a wide range of cognitive and systemic biases to corrupt the analytic product (see Heuer and Tetlock for excellent overviews of these problems).

Beyond the intuitive method (and the interesting defenses of it offered by books such as Blink and Gut Feelings), what, then, are the best methods for conducting intelligence analysis? Given the wide range of intelligence analysis problems (tactical, operational, strategic) and the large number of disciplines using intelligence analysis to support decisionmaking (national security, law enforcement and business) is there any chance that I can identify the five best methods?

My answer is, obviously, "Yes!" but before the fighting begins (and there will be fighting...), I intend to give myself a chance of convincing you by defining not only what I mean when I say "method", but also what makes for a good one.

What Is An Intelligence Analysis Method?

The word "method" is often used casually by analysts. When used this way, processes as different as brainstorming and Analysis Of Competing Hypotheses can both be seen as "methods" or ways to improve thinking. While such an informal definition might work at a cocktail party, it is not very helpful for professional purposes. "Method", in my opinion, should be reserved for processes that produce or substantially help the analyst produce estimative results.

Why?

It is simple, really. Estimative results are what decisionmakers want most from intelligence. It is nice to have a good description of an item of interest or a decent explanation of why something did or did not happen. Both provide useful context for the decisionmaker, but nothing beats a good, solid estimate of what the enemy or competitor or criminal is likely to do next. Defining method as something that produces estimative results means that I am connecting the most common term with the most desired result.

All the processes that help the analyst think but do not, by themselves, produce estimative results (such as brainstorming) I call "analytic multipliers". I get this from my military background, I suppose, where there are elements of combat power, such as armor or artillery, and combat multipliers, such as morale.

Analytic tools, then, are particular pieces of software, etc. that operationalize the method or the multiplier (or in some cases multiple methods and multipliers) in a particular way. For example, ACH is a method but the PARC ACH 2.0.3 software is a tool that allows the analyst to more easily do ACH.

I find these distinctions very useful in discussing the analytic process with students. If everything is a method -- if free association exercises are treated, linguistically, the same as multi-attribute utility analysis, for example -- then nothing, in the mind of the student, is a method. Clearly, not every process falls neatly into the method or multiplier camp (what is SWOT, for example, under these definitions?) but some generally agreed upon set of words to capture the large and easily recognizable differences between things such as ACH and brainstorming seems useful.

Tomorrow: What makes a good method?

Unity: A 3D Virtual World In Your Browser ... Plus A Bonus And Entirely Gratuitous Rant (Unity3d.com via Makeuseof.com)

Makeuseof.com featured the Unity plug-in for Firefox, IE and Safari the other day. They gave it pretty strong reviews, so I downloaded it to my Firefox browser and was equally impressed with the high quality 3D world the simple browser plug-in was able to create.

Unity is fundamentally a tool for developing 3D games and virtual worlds available on-demand through the internet. I am not technically sophisisticated enough to know if all of the promises the Unity team makes about the ease of programming are worth the pixels they are written with, but I was massively impressed with the simple installation and immersive quality of the 3D graphics.

With only a small plug-in (less time consuming to install than you would expect), the world that Unity creates is very realistic with excellent lighting effects, realistic physics and the capacity for ambient sound and interactive objects. You can see a screen shot below of Unity's demo island.

Clicking on the screen shot (when will people learn to always make an embeddable version...) will take you to the page where you can download the plug-in and explore the island yourself (use the mouse to look around/change direction and arrow keys to move. Hit the space bar to jump over obstacles). It takes seconds to download and install and is well worth it. While in the world, right click on the mouse and try the software on full screen as well.

What does this have to do with either teaching or intel? Quite a bit, I'm afraid. As this kind of technology becomes less and less expensive (in time and money) to use and easier to develop and deploy, the more it will be used to create educational environments that are more immersive -- and do a better job -- than traditional methods. It seems to me that understanding these technologies, their capabilities and their limitations, along with other similar technologies (like the augmented reality demo from yesterday) is critical to planning for the educational environment ahead.

On the intel side, I think it is even more important. It seems to me that virtual worlds/augmented reality are just more places where the national security intelligence community has the potential to fall behind. Back in the 90's, the IC was in the forefront of productivity technologies like instant messaging but, since then, the community has had to play catch-up on the adoption of wikis, blogs and now social networking tools like MySpace and Facebook.

The lastest example of ceding the technological high ground comes at the expense of President-elect Obama's Blackberry. Rather than figure out a way to securely and legally allow the President to use his portable communications device, the generally accepted solution appears to be to simply take it away.

Whether you like Obama or not, his ability to effectively utilize technology is one of the reasons he is where he is today. To simply take those tools away from the president -- without a fight -- seems ludicrous to me. How effective would you be if someone took away all of the tools that you use on a daily basis (electronic or otherwise)? What if you had to learn entirely new and arguably less efficient means of doing work? How long would it take you to adjust? How productive would you be in the meantime? Can we afford that right now? It is not the question of whether the President gets to keep his Blackberry or not that bothers me; it is the attitude that the technology does not matter that really scares me.

I would encourage the President to take a page from Colin Powell's playbook (which he may well be in the process of doing). When Powell became Secretary of State he had grown accustomed to using refdesk.com as part of his daily workflow. He was told he would have to give it up for "security" reasons. Secretary Powell indicated, in polite but unequivocal terms, that he was the Secretary and that security was there to support him, not the other way around. Surprise, surprise, they managed to come up with a solution that got the internet not just on the Secretary's desk but on many of the other desks in Foggy Bottom.

It can happen Mr. President-elect, it can happen...

More From The Rapidly Developing World Of Augmented Reality (Augmented Environments Lab via Serious Games)

I have written about augmented reality before but this new video from Georgia Tech's Augmented Environments Lab demonstrates a new level of ease of use and functionality. Using nothing but their software, Graz University's Studierstude Marker Trackers and an iPhone, the Yellow Jackets were able to create a well-integrated (with the real world), interactive AR application. Take a look at the video below:



The consequences of this technology to both teaching and intelligence as it becomes more well-developed and mainstream are simply staggering in my estimation.

Wirtland: A New (?) Experimental (??) Cyber (???) Nation (????) (Wirtland.com via The Virtual Worlds Roadmap)

Sorry for all the ?'s in the title but I really don't know how to introduce something this obviously out to lunch but, at the same time, intriguing and interesting.

Wirtland is, apparently, a real attempt to establish a new country. In the founders' own words, Wirtland "is an experiment into legitimacy and self-sustainability of a sovereign country without its own soil."

It is certainly not the only self-proclaimed (or micro-) nation, nor is it the first virtual one (Lizbekistan and the Kingdom of Lovely might spring to mind...) but it seems to be the first semi-serious attempt to establish such an entity. The creators of the country have put an awful lot of time into making things like flags (see picture above), Permanent Residency Forms and Applications for Citizenship. It even claims that it meets most of the qualifications under the Montevideo Convention and has applied for top level domain status with the IANA. This is an awful lot of work for just a joke.

Wirtland's current population seems to be about 31, mostly Europeans but, in all, from a surprisingly large number of countries and regions of the world. The pictures suggest a fairly young, tech savvy bunch (about what you would expect).

Personally, I hope that it is not only a joke (I recognize that part of doing something like this has to be for the fun of it...). As the last several months have painfully reminded us, our economy is mostly an elaborate social construct, virtual in almost every meaningful sense of the word. The speed and capacity of modern telecommunications networks did not cause but certainly enabled bankers and brokers to create markets in the virtual objects that are at the center of this latest crisis.

The recent instability in this ethereal economy, then, should call into question other notions of permanency. What does it mean when real things exist in an increasingly symbiotic relationship with electronic equivalents? What does it mean to "own" something in such an environment? What does it mean to be a "citizen", to have a "right to privacy", to have an "identity" at all? What are the long-term consequences of the increasing inseparability of virtual and real? Considering such things before there is another crisis seems prudent and if a thought experiment like Wirtland can help, I wish it luck.

(Note: I did not set out this morning to find out if someone had set up a new, virtual country. Instead, it was one of those serendipitous Internet things that started with the Federation of American Scientists new wiki on virtual worlds (Thanks, Kevin!), passed through the Virtual Worlds Road Map site (and associated LinkedIn group) and wound up in Wirtland.)

Surreal Saturday: The Role Of Vitamin D In Beta Cell Function -- The Musical (Gonzolabs.org via NYT)

Apparently the American Association For the Advancement of Science sponsors a "Dance Your Dissertation Contest" every year at its annual convention. You can see last year's winner below or go to the Gonzolab website and see them all. Could a "Dance Your NIE" competition be next? (Many thanks to Rex for this enlightening tidbit and for the inspriation for the cheesy "Dance your NIE" joke!)

Pump Up Your Google Search Skills (YouTube via Digital Inspiration)

Digital Inspiration featured this excellent short video on getting the most out of your Google searches featuring Matt Cutts, a senior engineer at Google. The video starts out with a number of well-known tips but quickly gets to some stuff that I did not know Google could do. Even if you are an expert, this is a good 5 minute refresher course on getting the most out of Google.

Active Protection System Detects, Destroys Incoming (YouTube via Gizmodo)

The amazing piece of footage below shows a Raytheon-developed device that can launch and direct a counter-munition at an incoming tank round so that it destroys the incoming round before it hits the target. Amazing capability and equally amazing footage. Seen first on Gizmodo.

T-INT: Thanksgiving Week Intelligence You Can Use (Link List)

Thanksgiving is a great holiday. Not only is it a time for us all to give thanks for our blessings, it is also a time to give something back to the communities in which we live. So, be thankful for your health, your family, your friends and your freedom and get out there and give to the local food drive, work for a good cause or just write a check to your favorite charity.

But...

We'd all be lying if we didn't admit that we are going to be doing a little bit of eating and maybe even some shopping over the next several days. For some of us (those who love Black Friday sales) this is heaven and, for some of us (on a !@#$ diet), this is ... not heaven.

Here are some links to get you through the season:

Topics To Avoid At Thanksgiving Dinner. What a perfect way to start the holler-days: A huge fight with your crazy Aunt Edna over elections/religion/economy/what-have-you. Don't do it. Really. Don't. Do. It.

Last Minute Thanksgiving Tips And Tricks. Never cooked a turkey in your life? Your significant other's parents coming over for dinner? Been there, my friend, and so have these people.

Coupon Codes For Discount Eating Out. So the turkey is dry and Aunt Edna put the rum in the dressing instead of the sweet potato pie. Don't let this spoil your dinner; go out! No need to pay full price, though, and this site offers links to coupons, etc. for discount meals at a variety of popular chain restaurants.

Full now? Try these shopping sites:

Top 5 Black Friday Tips. For shopaholics only.

The Pogue-o-matic. Not sure what gadget to get someone on your gift list? Not sure what gadget to ask for? The personal technology columnist for the NYT, David Pogue, has a fun, interactive site to help you decide.

Cool Spy Gadgets. Mom, Dad, don't know what to get your intel studies student? Here is your shopping list. I guarantee that they would enjoy almost anything on this list.

OLPC Give One, Get One. Here's a way to give something AND do some shopping. The One Laptop Per Child Foundation is offering its buy one, give one promotion again this year. You pay for two OLPCs and you get to keep one of them while the other is given to a child who needs one. It is a very cool piece of technology that works exactly as advertised (I got one and gave one last year).

I hope this helps at least some of you and, to the 50,000(!) of you who took some time out of your day to read, comment or link to these posts over the last year, thank you for making this a great first year blogging at SAM! Happy Thanksgiving!

COOLINT: Minority Report Interface Now Available (Oblong.com via KZERO)

Everybody remembers the COOLINT application from the 2002 film Minority Report. It was this big computer screen that Tom Cruise was able to manipulate by waving his hands in the air. Well, Oblong Industries has built it. Take a look at the video below (seen first on KZERO).


g-speak overview 1828121108 from john underkoffler on Vimeo.

Six years from science fiction to reality. Pretty good..and a little scary.

Obama's Intelligence Advisors (Intelligence Online via Intellibriefs)

Intellibriefs pointed to an interesting Intelligence Online article (free with registration) about President-elect Obama's intelligence advisors last week. Intelligence Online is produced in French and English out of Paris (think of it as kind of a French Stratfor). They have several subordinate publications including Africa Intelligence.

The most interesting thing about the Obama piece has to be the "cheat sheet" associated with it (click on it to see the full version).

Live Pirate Map! No, Not That Kind... (ICC-CCS.org via beSpacific)

The law and technology blog, beSpacific, points to an interesting product by the International Chamber of Commerce's Commercial Crime Services division today. It is a Google map mash-up that shows all of the piracy and armed robbery incidents reported to the International Maritime Bureau in 2008. There is no embeddable version but click on the map below and it will take you to the live site. While you are there, click around a bit. There are some interesting additional links.

Related Posts:

Ship Tracking Resources

Elementary School Intelligence Analysts Or Why Bob Heibel Is The Man (Goerie.com)

Bob Heibel is responsible for some amazing innovations in intelligence over the years. After a full career in the FBI and a short stint in the corporate security world, he started the program here at Mercyhurst 16+ years ago -- long before anyone else was even thinking about applied intelligence studies programs.

Back in 2003, he thought that intelligence educators worldwide should have an organization of their own and was the driving force behind the development of the International Association For Intelligence Education (IAFIE).

Now he is getting students interested in intelligence analysis while they are still in elementary school! It is a really interesting program and these are some lucky young students. Congratulations to Bob and the Mercyhurst intel studies students who are making this happen!

Japanese Space Elevators, Terrorism In The Phillipines And Graffiti (Catching Up)

I love teaching strategic intelligence but it does take it out of me. The centerpiece of the class is a series of small group projects for real world decisionmakers on a variety of business, law enforcement or national security topics. In the past, we have done projects, for example, for the NIC, the 66th MI and the Army War College on topics ranging from bioterror preparedness in Europe to the role of non-state actors in sub-saharan Africa.

This last term (we are on the quarter system at Mercyhurst so we are just winding up our fall term) was no exception. I had good solid projects from decisionmakers that wanted and needed the results coupled with teams of hard-working, dedicated student analysts to get things done. 10 teams to be exact. For those of you keeping score at home, that is technically known as "a ton of strategic intelligence projects".

Anyway, Strategic (as it is fondly -- maybe? -- called by the students) kept me from blogging about all of the other interesting stuff that has been happening lately as well. I will try to catch up next week, but here are some recent highlights.

Cyndi Lee, a current grad student, recently had an article published by ISN talking about the prospects for a Japanese space elevator. Once thought to be just science fiction, Cyndi's research suggests that the technology is either here, right now, or within reach and that the Japanese are serious about trying to make it happen.

I may have mentioned this before but my long-time writing partner, Diane Chido, also had a piece published by ISN on the "exportability" of the model used in the Phillipines to reduce the threat posed by Abu Sayyaf. If you missed it (or I forgot to mention it), it is worth your time.

Diane has also recently established herself as a freelance intelligence analyst and launched the company DC Analytics. I am prejudiced, of course, but if you have read her thesis or any of her other articles on ISN or her books or her essays in Competitive Intelligence Magazine, you know she is a talented professional with all the right credentials to go "indie". She joins Mark Blair and Mike Thomas of Dagir Co. as Mercyhurst grads who have become intelligence entrepreneurs.

Finally, one of the strategic teams mentioned earlier, working for the Erie Graffiti Task Force, has really been making a spash locally. The Task Force is an ad hoc group of local business owners, politicians and academics trying to constructively tackle the issue of graffiti. With the limited resources they have at their disposal, they really needed strategic intelligence to help inform their strategy. The project turned out very well and was extremely well-received by not only the Task Force, but also by the Mayor of Erie, Joe Sinnott, who sat in on the final brief.