How To Write A Mindnumbingly Dogmatic (But Surprisingly Effective) Estimate (Part 2 - Nuance)

In my last post on this topic, I outlined what I considered to be a pretty good formula for a pretty good estimate:

• Good WEP +
• Nuance +
• Due to's +
• Despite's +
• Statement of AC = 
• Good estimate!

I also talked about the difference between good WEPs, bad WEPs and best WEPs and if you are interested in all that, go back and read it.  What I intend to talk about today is the idea of nuance in an estimate.

Outline of the series so far (Click for full page version)

Let me give you an example of what I mean:  

• The GDP of Yougaria is likely to grow.
• The GDP of Yougaria is likely to grow by 3-4% over the next 12 months.

Both of these are estimates and both of these use good WEPs but one is obviously better than the other.  Why?  Nuance.

Mercyhurst Alum Mike Lyden made a stab at defining what we mean by "nuance" in his 2007 thesis, The Efficacy of Accelerated Analysis in Strategic Level Intelligence Estimates.  There he defined it as how many of the basic journalistic questions (Who, What, When, Why, Where, and How) the estimate addressed.  

For example, Mike would likely give the first estimate above a nuance score of 1.  It really only answers the "What" question.  I think he would give the second estimate a 3 as it appears to answer not only the "What" question but also the "When" and "How (or how much)" questions as well.  Its not a perfect system but it makes the point.

In general, I think it is obvious that more nuance is better than less.  A more nuanced estimate is more likely to be useful and it is less likely to be misinterpreted.  There are some issues that crop up and need to be addressed, however - nuances to the nuance rule, if you will.

• What if I don't have the evidence to support a more nuanced estimate?  Look at the second estimate above.  What if you had information to support a growing economy but not enough information (or too much uncertainty in the information you did have) to make an estimate regarding the size and time frame for that growth?  I get it.  You wouldn't feel comfortable putting numbers and dates to this growth.  What would you feel comfortable with?  Would you be more comfortable with an adverb ("grow moderately")?  Would you be more comfortable with a date range ("over the next 6 to 18 months")?  Is there a way to add more nuance in any form with which you can still be comfortable as an analyst?  The cardinal rule here is to not add anything that you can't support with facts and analysis - that you are not willing to personally stand behind.  If, in the end, all you are comfortable with is "The economy is likely to grow" then say that.  I think, however, if you ponder it for a while, you may be able to come up with another formulation that addresses the decisionmaker's need for nuance and your need to be comfortable with your analysis.
• What if the requirement does not demand a nuanced estimate?  What if all the decisionmaker needed to know was whether the economy of Yougaria was likely to grow?  He/She doesn't need to know any more to make his/her decision.  In fact, spending time and effort to add nuance would actually be counterproductive.  In this case, there is no need to add nuance.  Answer the question and move on.  That said, my experience suggests that this condition is rather more rare than not.  Even when DMs say they just need a "simple" answer, they often actually needs something, well, more nuanced.  Whether this is the case or not is something that should be worked out in the requirements process.  I am currently writing a three part series on this and you can find Part 1 here and Part 2 here.  Part 3 will have to wait until a little later in the summer.
• What if all this nuance makes my estimate sound clunky?  So, yeah.  An estimate with six clauses in it is going to be technically accurate and very nuanced but sound as clunky and awkward as a sentence can sound.  Well-written estimates fall at the intersection of good estimative practice and good grammar.  You can't sacrifice either, which is why they can be very hard to craft.  The solution is, of course, to either refine your single estimative sentence or to break up the estimative sentence into several sentences.  In my next post on this, where I will talk about "due to's and "despite's", I will give you a little analytic sleight of hand that can help you with this problem.

Farengar Secret-Fire Has A Quest For You! Or What Video Games Can Teach Us About Virtual Intel Requirements

A couple of weeks ago, I wrote a post about the 3 Things You Must Know Before You Discuss Intelligence Requirements With A Decisionmaker.  That post was designed for intel professionals who have the luxury of being able to sit down with the decisionmakers they support and have a conversation with them about what it is they want from their intelligence unit.  I also stated that doing this in a virtual environment or on an automated requirements management system like COLISEUM was both more difficult and something I would discuss in the future.

Well, today is your lucky day!  The future is here!

When I think about who does requirements best in a virtual environment, I think about video games.  Particularly, I think about massively multi-player online role-playing games (MMORPGs for short).  Very, very specifically, I think about the questing systems that are standard fare in virtually all of these types of games.  

Quests are the requirements statements of games.  I have included an example of a quest below (it is from a game called Skyrim which is awesome and highly recommended).  These quests differ from intel requirements in that almost all of them are operationally focused (What do we need to do to accomplish our mission?) instead of intelligence focused (What do we need to know about the other guy to accomplish our mission?).  That said, there are still a number of things we can learn from a well-formulated quest that will make intel requirements in a virtual environment easier to craft and to understand.

Retrieved from Immersive Questing

Video game designers know they have to get quests right the first time.  They don't have an opportunity to talk to the players outside the game so all the necessary information needs to be in the quest itself.  On the other hand, they have to make the quest seem realistic.  Failing to maintain this balance runs the risk of creating an unplayable game.  As a result, video game designers have developed a number of conventions that allow the quests to sound real but be complete.  Intel has the "real" part down so it is about making sure that it is complete that matters.  In this respect, the final version of a good virtual intel requirement bears a remarkable resemblance to the final version of a good quest.  Here are the specifics:

• They both provide background.  Why am I doing this?  What is the context for this quest?  In video games, putting the quest in context allows the story to unfold.  In intelligence work, this context allows the intelligence professional to better understand the decisionmaker's intent.  This, in turn, allows the intelligence professional to have a better understanding of the kinds of information and estimates that will prove most useful.
• They both define terms.  In the quest above, I am to look for a Dragonstone.  What is a Dragonstone?  The quest defines that for me.  In intelligence work, agreeing on definitions of terms (particularly common terms) is incredibly helpful.  For example, you get a request to do a stability study on Ghana.  What term needs to be defined before you go ahead?  Stability.  We do this exercise every year in our intro classes.  There are multiple definitions of stability out there.  Which one is most appropriate for this decisionmaker is a critical question to ask and answer before proceeding.
• They both use terms consistently.  If I encounter another quest asking for me to find a Dragonstone, I can count on it being the same thing I am looking for in this quest.  Likewise, in an intelligence requirement, if I define a term in a certain way in one place, I will use that term - not what I think is a synonym, no matter how reasonable it sounds to me - consistently throughout the requirement.  
• They both often come in standard formats.  All video game players are familiar with a variety of standard quest formats such as the Fetch Quest (like the one above) where the task is to go, get something, and bring it back.  Intelligence requirements also come in more-or-less standard forms such as requests for descriptive or estimative intelligence.  Categorizing requests for intelligence and then studying them for similarities should allow an intelligence unit to develop a list of useful questions to ask based simply on the type of request it is.   

Requirements statements, whether managed in person or virtually, are almost always going to start out messy.  Without the advantage of a back-and-forth, personal conversation, the virtual requirements process has a greater potential, however, for breakdown.  Thinking of the requirement as a quest allows intelligence professionals to re-frame the process and focus on the essential elements of the requirement and, perhaps,  anticipate and address predictable points of potential failure in advance.

Look for the final part of this series later this summer when I talk about all the things you need to think about in the middle of requirements discussion with a decisionmaker!