Monday, November 19, 2012

Want To Do Something Different This Thanksgiving?

Some of the action on DAGGRE.org
Eating, shopping, driving, yelling, more eating...I get it.  What is Thanksgiving without its traditional activities?

But, if you were looking to do something a bit different, you might want to check out the recent updates to the DAGGRE prediction market.

(I know, I know, I am such a shill for DAGGRE...)

Yes, in the interests of full disclosure, I am a member of this IARPA funded research project and yes, I probably love it way too much but the most recent changes to our site's software are, frankly...cool.

Real cool.  Like never before seen cool.

Why?  Because now you can do real-world, real time linchpin and driver analysis in a prediction market.

I am pretty excited about this but it might not be obvious why, so let me break it down.  Prediction markets typically work by asking resolvable questions like "Will Despot X be out of power by 31 DEC 2012?"  Then various people interested in that question will go into the market and make "edits" that assess the probability of the answer to this question being yes or no.  For example, if I thought Despot X would almost certainly be out of power by the end of the year, I would go in and change the probability to, say, 90%.  Someone else might go in after me and think, "What an idiot!" and change it back to 25%.  When 31 DEC rolls around one of us will be closer to right and the other closer to wrong.  The one closest to right scores the most points on the market.

This system works pretty well with straightforward questions that obviously lean strongly in one direction or another (EX:  "Will a shooting war break out between the US and Canada before 31 DEC 2012?"  Uh...no.).  It works significantly less well with more nuanced questions that really deserve to be teased apart.

This is essentially what the CIA's Deputy Director For Intelligence, Douglas MacEachin, was trying to do back in the early 1990's when he insisted on having his analysts identify linchpins and drivers.  To quote an early version of the CIA Tradecraft Manual, drivers are "key variables...that analysts judge most likely to determine the outcome of a complex situation" while linchpins are "the premises that hold the argument together and warrant the validity of the conclusion." 

This kind of analysis is pretty sophisticated stuff and it really is what makes the difference between a bald estimate ("X is 80% likely to happen") and the kind of estimate most decisionmakers expect ("Despite A and due primarily to B and C, X is 80% likely to happen.").

Before, you simply couldn't do this kind of stuff in a prediction market.  Now, with the most recent upgrade to the DAGGRE market, you can.  Let's go back to our Despot X example.  We know that rebel forces are trying to take a key city in this despot's country.  We also strongly believe that if the rebels take the city the despot's days are numbered.  Before, our estimate of the despot's longevity was a mish-mash of many factors, one of which was the possible(?)/probable(?) fall of the key city and our estimate probably was much more wishy-washy than we wanted it to be.

The DAGGRE market now lets you not only make estimates on both how likely the city is to fall and on how likely the despot is to stay in power but also allows you to make the answer to one of these questions, an assumption for the other (EX:  "Assuming key city falls, Despot X is 90% likely to be out of power by 31 DEC 2012.").

In my mind this is a huge step forward in making prediction markets more useful to real-world analysts working on real-world questions.  Definitely worth taking a few minutes to check out over Thanksgiving!

No comments: