Friday, March 11, 2011

Passport Ownership Cures Diabetes (Analytic Traps)

(A big tip of the hat to Allen Thomson for this one!  Besides having an obviously  keen eye for learning opportunities, Allen is known for his exhaustive source documents typically hosted on the Federation Of American Scientists website.  You can see some of his work on the Israeli Strike on Syria, the allegations of cooperation between Burma (Myanmar) and North Korea or even his source report on the Russian Okno and Krona space surveillance systems by clicking on the links.  Note:  These are sizable reports and may take some time to download but are definitely worth it.)

Check out the two maps on the right.

It proves conclusively that passport ownership is a cheap and easy cure for diabetes, right?  I mean, look at the maps!  There is almost a perfect correlation between the so-called "diabetes belt" in the south and the lack of ownership of passports in the same region.  Increase the number of passport holders and the diabetes epidemic is over! 

If you have ever heard the classic scientific warning that "correlation does not imply causation" and did not understand what that saying meant, this is a perfect example.  Just because two things are happening at the same time does not necessarily mean that one caused the other.

Analysts typically spring this trap when the connection is not as obviously flawed as it is in this case.  The human mind is extremely good at seeing patterns -- even when they are not there.

Does correlation never indicate causation?  No, that is clearly false as well.   In fact, correlation is a necessary condition for causation -- necessary but not sufficient

The best way to expose this trap appears to be to imagine the counterfactual.  In the case above, imagine what it would be like if all those southerners actually had passports.  Would that, in turn, reduce any of the known risk factors for diabetes?  Unlikely.  It would appear to be merely a coincidence.

Should the analyst just discard the evidence at this point?  The answer is "No" once again.  Kicking back and pondering why this apparent coincidence exists might well yield new insights into diabetes or passport ownership or both.  Even if such thinking might not seem too valuable now, who knows what the future will bring? 

In case you need another example, consider the third chart which shows the rise in global average temperature over time as a function of the number of pirates.  Clearly, global warming is caused by the decrease in the number of pirates. 

For those of you who find this argument particularly compelling despite what I have written, I would remind you that "Talk Like A Pirate Day" is September 19th of each year where you, too, can do your part for the environment...


Ray said...

Thanks. Interesting example.

Danny said...

Just one more reason why structured methods are so important. said...

PERFECT cartoon highlighting this here - enjoy!

Anonymous said...

I was looking at these maps, but from a critical thinking perspective, I would like to see some data on the weight, fitness level, diet, income, medical access of passport holders. Since you need a passport for international travel, perhaps passport holders have higher incomes, which could lead to better diet, increased levels of personal fitness, etc., which could lead to lower levels of diabetes. I am not saying that getting a passort will reduce a persons risk of developing diabetes. I am just saying that if you can obtain and analyze all available data, there may actually be a link between holding a passport and likelyhood of developing diabetes.

Mike said...

On a related note ... "How to Use Statistics to Live Longer" on Lifehacker:!5781482/how-to-live-forever

ctwardy said...

As a few of the links note, a correlation like this does suggest that *something* causal is going on, just not necessarily A->B. In this case, likely a common cause, eg socioeconomic status.