In Praise Of Open Source (Wired and Haft Of The Spear)

Michael Tanji of Haft of The Spear and Wired's The Danger Room, has posted a good explanation (and a little history) of the value of open source intelligence.

My own thoughts run close to his but I have a few additional ideas that I think are worth considering. First, when most people think of open sources, they think of the internet and all of the information that is now available. This thought, particularly among some of those of a certain generation, it typically followed by the thought that you can't believe any of it; that little or none of it is credible.

That, of course, is just plain false.

The truth is that there is a vast and growing body of highly credible open source information available through the internet and countless other open sources. To get it you need to be trained in how to use the tools of the trade and be well equipped with critical thinking and critical reading skills. Several good academic libraries -- such as the University of Wisconsin-Milwaukee and Cal Berkeley -- have taken up the challenge and posted a good set of guidelines for assessing source reliability on the internet.

Second, open source information is just a small part of what open source intelligence can do. Adding a complete analytic capability to the wealth of open source information creates a powerful tool for assisting decisionmaking. Here are a couple of examples:

• Alternative Analysis. The recent CRS report highlighted the potential value of conducting an open source review in addition to a classified review of important intelligence questions. First, such a document would be a worthy test of the classified intelligence. Second, if the analysis was consistent with the classified view, then the open source version could be used to ensure that the estimate received the widest possible dissemination (highly classified documents often come with so many bureaucratic obstacles to using the intelligence that they may not be read at all). If the open source analysis was not consistent with the classified view (i.e. the classified view not only passed the test but was clearly better), then it would help justify the 43.5 billion dollar US intelligence budget.
  
• Stretching Limited Resources. Even with a 43.5 billion dollar budget, it is still a big world. That money can run out very quickly and many parts of the world get little if any coverage. For a fraction of the cost of most classified operations, the intelligence community could contract with academia (yeah!) and other commercial vendors to track important but unresourced intelligence requirements. Such a program would also go a long way toward supporting the intelligence studies programs starting up across the country.
  
• Some would say that the community already does this through its own, very capable OpenSource Center and by contracting with such vendors as Janes, Stratfor, Oxford Analytica, the Economist Intelligence Unit and the amazing Australian telecoms research firm, Budde.com. While these services are very, very good, what I have in mind is different. I am talking about a fully loaded intelligence system that is capable of both watching and producing routine analytic products in its assigned requirements but also of surging to meet short term demands. Such a system, consisting of a series of centers around the country would not cost much but would add perspective and diversity to the intelligence analysis equation and could respond to a much wider variety of decisionmakers including law enforcement, businesses, non-governmental organizations and even private citizens.
• Providing A Starting Place. Classified systems, by their nature, have to keep intelligence collection operations focused and intelligence products in restricted channels. Open source can cast a wider net. Particularly useful in a crisis, analysts using just open source information can rapidly build a useful conceptual model upon which, at a minimum, other analysts can layer classified intelligence to add nuance.
  
• Take Maximum Advantage of Technology. There was a day when the intelligence community had the latest technology. That advantage is all but gone. In fact, because of security requirements, today's technology often comes to analysts very late and is often very expensive. Open source, in this environment, is very nimble. It can test and adopt new technologies quickly and can evolve to meet new challenges with the latest tools.
  

These are reasons for the intelligence community to push the boundaries of what we can do with open sources today, while the classical view of intelligence is still considered relevant. The future, whether the current crop of intelligence professionals sees it, believes it or understands it, belongs to open source.

Finding Key Words In Video/Audio (MIT)

MIT has posted a neat piece of web -based code designed to search its growing library of free lectures. There doesn't seem to be too many lectures of interest to the intelligence community so far but I only did a cursory search. The interface is worth a look, however, as it is very slick. I suspect that the database will only get better with time.

Gladwell On The Value Of Learning Like A Marine (YouTube)

Malcolm Gladwell (author of Blink and The Tipping Point) recently spoke at the 92 St. YMCA in New York City. His talk centered on some recent writing he had done but he touches, in this clip, upon the value of treating education the way the Marines treat training rather than the way modeling agencies treat models. It is only about 4 minutes but worth it.

What Do Che Guevara And The CIA Have In Common? (COIN: CARL)

They have both written manuals on staging a revolution! The good people at the Combined Arms Research Library have made available online both Guevara's 1961 text, "Guerrilla Warfare" (direct download here) and the CIA's 1984 book used by the Contras titled, "Psychological Operations in Guerrilla Warfare" (direct download here).

The two texts overlap only in areas. Guevara's book is a more complete manual for the revolutionary while the CIA text is more about crafting the message of the revolutionary forces. There are, however, a number of interesting sections that will resonate with today's COIN specialists. I found Guevara's section on intelligence (which he begins with a quote from Sun Tzu: Know yourself and your adversary and you will be able to fight a hundred battles without a single disaster.) to be the most interesting. Here are a few highlights:

• "Nothing gives more help to combatant forces than correct information. This arrives spontaneously from the local inhabitants, who will come to tell its friendly army, its allies, what is happening in various places; but in addition it should be completely systematized."
  

• "Men and women, especially women, should infiltrate; they should be in permanent contact with soldiers and gradually discover what there is to be discovered."
  

• "The peasants, not accustomed to precise battle language, have a strong tendency to exaggerate, so their reports must be checked."

The CIA manual contains a number of interesting nuggets as well:

• "Every member of the struggle should know that his political mission is as important as, if not more important than, his tactical mission."
  

• "Armed propaganda in small towns, rural villages, and city residential districts should give the impression that our weapons are not for exercising power over the people, but rather that the weapons are for protecting the people..."
  

• "Cover ("Facade") Organizations. The fusion of several organizations and associations recognized by the government, through internal subjective control, occurs in the final stages of the operation, in close cooperation with mass meetings."
  

• "Control Of Mass Demonstrations. The mixture of elements of the struggle with participants in the demonstration will give the appearance of a spontaneous demonstration, lacking direction, which will be used by the agitators of the struggle to control the behavior of the masses."
  

• "Too often we see guerrilla warfare only from the point of view of combat actions. This view is erroneous and extremely dangerous. Combat actions are not the key to victory in guerrilla warfare but rather form part of one of the six basic efforts. There is no priority in any of the efforts, but rather they should progress in a parallel manner. The emphasis or exclusion of any of these efforts could bring about serious difficulties, and in the worst of cases, even failure. The history of revolutionary wars has shown this reality."
  

By far, my favorite part of this text, however, is in the appendix which seeks to acquaint the revolutionary with the rhetorical practices of anaphora, prolepsis and preterition (among others). You kinda have to see it to believe it...

Origins Of FBI Counterintelligence And More (LOC)

The Library of Congress provides an excellent teaching/training/research resource through its periodic webcasts. One of the most recent is a 50+ minute presentation on the Origins of FBI Counterintelligence done by Ray Batvinis (who has just authored a book on the same topic).

Other webcasts that may be of interest include:

Illicit: How Smugglers, Traffickers and Copycats are Hijacking the Global Economy

Why Terrorist Ideologies are So Powerful

War of Nerves: Chemical Warfare from World War I to Al-Qaeda"

The African Colonial State and the Encounter with Decolonization

China's Return to Tradition: How to Interpret the New Forces Emerging in China

China Rediscovers its Own History

You will need a RealPlayer plug-in to view the webcasts.

SAM Afghan Edition, Part II (IEDs In Iraq And Afghanistan: CRS)

IEDs are well known in Iraq but I have not seen as much concerning their use in Afghanistan. This recent report from the CRS gives some insight into IED use in both places and some of the countermeasure initiatives currently under way.

Highlights from the text include:

• "Improvised explosive devices, also known as IEDs, roadside bombs, and suicide car bombs, have caused over 70% of all American combat casualties in Iraq and 50% of combat casualties in Afghanistan, both killed and wounded."
  

• "In Afghanistan, the IED munitions supply is supported by funds from an expanding opium trade."
  

• "However, DOD officials have also stated that in Afghanistan, Taliban fighters have increased both the number and lethality of their IED attacks."
  

• "DOD has found that insurgents build and deploy IEDs by using networks that, for centuries in Afghanistan and Iraq, have formed the sinews of commerce and survival for tribes and factions. A typical IED terrorist cell consists of six to eight people, including a financier, bomb maker, emplacer, triggerman, spotter, and often a cameraman. Videos of exploding U.S. vehicles and dead Americans are distributed via the Internet to win new supporters."
  

• "Threat data about IEDs is tightly controlled by DOD to avoid giving feedback to the enemy about the effectiveness or ineffectiveness of different IED designs. Also, proprietary rights must be protected for those companies who produce IED countermeasures. However, these controls may sometimes limit access by other companies to important information about the effectiveness of anti-IED systems as they are tested or used in battle."
  

SAM Afghan Edition, Part I (RAND)

There are two good reports on Afghanistan in my feeds this morning. The first is by Dr. Seth Jones from RAND and Georgetown University and contains testimony dated December, 2007 regarding the state of the Afghan insurgency given to Canada's Senate. You can download the full text here. I will try to get the second report out later today.

The assessment is fairly grim in the details but provides some concrete suggestions for the way ahead. Here are some of the highlights (Boldface, notes, italics and hyperlinks are mine):

• "The evidence I have collected from repeated trips to Afghanistan between 2003 and 2007 indicates that there is an increasingly violent insurgency that threatens the country. It includes a range of insurgent groups, such as the Taliban, the Haqqani network (Note: Operates in the Fata region of Pakistan. For a complete tribal breakdown of the region click here), foreign fighters (including al Qaeda), Hezb-i-Islami, criminal organizations, and some allied tribes and sub-tribes. The overall number of insurgent-initiated attacks increased by 400 percent from 2002 to 2006, and the number of deaths from these attacks increased over 800 percent during the same period."
  

• "Provinces that I could drive to only a few months ago, such as Wardak and Lowgar, are now off limits except to those willing to gamble with their lives."
  

• "As one senior NATO official told me, NATO and Afghan forces control at most 20 percent of southern Afghanistan. The rest is controlled by Taliban or a range of sub-state groups."
  

• "The situation in Afghanistan and Pakistan significantly impacts the national security of NATO countries, including Canada. The Pakistan-Afghanistan border region is the headquarters of al Qaeda, which is in some ways a more competent international terrorist organization than it was on September 11, 2001."
  

• "What explains the insurgency in Afghanistan that now engulfs roughly half the country? 'The answer is simple,” one senior Afghan government official told me in October 2007. 'The people are losing faith in the government. Our security forces cannot protect local villages, and our institutions struggle to deliver basic services.'"
  

• "At its core, the insurgency in Afghanistan is not about religion, as some mistakenly believe...In general, the problem is not that most Afghans inherently support the Taliban. It is that patience with the Afghan government is wearing thin...Indeed, the primary challenge in Afghanistan is one of governance."
  

• "Perhaps the most basic governance challenge in Afghanistan is security."

• "Another major challenge is corruption. Afghans have become increasingly frustrated with national and local government officials who are viewed as corrupt and self-serving."
  

• "Afghanistan has also faced challenges from outside actors, which have undermined governance. The first is a limited NATO role."
  

• "Unfortunately, there are no short-term solutions to Afghanistan’s challenges. Research that the RAND Corporation has done indicates that it takes an average of 14 years for governments to defeat insurgent groups. Many also end in a draw, with neither side winning.Insurgencies can also have long tails: approximately 25 percent of insurgencies won by the government and 11 percent won by insurgents lasted more than 20 years."
  

• "This does not mean, however, that Canada or other NATO countries need to – or should – win the insurgency for Afghans. Quite the reverse. While outside actors often play an important role, victory is usually a function of the struggle between the local government and insurgents."
  

• "This means that Canada and other NATO countries can be helpful in assisting the Afghan government. Four steps may be helpful."
  

• "1. Remove from power and prosecute key individuals involved in corruption and criminal activity, including Afghan government officials."
  

• "2. Increase NATO and Afghan National Army resources in the south."

• "3. Establish a regional approach to Afghanistan, including countering the sanctuary in Pakistan."

• 4. Establish an institutional arrangement to improve international cooperation.

• "Afghanistan is not hopeless. To be fair, NATO operations have had mixed success thus far. But the insurgency will ultimately be won or lost in the rural areas of Afghanistan, not in the cities. Success in ending the insurgency will take time and sufficient resources. It would be a tragedy if the naysayers in Canada succeeded in reducing their country’s commitment."
  

New, Simple Prediction Market Tool (Predictify.com)

Prediction markets have been around for a long time and I have mentioned them here briefly before. Fundamentally, they operate like futures markets (some would say gambling establishments). In the simplest version of these systems, people essentially place bets on what the future will bring. The person who gets closest wins. In more complex systems, people can actually buy and sell the "bets".

The idea is that, if you have enough people involved, the going price will converge, over time, on the correct price. Of course, anything can be "valued" this way and probably the most famous predictive markets are the Iowa Electronic Markets. They have run a market on who will win the presidential election (among others) for a number of years and have been very successful at predicting the results. Currently you can buy futures -- i.e. contracts that will pay a dollar the day after the presidential election -- in the eventual Democratic candidate for about 60 cents and contracts for the eventual Republican candidate for about 40 cents. These prices predict, at this point, a Democratic victory because people are willing to pay more for a Democratic candidate than a Republican candidate while still only getting a dollar after the election.

The US government has played around with this idea (FutureMap was a predictive market idea that was linked to the ill-fated Total Information Awareness program which is a whole other story...) and probably still is in one form or another.

The goal of all these markets is to tap into the collective wisdom of many people to help make accurate predictions concerning the future or at least the odds that certain futures will occur. There are a number of books and papers that touch on this topic right now. The Wisdom of Crowds is one but my favorite is Gut Feelings.

That is a long preface to get to a new predictive market tool available at Predictify.com. I have used it to set up a market in oil prices that you can see here. It is an easy way to get input on discrete questions. The team at Predictify help you mold your question and make it more specific in addition to helping you identify the exact source you will use (in my case, Bloomberg.com) to identify the winners and losers. In all, it was painless and I already have over 50 "answers" to my question.

If you are interested in exploring the power of another predictive market, particularly one with a national security focus, see Strategypage.com. You can search for other predictive markets here. For general information about the prediction market industry (yes, it is an industry) follow the newly formed Prediction Market Industry Association.

Blogs Of Interest (INMM and Web 2.0)

There are two new blogs that are worth following if you are interested in the particular niche of information that they address. The Mercyhurst student chapter of the Institute of Nuclear Material Management maintains an on-going blog about nuclear proliferation, counter-proliferation and other issues here. Likewise, Chris, a student at Mercyhurst, is exploring how Web 2.0 tools could be used by the intelligence community here.

Intel Brief: Afghan Poppies Fear Not (ISN)

Luke Handley just published a well researched intel brief on the ISN concerning the future of Afghanistan's opium poppy harvest.

Open Source Intelligence: Issues For Congress (CRS)

The good people at OpenCRS have made a December 5, 2007 report on OSINT available to the public on their website. The report contains quite a bit of good background info including a summary of the debate on the value of open source to the intelligence community. According to the report, while "Intelligence professionals generally agree that open source information is useful", there are three main positions within the community with regard to the true value of open source information:

• "The first holds that policymakers simply derive less value from such information than from clandestinely-collected secrets."
• "The second view asserts that open source information should be viewed not only as an important contextual supplement to classified data, but also as a potential source of valuable intelligence, in and of itself."
  
• "Proponents of the third view adopt a “middle-ground” position, arguing that open source information probably will never provide the “smoking gun” about some issue or threat, but that it can be instrumental in helping analysts to better focus or “drive” clandestine collection activities by first identifying what is truly secret. Open sources therefore should be viewed as an analyst’s “source of first resort.”
  

Also of interest are the laundry list of current obstacles to analysts trying to using open source and the recent history of criticism of the community's failure to use open source more extensively (it is a lengthy parade of horribles...). Probably of most interest is the list of things the CRS mentions that Congress could do to help the open source movement. These include, according to the report:

• "One way is to examine specific budget areas in which spending on open source currently can be identified."
• "Another opportunity occurs when the DNI submits to congressional intelligence committees an annual report reviewing analytical products. Arguably, these annual reports should address the use of open source information. However, oversight committees could ask for additional information on open source utilization if needed."
• "...an approach that might be considered in some situations would be a request for an alternative analysis of a specified topic solely based on open sources in order to compare it with all-source analyses." (Note: I'll do it!)
  
• "Some may argue that Congress should consider an amendment to copyright law that would cover the open source efforts of intelligence agencies. Removing uncertainty of the extent of copyright would facilitate open source efforts and facilitate the widest possible use of the information by public officials."
• "Some have proposed making the Open Source Center a component of the Office of the Director of National Intelligence (ODNI) while essentially retaining its current roles and missions."
• "The advantage of placing the NOSC directly under the DNI would be to enhance the prestige of the open source discipline by raising its profile, fencing the funding, and ensuring its independence from shifting priorities within the CIA where human intelligence collection inevitably makes heavy and continuing demands on senior officials."
• "Placing the NOSC within the ODNI could also facilitate the NOSC’s ability to support law enforcement agencies and state, local, and tribal entities." (Note: Good point.)
  
• "A more radical, approach would be to establish an Open Source Agency completely outside the Intelligence Community (in addition to the existing Open Source Center). The goal would be to provide open source information not just to intelligence analysts but to all elements of the Federal Government including congressional committees." (Note: The basis for the PIA -- Public Intelligence Agency. Where do I sign up?)
  

Lessons Learned From Past Counter-Insurgency Operations (RAND)

Not sure how early in 2007 RAND published this study but it draws some useful conclusions from a comparative case study analysis of six previous COIN operations including these highlights from the summary:

• "It is important that counterinsurgents understand local dynamics so that all theaters of the conflict can be understood in context. This knowledge can help exploit cleavages and encourage competition among insurgent factions, which was done in the Philippines and, with less success, in Vietnam. In Vietnam, El Salvador, and Colombia, counterinsurgents used indigenous intermediaries with established social networks to earn the trust of the population and psychologically unhinge the insurgents."
  

• "Depending on the situation, a hands-off approach is sometimes necessary to allow the host nation to learn which methods are most effective in dealing with an insurgency, considering its own strengths and limitations."
  

• "Foreign or even host nation counterinsurgents who are not from the local area of operations should assume that they will have limited opportunities to convey their good intentions. Consequently, they may be viewed more favorably from the outset if they are perceived as contributing to progress and not to chaos."
  

• "Counterinsurgents should strive for “unity of command,” akin to the bureaucratic structure of the CORDS program in Vietnam, so that there is fusion and continuity among counterinsurgency programs."
  

• "Finally, counterinsurgents should analyze solutions in terms of long-term effectiveness, not short-term necessity."