Intelligence And Crowdmapping

I realized today that, while I had written in the past about the idea of crowdmapping, I had never actually used that term in a post before.

That was a mistake.

Don't get me wrong, Group editable maps have been around for some time and are quite successful.  We have used CommunityWalk, for example in a number of projects and it has served its purpose excellently.

CommunityWalk Map - North Caucasus Violence Sep-06 to Nov-06



Likewise, automatically edited maps are also quite helpful.  The comprehensive map at RSOE EDIS, for example, just recently got some new competition with Google Public Alerts.

Crowdmapping, though, is something a bit different.  Here, dozens and sometimes hundreds of people are providing information from a variety of sources (including the web, of course, but also through SMS and Twitter) that are then mapped in real time.

Right now, this space is occupied almost exclusively by Crowdmap.com, an offshoot of the much admired Ushahidi project.  It is not too hard to see a time, however, when other companies and organizations will enter this space with competing offerings.

I, along with a small group of intrepid students, have been experimenting with this system for a few months and, while managing the input has proven to be more challenging than expected, the potential (and the relative sophistication of Crowdmap) is enormous.

The best way to get a sense of the value of a crowdmap, however, is to look at them.  Below are three of my favorites:  Syria Tracker (a map tracking eyewitness accounts of missing, killed or arrested people in Syria in English and Arabic), China Strikes (a map tracking instances of labor unrest in China), and Energy Shortage (a map tracking reports of energy related issues worldwide).  You can see all of these maps below (Syria Tracker is live; the other two maps need to be clicked on to get to the live versions).

IBM Creates Interactive Map/Infographic Of CIA World Factbook (IBM.com)

IBM, in order to demonstrate some of their latest web based technologies, has taken the data from the CIA's World Factbook and re-mixed it in the form of a stunning, interactive infographic.  

The final product allows the user to much more quickly engage and compare the data for the various countries in the world.  The screenshot to the right does not (as usual) do the product justice.  I have zoomed in on central Africa to show some of the detail but you can just as easily take a look at the whole world and can instantly get a sense of where various regions lie with respect to any of the data the World Factbook contains.  

I strongly recommend you go here to see the full product.  Play around with it; I think you will be impressed.

If you are interested in additional information about IBM's initiative, you can go to the cryptically named IBM ILOG Elixir Blog or to Information Aesthetics, where I first picked up on this product.

Note:  This has been a very good week for maps (See also here and here) ...

Intelligence Tools For Understanding Any Website (YouTube)

Jeffrey Kotvas on the always insightful Competitive Intelligence Forum points to the excellent video below that reviews various online tools for understanding the traffic patterns behind any website (sort of a modern and open-source version of the much more venerable intel sub-discipline of traffic analysis...).

The video, put together by Scrappybusiness.com, focuses on the use of these tools by intelligence professionals in the business world but these tools might also be of some use to amateur (and, perhaps, professional) cyberthreat analysts as well. 



For those of you not familiar with the Competitive Intelligence Forum, I recommend you check it out as well. There are lots of good places to talk about intelligence on the internet these days but the CI Forum is one of the best. Founded by longtime CI professional, Arik Johnson, this social networking site for those involved in intelligence work for the business community has really taken off and now boasts over 1300 members. Click on this link to preview the site.

Related Post: 
Using Search Engine Optimization (SEO) Tools To Do Intel Analysis

Chinese, Russian, Turkish Hackers Almost Certainly Targeting, Penetrating US Energy Provider Networks (Project Grey Goose)

Jeff Carr, author of Inside Cyber Warfare and IntelFusion, along with Sanjay Goel at the State University of New York, Albany and other contributors, has recently completed another of the Grey Goose reports, this time on hacker attacks on the power grid, both domestically and internationally.

The report's key findings are chilling:

• "State and/or Non-state actors from the Peoples Republic of China, the Russian Federation/Commonwealth of Independent States, and Turkey are almost certainly targeting and penetrating the networks of energy providers and other critical infrastructures in the U.S., Brazil, the Russian Federation, and the European Union."

• "Network attacks against the bulk power grid will almost certainly escalate steadily in frequency and sophistication over the next 12 months due in part to international emphasis among the G20 nations on Smart Grid research, collaborative development projects and the rich environment that creates for acts of cyber espionage"

• "The appeal of network intrusions against the U.S. Grid is enhanced by two key factors:"

• "90% of the U.S. Department of Defense's most critical assets are entirely dependent on the bulk power grid."
• "Most Grid asset owners and operators have been historically resistant to report cyber attacks against their networks as well as make the necessary investments to upgrade and secure their networks."

Grey Goose reports are volunteer efforts to analyze various cyber threats through the use of open source information. Previous reports have analyzed the Russia-Georgia cyber war and the evolution of cyber warfare.

In the interest of full disclosure: Jeff kindly listed me as a "reviewer" in the recent report but my input was limited to a little light editing. I don't consider myself a cyber war expert. I do think, however, that Jeff's record and the records of his co-contributors' speak for themselves and believe that those interested in this area (and those who should be interested in this area) need to read this report carefully (whether you ultimately agree with its conclusions or not).

Sabotaging The System, The Cyberwar Plan, The PRC's Cyberwar Capability and Trillions (Cyberwar Link List)

Cyber stuff is everywhere these days and the last couple of weeks have seen a number of interesting articles and videos make the rounds:

• Sabotaging The System. I would start the tour of this particular horizon with the recent 60 Minutes report on the cyberthreat. Readers should note that, in addition to the video below, there is quite a bit of additional material on the website as well.
  

Watch CBS News Videos Online

• The Cyberwar Plan. The National Journal recently posted an online article that discusses in some detail the US's own offensive cyberwar capabilities. Whether you agree with Jeff Carr or not over the appropriateness of this article, it is the most comprehensive piece I have seen on what is likely an open secret among the hackers of the world and those that sponsor them.
• Capability of the People’s Republic of China to Conduct Cyber Warfare and Computer Network Exploitation. This research report actually came out last month but it is one of those kinds of reports that, if you haven't seen it, you should. It is very solid open source analysis by the good people at Northrop-Grumman for the US-China Economic and Security Review Commission.
• Trillions (via Gizmodo). This one is probably my favorite entry on the list. It is not directly related to cyberwar but it provides an interesting perspective about where all this is going to go -- is going to have to go -- before too long.

Trillions from MAYAnMAYA on Vimeo.

Hey, Admiral Blair! Did You Get Tetlock's Memo? (NationalInterest.org)

Philip Tetlock's Expert Political Judgment is required reading here at Mercyhurst. While some analysts take issue with Tetlock's findings, I have yet to see anyone create as compelling a study as his regarding the essential qualities of a good forecaster.

So, I pay attention whenever Tetlock writes something, even a book review. Thanks to my colleague, Steve Marrin (who knows of my interest in all things Tetlockian...), I recently had a chance to read Tetlock's review of three new books by people who claim to have an inside track when it comes to predicting the future.

I will let you read Tetlock's review of the books on your own. Suffice it to say that his critique of all three works is based on his research and, as a result, is skeptical (in varying degrees) of the claims of accuracy in the three books.

What really fascinated me about this review is buried at the end of it, though. Tetlock worries about how to improve forecasting, about how we can know which forecasters are worth listening to and which are modern day snake oil salesmen. Then, he makes a point that I agree with in whole:

• "There is one potential savior on the horizon: a big institutional purchaser of forecasting services that has the financial clout and technical-support staff ready to run forecasting tournaments that would shed light on the relative performance of competing approaches—a big player that also has powerful incentives to discover superior analytical strategies, for even small improvements in its prediction accuracy can translate into billions of dollars and millions of lives saved. And that player is the Office of the Director of National Intelligence."

Right on, Professor T! The DNI has every reason to want to improve forecasting and has the purchasing power to insist upon it. Specifically, according to Tetlock:

• "Players high up in the political system—who really do want the best-possible forecasts—could decide that it is worth investing a nontrivial share of their intelligence agencies’ budgets into a series of long-term forecasting tournaments designed to distinguish the more from the less promising forecasting approaches across policy problems."

The DNI dipped its toes in this water back in 2008 when it sponsored the Open Source Challenge. Since then, of course, we have not had another Challenge or another Open Source Conference. This does not bode well for Tetlock's very reasonable suggestion.

Still, it is a great idea. If we are ever to break out of the "two-thirds right" trap we are in, we need a robust research program aimed at validating intelligence methods and the DNI will have to be the one to sponsor it.

Intel Legend Art Hulnick On The Future Of OSINT (ISN)

The ISN, on one of its recent podcasts, scored a very good (if too short) interview with Professor Arthur Hulnick (see picture at right). Art currently teaches intelligence related courses at Boston University and has for a number of years. Before that he was at the CIA for several decades and has contributed significantly to the open body of literature on intelligence through his books (including Keeping Us Safe: Secret Intelligence and Homeland Security (2004) and Fixing the Spy Machine: Preparing American Intelligence for the 21st Century (1999)) and many articles.

Art is one of those guys who has been around, as we used to say in the army, "since Christ was a corporal" and is always worth listening to. Unfortunately, the ISN did not make an embeddable version of the podcast but you can get it on iTunes, download the MP3 or just go to the ISN site to listen to it.

"Quarantine, shmarantine! Let's send MORE flights into the infected areas!" (My Linux Experiment)

(Note: First, I want to welcome all the Linux lovers to SAM. I had heard that the Ubuntu and the broader Linux communities were pretty good people but wow! I have received a number of very nice emails and even some telephone calls offering both support and help and the referral traffic from sites like Ubuntu-news.net, Tuxmachines.org and Free Software Daily has been impressive. Thanks to all!)

A couple of days ago, I talked about adopting a Linux distribution, Ubuntu, for one of my old laptops. I had a number of personal reasons for adopting this open source operating system but one of my explicit reasons for doing so was in order to become more resilient.

Resiliency, as I use the term here, is about being able to withstand bad times. Microsoft products, because of their popularity, are the primary targets for state and non-state sponsored hackers. One day, the bad guys are going to win and win big. This victory may be only temporary and the perpetrators may pay dearly for it in the long run but do I really want to be just another victim? Having (and knowing how to use) a Linux machine makes sense in this context.

One of the pieces of evidence I pointed to in that article was Jeff Carr's (of Intel Fusion) characterization of Africa as in the midst of a cyber pandemic. Most of the machines there are infected with viruses or are part of zombie networks (or both) due to pirated Windows software and a wholesale lack of anti virus protection.

Associated with Jeff's article is a pretty neat map of the projected level of connectivity (via undersea fiber optic cables) in Africa by 2011. This map doesn't show the explosive growth of the "big pipes" -- the undersea cables that carry most of the internet's traffic -- around the globe, however. To get this picture, you need to go to the BBC (Note: The pic below is just a screenshot. The full map is interactive and shows growth over time).


For me, this explosive growth (which is unlikely to stop in 2011) is the epidemiological equivalent of increasing the number of transmission vectors from an infected area instead of quarantining it.

That said, I am not sure what the answer to the problem is. I consider it almost inhumane to deny these parts of the world the benefits that robust internet and communication facilities provide. Likewise, I don't think you are ever going to see a company (such as Microsoft) take a "responsible" position that is fundamentally contrary to its shareholder's interests. Government control sounds even less palatable (though the government, after a hue here and a cry there, seems to now be taking the risk seriously).

Which sort of leaves just us. We have to act in our own best interests and, for me, at least, that means not putting all your ova into one open-top, woven reed container. So, I converted one of my machines to a Linux machine.

Having used my Ubuntu/Linux machine all week, what have I learned? First, it boots way faster than Windows did. Everything is quicker, snappier now. Second, all of the apps that come pre-loaded with Ubuntu mimic or improve on similar Windows-based apps. Third, most things are the same or about the same. Firefox, for example, works the same. Some of the drop down menus are in different places but I strongly suspect you can move them around if you don't like them where they are. I consider the fact that the user experience is similar in many ways to Windows to be a huge plus, by the way. The last thing I wanted to do was to have to learn a whole new workflow. The similarity in experience also allows me, I think, to really appreciate where Ubuntu is better than Windows, as well.

I also ran into my first snag this week. I went to download a piece of freeware only to realize that there was no Linux version of it. This led me to do a little research and was quickly able to find that the capability was already built into Ubuntu. I am pretty sure that all my surprises will not be that pleasant but that was pretty cool.

There are also things that I don't yet understand. System maintenance, for example. I am used to a whole series of activities (defragging the hard drive, clearing the temp files, etc.) to keep my Windows machine operating at peak efficiency. Do you not have to do this with Linux? I don't know. It is too early to be worried about too much of this stuff but at some point in the future, I suspect that I am going to have to figure it out.

In short, so far, so good.

Ubuntu Is Pretty Cool (My Linux Experiment)

I decided to install Ubuntu on an old laptop of mine this weekend and I feel, I have to say, more, well, resilient already. The intelligence implications are pretty interesting, too.

For those of you unfamiliar with Ubuntu, it is a Linux-based operating system (For those of you who are also unfamiliar with Linux, it is a catch-all term for a wide variety of operating systems based off a common, open source core. Linux-based operating systems are alternatives to operating systems offered by Microsoft (Windows) or Apple (Mac OS X). (For those of you unfamiliar with the term "operating system", you need to join the rest of us here in the 21st Century...)).

Now Linux is much loved by the technically proficient but not so much by the rest of us. As you can see in the pie chart, Linux has not quite captured a whopping one percent of the operating system market.

What makes it worse is that Linux has more flavors than Baskin-Robbins. Because the core of Linux (the so-called "kernel") is open source, anyone with the technical skill to do so can make a Linux variant (called a "distribution" in Linux-speak). So, the one percent? It is actually divided up among 50 or so different distributions and Ubuntu is just one of them.

Ubuntu is, however, one of the most popular and best supported of the Linux distributions. Because it is free and focuses on usability, it is often the first choice for newbies like me.

But why choose Linux at all? Here are my reasons (in no particular order):

• I was curious. Trying out new tech widgets and gadgets is something I do for fun. I have been toying around with Linux for years now (using live CDs mostly) and had an opportunity to try it out so I decided, "What the heck?"

• I had an old Windows XP laptop that was slow and required constant attention. One of the great things about almost any Linux distribution is that is small and efficient. It is often recommended as a good way to get some new life out of an older machine.

• Ubuntu makes it easy. I picked the Ubuntu distribution because it was easy to figure out and install. The software takes you step by step through the process and even gives you the option to split your hard drive so you can have both Linux and Windows (or whatever) on the same machine.

• I am not sacrificing much (if anything). As the title to this post suggests, Ubuntu is pretty cool. True, the user interface is a little different but, having oriented myself (and pretty quickly for an old guy, I am proud to say), it seems a little better than Windows. It does well all of the things my old Windows machine did poorly. I have faster web-browsing now through my trusty Firefox browser. Web apps (like Google Docs) are operating system agnostic and I have yet to run into a major plugin that is not also available for Linux distributions. Open Office (a free Office-like application) works very well with most of my Office files (and others). There are also tons of new productivity and gaming applications to explore as well, all with little (some would say no) risk of virus or malware infection.
  

Finally, and most interestingly, it makes me more resilient (Here is where the intelligence implications come in). Centralized networks attract attention. On the positive side (at least from the standpoint of those that control the network), the "rich get richer", meaning the most powerful node attracts other nodes to it. This is great if you have a product that dominates the market the way Microsoft does with Windows. On the other hand, it also attracts negative attention as well. One of the major reasons hackers go after Windows-based systems so much is because so many people use it.

Machiavelli first outlined the problems with centralized networks in The Prince (Don't believe me? See Chapter 4...). Good ones are difficult to take down but once taken down, they are easy to control because of the efficiencies inherent in the centralized system. Decentralized networks, on the other hand, are very difficult to take down but are also very difficult to control.

There ought to be (and, in fact, there is) an optimal balance between efficiency and robustness in any system. To me, a resilient system ought be closer to this optimal balance than not. I am not a Windows hater and will likely continue to use Windows. That said, I feel better knowing, understanding and owning a Linux-based system as well. A "black swan" event like a zero-day virus that wipes every Windows-based computer is pretty unlikely but if it does, I will still have a computer and (maybe) internet access (many servers run on some form of Linux already).

I am no cyber analyst and do not pretend to know the ins and outs of the subject matter. I am not the only person to note the problems with an over dependence on Windows, however. Fellow blogger, Jeff Carr, over at IntelFusion notes that Africa is in the midst of a "cyber pandemic" due primarily to an over-dependence on pirated versions of Windows.

In the end, understanding something about Linux, what it can and cannot do, seemed to make some sense -- an experiment in resiliency. As I proceed, my intent is to report here periodically about what I find. Your comments and questions about both the process and my findings are, as always, welcome.

Note: Two recent authors, John Robb and Joshua Cooper Ramo, have both written more extensively (and more eloquently) about this concept of resiliency for anyone interested.

Another Russia-Georgia Conflict Brewing? (Multiple Sources)

The Russians are objecting to a series of low-scale NATO Exercises scheduled to begin in early May at an airbase outside of Tbilisi, Georgia.

Such diplomatic maneuvering would be within normal limits if it weren't for the disturbing news regarding the forward deployment of troops, tanks and artillery reported yesterday by Reuters and the sortie of the Black Sea Fleet from Sevastopol reported yesterday by the armchair admirals over at the Information Dissemination blog.

Add to this the possibility of Russian funding for the protests against Georgian President Saakashvili that began last week and a worrisome pattern begins to emerge. In fact, the Jamestown Foundation puts these exact pieces together in a recent report while the Caucasus Analytic Digest claims that NATO doesn't have the stomach for a fight in Georgia.

Georgia's Foreign Minister, on the other hand, thinks nothing will happen and even decided to poke the bear a bit: "...Russia would be afraid to undertake a new military aggression against Georgia because it would be entering in confrontation with the rest of the civilized world." (Hey, buddy, are we talking about the same Russia?)

Yoikes!

I don't have a background in this area and so am unqualified to comment on the news or the reliability of the sources. On the off-chance that this region of the world is going to heat up in the next couple of weeks, though, I thought I would put together some good open sources to help new analysts get started:

History/Future of the Conflict:
Wikipedia on the 2008 South Ossetia War. Very detailed, comprehensively sourced with a truly outstanding map.
NATO After the Georgian Conflict. A recent Polish Institute Of International Affairs Study (in English).

The Black Sea Fleet and the Russian Military:
The unofficial Black Sea Fleet Website. Good pictures, info and history.
Wikipedia on the Black Sea Fleet. Amazingly comprehensive site.
Warfare.RU. I did not spend much time on this site but I found what I did see to be pretty good stuff on the Russian military. Tons of pictures. The guys who write this site are also referenced in Google Earth through the Google Earth Community (all of the "i's" in the image of the Sevastopol navy base to the left).

News and Other General Information Sources
Information Dissemination. These guys are watching the naval part of this and will likely have some good armchair analysis if it progresses.
UN Observer Mission In Georgia. Probably the best one stop shopping place for current, detailed news in English from inside Georgia.
EU Monitoring Mission In Georgia. Not as current as the UN site but has some good maps and background data.
ReliefWeb on Georgia. ReliefWeb is a UN effort of consolidate news and info from a number of sources into a single place for NGOs. It usually has the best, most current open source maps available as well.
Reuters AlertNet. Reuters has done a good job of consolidating news and background info into a single site.
The Institute Of War And Peace Reporting, the International Crisis Group and the International Relations and Security Network. All three organizations maintain special sections on the crisis in Georgia.

Do you have another source of interest? Drop it in the comments...

How To Detect Deception, Using Second Life In The Classroom And Sister Wikis (Link List)

Image by a4gpa via Flickr

Just back from the high desert and busy catching up. Here are a few links that caught my eye:

• How To Detect Deception. The Confidential Resource points to this series of posts (Part 1, Part 2) on detecting deception in interviews over at The Keyboard Detective. The posts contain some good material and a fair number of links to other interesting stuff.
• Second Life In Education. I have often blogged about using Wikispaces for analytic projects but Wikispaces is a platform to over a half a million wikis of various types. One of the more interesting ones I recently ran across (via New World Notes) is a wiki on using Second Life In Education. Lots of cool projects you can do/places you can go here with students.
• 8 Sister Wikis From Wikipedia. The always helpful MakeUseOf.com has put together a list of "sister" projects to Wikipedia that everyone should know about. My favorites are the Wikimedia Commons, because it links to a number of free or public domain graphics, and Wikiversity, because it often has useful tutorials on a variety of topics.