Reviewers Needed For The Visual Analytics Science And Technology (VAST) 2014 Challenge!

The Visual Analytics Community and their member organizations (including the Department of Homeland Security, Pacific Northwest National Laboratories and the Defense Threat Reduction Agency), in coordination with the IEEE, sponsors a visual analytics challenge each year at the IEEE conference for students and researchers.  

In order to judge the output from the participants, the challenge organizers asks for analysts to participate as reviewers of the submissions.  Kris Cook, who is on the contest committee, has asked me to put the word out that the contest needs reviewers for this year’s challenge.  

This is an unpaid, all volunteer effort to assist a non-profit sponsored contest.  Kris’ note to me is reproduced below with additional links.  If you are interested in participating or have any additional questions, please contact her directly.  

For what it is worth, taking a look at the VAST entries is a very interesting and rewarding way to learn what is happening in the world of visual analytics.

Begin text of note:

We invite you to be a reviewer for this year’s IEEE Visual Analytics Science and Technology (VAST) Challenge.  The VAST Challenge poses interesting problems that contestants solve using visualization-based software tools that support analysis of complex data sets.

We are soliciting reviewers for three mini-challenges and a grand challenge this year. 

• Mini-Challenge 1 challenges participants to identify the current organization of a fictitious terrorist organization and how that organization has changed over time, as well as to characterize the events surrounding the disappearance of multiple people.  Participants will use visual analytics to analyze the specified data.
• Mini-Challenge 2 challenges participants to describe the daily routines of employees of a fictitious company and to identify suspicious behaviors. This task focuses on the analysis of movement and tracking data and is thus primarily a spatiotemporal challenge.
• Mini-Challenge 3 challenges participants to identify a timeline of significant events in a fictitious city and identify important participants, locations, and durations by monitoring real-time data feeds. This task poses a streaming analysis challenge.
• The Grand Challenge asks participants to synthesize the discoveries made across the three mini-challenges to form a high level description of the entire scenario. This task focuses on the identification of who disappeared, who was responsible, and the underlying motivations. Significant information gaps will also be addressed by the participants.

More specific information about the tasks may be found at http://vacommunity.org/VASTChallenge2014.

As a reviewer you will be responsible for reading 3-4 submissions and providing written feedback for the committee and the submitters. Each submission consists of an entry form describing the submitter’s software, their technical approach, and their answers to the mini-challenge questions, as well as a short video showing an example of the analytic processes used by the submitters. 

This year, the reviewing period is as follows:  Entries will be available for review by July 12.  Your reviews will be due by July 28.

All review materials will be accessible over the internet. Reviews will be conducted using the Precision Conference web-based reviewing system. Reviewers will be registered in the Precision Conference system and will submit their reviews using Precision Conference web pages.

If you are interested in reviewing please respond to vast_challenge@ieeevis.org no later than July 1.  Please indicate which mini-challenges you would be most interested in reviewing and how many entries you are willing to review. 

Thank you for your time and consideration!

VAST Challenge Committee

Kris Cook, Georges Grinstein, and Mark Whiting, co-chairs

Orienting The Intelligence Requirement: The Zooming Technique

In a world where infographics present everything from the world's biggest data breaches to the history of music media to beer varietals, its clear that data visualization has become kind of a big deal. 

But that's not what I want to talk about.

How about an interactive map presenting the age of every building in The Netherlands... or a flavor connection graphic for the more culinarily-inclined... what about a language map of New York City based on recent Tweets...

Nope.  

What about Chaomei Chen, a leading authority on information visualization, and her countless contributions to the field of functional infosthetics (see her paper on the top 10 unsolved problems of information visualization)?

Not even that.

For me, even more interesting than how to present information visually, is how to visualize information mentally.

All too often, analysts are given an intelligence requirement which they attempt to answer by searching for a specific answer or number or outcome, what I call working downward. What they forget to do is to take a more macro approach, to zoom out, if you will, and put the question into a broader context - to orient the intelligence requirement in its problem space. 

Figure 1: Working downward

Interesting.  How does this work? 

For this, I turn to Enrico Fermi... 

Fermi was a physicist who worked on the Manhattan Project.  He loved to ask his students to try to solve seemingly intractable problems using only thought experiments.  For example, Fermi would ask, "What is your best estimate for the number of piano tuners in Chicago?" Don't run to the Bureau of Labor Statistics website... as a matter of fact, don't even use a computer. 

These broad questions, known as Fermi Questions, are difficult or tedious to answer by only working downward (quick! pull out your phone book, flip to 'musical instruments' and start counting!), and almost impossible to answer without access to the internet (See Figure 1). But by zooming out, orienting yourself within the problem's scope and starting more broadly, you can actually drill down to a reasonable estimate of the number of piano tuners in Chicago in a matter of minutes. Here's how:

• Start with the approximate population of Chicago, 3 million. 
• On average, there are four people in a family, meaning that there are approximately 750,000 families in Chicago (3 million / 4) (See how general we are being?)
• Not every family owns a piano, though. A high estimate could be 1 in 5, which means that there are approximately 150,000 pianos in Chicago that need tuning while a low estimate might be 1 in 50 which would mean there are only 15,000 pianos in Chicago.   
• If each piano tuner works on 4 pianos per weekday, the average piano tuner will work on approximately 100-1,000 pianos per year. 
• Pianos need to be tuned about once a year, therefore there should be approximately 15 to 150 piano tuners in Chicago. 

Now, I have no idea how many piano tuners there are in Chicago (Won't sleep tonight unless you know? Go to Wolfram Alpha and find out.  Once you realize that not all instrument repair people are also piano tuners, you will see that the Fermi estimate is pretty darn good!), but you can see where taking a more broad approach produced at least a viable estimate. 

This rough order of magnitude estimate is actually very useful.  First, if that is all we need, then we are done.  No money, little time, and just a bit of thought and we have an answer that fits our needs.  

Second, it helps us identify potentially wrong answers more easily.  Say we really need to know and we send someone out to collect this information and they come back as say 10,000 piano tuners work in Chicago.  Our Fermi estimate should cause us to question that number and the methods used to derive it.  

Finally, it allows us to know where we can get the biggest bang for the buck in terms of collecting additional information.  For example, we can get a more precise estimate of the number of people who live in Chicago but unless we are off by millions, it probably wont make much of a difference.  Getting more information on the number of pianos sold and to whom, might, on the other hand, really help our estimate.

Figure 2: Working on a sliding scale

This technique may seem trivial when we are talking about piano tuners but it comes in real handy when we are trying to get a rough estimate on "unknowable" numbers such as how many Taliban there are in Paktia Province or how many spies there are in the US government.

Figure 2 presents the mental image of this problem. Think of this mental image as the order of magnitude scale for any intelligence requirement (or any problem, for that matter). The red line is the starting point, or where the problem is oriented within the problem space. See how much you would miss by only working downward?

The first (and most important) step is visualizing where your problem is in the problem space, thereby determining how much you are able to zoom in or zoom out. This oftentimes helps analysts put the problem into both perspective and context. (If you have an MBA, this technique might seem similar to PESTLE or STEEP  - analytic methods designed to analyze the macro environment to put a smaller intelligence requirement into context). 

Second, it is important to zoom out and think to yourself what is "above" your problem, what is the next step up, conceptually from your problem?  For example, if you are analyzing a specific company, you will want to step back and look at the industry as well.  This zooming out exercise is also very useful at helping you spot assumptions you are making about your target.  For example, it is very easy to make a number of assumptions about the dictatorship in North Korea.  Stepping back and looking at China's interests in the region, however, adds a whole new level of nuance.

Finally, Fermi estimates are most helpful at the beginning of an analytic process, when you don't have lots of information, or when gathering the detailed information is expensive and time consuming.  But be careful!  Fermi estimates are just that - rough order of magnitude estimates that help you orient yourself and focus your collection and analysis activities.  If the situation warrants, more detailed estimates based on additional information may be required.

Independent of the intelligence requirement, the zooming technique is a beneficial way to visualize a problem space, identify information gaps, contextualize information, recognize assumptions and, above all, approximate and approximate quickly, a skill highly relevant to an intel analyst in any field.

Powerful Visual Analytics Tool, Jigsaw, Now Available Free For Download!

John Stasko, and the programming geniuses at Georgia Tech, have finally released their powerful and recently updated visual analysis software, Jigsaw, into the wild.

For those of you who just want the software now, click here to go to John's newly updated website.

It has been a while since I have looked at Jigsaw.  I examined it first back in 2007 (here and here) and we last used it in a project in 2009 (here).  Back then, it was an extremely good product for a fairly narrowly focused type of analytic work.

Today, though, it appears to be a much easier to use and more general purpose type of analytic software.  I was especially blown away by the use cases demonstrated in the video below.  The video is a little long but worth it if you don't know a lot about the state of the art in visual analytics.

I am also particularly impressed with the growing list of publications assessing the software's validity and the video tutorials to help new users get started.

If you are interested in new ideas and new tools for visualizing the results of your analysis -- particularly free ones -- I strongly recommend you check Jigsaw out!

Can't Be Both: Visually Displaying Inconsistencies

I like finding inconsistencies. It means that something is wrong with my argument. Inevitably, addressing the inconsistency leads to greater nuance in my analysis. That, I would argue, is a good thing.

One of my most popular recent posts (see the "Top Posts" box to the right) was on form and content and what intelligence products say by the way they look.

Today, I saw this infographic (via FlowingData, by way of the Consumerist, and originating at The Physicians Committee For Responsible Medicine) and it started me thinking along those same lines again:


Assuming the data is accurate (and I have no reason to suspect it), it does a good job of pointing out the inconsistency between what the government says we should eat and what foods the government actually funds (through subsidies).

Understanding how to display these kinds of inconsistencies is important to intel analysts, too. For example, one of the most common inconsistencies leveled against the intelligence community is that it is both all powerful and incompetent. It can't be both (obviously) but how can you capture this inconsistency in a graphic? See my graphic design-challenged attempt below:
I simply used Google to search for the phrase "CIA is all powerful" and "CIA is incompetent" and then made the graph based on the number of hits the two phrases received. It is not rocket science and it is definitely not a very good graphic but it makes the point, I think. Furthermore, I can imagine a much more complex graphic muddying the waters

• By the way, ADM Blair will be glad to hear, I suspect, that the phrase "DNI is incompetent" yields no hits...Of course, neither does the phrase "DNI is all powerful".

• Note, too, the powerful new metric I have created: The I-TOT -- the Intelligence is Taking Over The-world index (Hat tip to NGA for the idea).

Lesson learned? Simplicity wins. Simple graphics communicate big inconsistencies in analytic findings best. This is probably pretty obvious to a graphic designer but, for me, it was a bit of an insight.

Of Form And Content (An Experiment In Communicating The Results Of Analysis)

Embed-O-Matic

• 1oTs10OK5Yg&hl=en_US&fs=1&color1=0x5d1719&color2=0xcd311b
• uWHn5dxOUY4&hl=en_US&fs=1&color1=0x234900&color2=0x4e9e00
• g9M1gbi4eQI%2Em4v
• MlCvJOSyRq4&hl=en_US&fs=1&color1=0x234900&color2=0x4e9e00

Embed-O-Matic

• 1oTs10OK5Yg&hl=en_US&fs=1&color1=0x5d1719&color2=0xcd311b
• uWHn5dxOUY4&hl=en_US&fs=1&color1=0x234900&color2=0x4e9e00
• g9M1gbi4eQI%2Em4v
• MlCvJOSyRq4&hl=en_US&fs=1&color1=0x234900&color2=0x4e9e00

Imagine a brilliant piece of intelligence analysis -- well-researched, well-written and actionable. Now imagine that same report written in an 8 point Gothic font over multiple pages with half inch margins. No title, no paragraphs, no sub-sections, no indentations; just a single block of text. Would you read it? Would anyone else?

Point 1: Form matters. How we say something is often as important, if not more important, than what we say.
__________________

Now, take a look at this video:



It is a fake. It was originally created with some off the shelf software by a CGI artist and then modified by someone to look like a NASA video. Here is the original:



You have to go to YouTube to see the dates (the original was loaded in February 2009 and the fake modified and uploaded in November, 2009).

The most distressing thing about the two videos, however, is not the fakery. It is the number of views. Again, you have to go to the YouTube sites to confirm this but the original has only 23,000 or so views while the fake has over 150,000 views.

Furthermore, cleverly modified videos are not the only way to twist, spin, modify and deceive. Check out FactCheck.org's Whoppers of 2009 for other ways that people have cleverly manipulated the form of the message to lie to us.

Which leads to Point 2: It is getting easier and easier to lie with form.
___________________

Richards Heuer pointed out in his classic, The Psychology of Intelligence Analysis, that "once information rings a bell, the bell cannot be unrung." He was capturing a phenomena that is well known to psychologists: People continue to act as if a piece of information were true even after the piece of information has been proven to be false.

Over and over again, people have been put in experiments that make them falsely believe that they have a capacity to do something -- distinguish the effect of risk-taking and success as a firefighter, for example -- that they do not have. Even after they have been shown conclusive proof that the experiment has been manipulated to give the subjects the impression that they have an ability they do not, in fact, have, these subjects continue to act as if the original information were correct.

This persistence of the impression of accuracy, even in the face of overwhelming evidence to the contrary, has recently been shown to exist in more realistic experiments that include statements made by politicians.

All that is bad enough but when you combine this psychological effect with the power of visualization, you get an absolutely scary combination. Check this video out:



Which leads to Point 3: Lies persist and visual lies likely persist more strongly than textual lies.
______________________

So what does this all have to do with communicating the results of intelligence analysis?

The US national security intelligence community has been accused of trying to sell its intelligence. The 2005 WMD Commission report accused the intelligence community of this with regards to the President's Daily Brief (PDB): "The daily reports seemed to be ‘selling’ intelligence—in order to keep its customers, or at least the First Customer, interested."

Luis Garicano and Richard Posner suggested that there were self-interested economic reasons for doing so, stating in 2005 that the PDB "has become the primary platform by which intelligence agencies seek to advertise their products in competition with each other..."

Which leads to Point 4: Good intelligence doesn't "sell" its products.
__________________________

When I took my first job as an analyst (back in the 80's...), I didn't make my own slides. PowerPoint was deemed to be too complicated and tricky. It required a specialist, trained in its vagaries, to generate the slides necessary to brief the decisionmakers who pulled my strings.

That did not last long. Very quickly it went from rare to common to expected that analysts would be able to generate their own slides. What's more, today analysts are increasingly being asked to create visuals to supplement or replace the results of what was previously text-based analysis.

Yet, analysts get very little training in appropriate ways to visualize information and virtually no training in how not to lie or mislead with colors and graphics, how to spot photoshopped pictures or fake video, or how to ensure that the form is as objective as the content.

Which leads to My Question: How do we know when we are lying (or misleading) with the form of our intelligence products?
________________________

It seems to me that we spend a good bit of time analyzing text for evidence of bias or puffery or misleading statements. In virtually every intelligence organization of any size, there is a quality control process to ensure that the content -- the words going out the door -- conform to the standards of the agency.

Within the US national security intelligence community these standards are laid out in ICD 203 and I suspect that other intelligence agencies and organizations worldwide have something similar.

But who makes sure the same thing is true for the form?

All of this is a very long precis to an exercise I do in my Intelligence Communications class. In the vast majority of the exercises and assignments in that class, I ask students to focus on the elements of good intelligence communication: Bottom-line up front estimates, concision, clarity, decisionmaker focus, accuracy, etc.

In one exercise, though, I ask them to take a written report and re-imagine it as a primarily visual product. I task them to keep all the elements of a good intelligence product but to visualize those elements rather than put them in print.

Over the years, I have received some wonderfully innovative products. This year was no different. One of the products stood out, however. Nimalan Paul, using online software from Xtranormal.com, created an amusing and compelling animated video that contains virtually exactly the same content as the written product on the same topic.

Before you see the video, I will share the written version of the report with you. It follows the generic form guidance that we use here at Mercyhurst in our intelligence communications classes for written products:

Nimalan Paul, Intelligence Communications 14 January 2010

Here is the animated version of the same report:



Which report is better at communicating the results of the analysis? One of our grad students actually did a study on this a number of years ago. His findings showed that if you are above a "certain age", the text document is the best at communicating but that if you are below that certain age, then the animation is likely to be more effective.

Beyond the age distinction, what else makes one format better than the other? Is it all personal preference? Is one more "honest" than the other or is one just more traditional?

Finally, if one of these forms is more honest than the other, shouldn't we be teaching how to recognize that difference?

Leave your thoughts in the comments...

Yes, Virginia, The Internet Is Big! (Lifehacker)

In case you were wondering...

Click on the image for the full view.

Via Lifehacker.

When Does Web 2.0 Work? Lessons For The Intelligence Communities (McKinsey Quarterly)

McKinsey, the capo di tutti capi of consulting firms, recently published a fascinating report titled How Companies Are Benefiting From Web 2.0. You have to register with McKinsey to read the full text but it is probably worth it if you are interested in how (and what) Web 2.0 technologies are actually making a difference in the very competitive, global business environment -- and, of course, which technologies appear to be falling out of favor as well.

The coolest thing about the report is the visualization tool they developed to supplement their report. I have a screenshot of one of the views of the data it provides below but that does not do it justice. Click here or on the picture to take you to the fully interactive set of charts and graphs (No registration required to play with the chart...).


The most interesting thing about the report, however, is the implications it holds for the intelligence community and its attempt to bring Web 2.0 technologies into the workplace. According to a report from earlier this year, Web 2.0 is in a midlife crisis within the national security intelligence community. The McKinsey report pretty clearly points to the likely reasons why. Specifically, they identified three major performance factors (ranked by the percentage that each factor made in the average company's success):

• "Management capabilities ranked highest at 54 percent, meaning that good management is more than half of the battle in ensuring satisfaction with Web 2.0, a high rate of adoption, and widespread use of the tools. The competitive environment explained 28 percent, size and location 17 percent."

Since management was such a fundamental part of the success or failure of these initiatives, McKinsey then dug into the numbers regarding management and found three critical management related sub-factors:

• "Parsing these results even further, we found that three aspects of management were particularly critical to superior performance: a lack of internal barriers to Web 2.0, a culture favoring open collaboration (a factor confirmed in the 2009 survey), and early adoption of Web 2.0 technologies."

Yoikes!

If McKinsey's results are accurate, then a true cynic would say the national security intel community already has three strikes against it. In these circumstances, it is only surprising that Web 2.0 has had any success -- at all.

That view is clearly unfair to the thousands of people who are already successfully working with these technologies inside the national security intelligence community. What would also be unfair, however, is to underestimate the roadblocks that conventional management approaches may be putting in the way of the productivity to be gained from implementing these technologies in intelligence.

Top 100 Tools For Learning, Top 15 Visualization Tools And A Heckuva Video Search Engine Disguised As A File Converter (Link List)

Here is a quick link list of some interesting tools for use in the classroom and on projects:

Top 100 Tools For Learning -- I am a big fan of Jane Hart and her blog, Jane's E-learning Pick of The Day. At the link above, she has aggregated a list of 100 very interesting online tools to support a variety of learning activities. Definitely worth the look.

15 Effective Tools For Visual Knowledge Management -- Eric Blue's Blog has done a very nice job of compiling a number of tools (some free, some not; some online, some for download) to help you organize and visualize knowledge. I picked this up by way of the always insightful Outil Froids (Cold Tools) -- another blog worth adding to your RSS feed.

Excellent Video Search Engine Disguised As A File Converter -- Free FLV Converter describes itself as a way to easily convert online video (FLV) files to AVI, MPEG4, etc. files. This little free download does way more than that, though. You can actualy search through YouTube, DailyMotion, Google Videos, Metacafe, Spike, iFilm, Veoh, Yahoo, Crackle, Broadcaster, ZippyVideos, MySpace, Revver, LiveLeak, EyeSpot, CollegeHumor, SevenLoad, Break, CrunchyRoll, Blip.tv, lulu.tv, archive.org, TinyPic, EbaumsWorld, Uncut AOL, Guba, Youku, Tudou, Reuters, BBC News and vh1 for online videos of interest. Enormously useful.

Visual Analysis Everywhere! (Original Research And NVAC.pnl.gov)

I have finally gotten over enough of my jet lag to get back to work and it seems like everywhere I turn there is something interesting cropping up on visual analysis.

Two projects, in particular, recently crossed my desk, one by a student in my Advanced Analytic Techniques class, Andrew Canfield, and the other from the US's premier resource on visual analysis, the National Visualization and Analytics Center at Pacific Northwest National Laboratories.

Visual analysis allows people to more easily identify patterns in the massive data flows that are pervasive today. There are a wide variety of tools available for this purpose but Andrew focused on one, Starlight, to conduct his preliminary analysis (which he conveniently documented in the YouTube video below) of Turkish foreign policy with respect to the Caucasus.



While subject matter experts on Turkey might disagree with Andrew's conclusions, the real intent of his efforts was to experiment with the method. I think his video gives a fine introduction to some of the benefits of visual analysis.

In addition to Andrew's work, the NVAC also recently published their quarterly magazine on visual analytics, called VAC Views. Under the heading of full disclosure, I suppose I have to mention that Mercyhurst gets a nice write-up (on page 19 for those of you keeping score at home...). Beyond the shameless self-promotion, however, there are a number of articles that are quite interesting, not the least of which is NVAC's taxonomy of visual analytics. Previous issues of VAC Views contain other, similar, gems.

The rest of the NVAC site is also worth checking out. It contains a wide variety of videos and other resources including the invaluable Illuminating The Path e-book. Much more than the research agenda it describes itself as, Illuminating The Path is an essential background reference for anyone interested in visual analysis.

Visual Short Form Analytic Reports And Resources For Producing Them (Teaching Intelligence/Link List)

One of the exercises we routinely assign in our Intelligence Communications and Intelligence Writing And Presentation classes is a "visual" short form analytic report. The task here is to translate a primarily text-based report (in this case the classic "one-pager" or SFAR as it is known around here) into another primarily non-text-based medium (video, audio, interpretative dance...).

The fundamental intent of the exercise is to engage students' creativity and to get them to think about other -- primarily visual -- ways to produce an intelligence report. Decisionmakers have always wanted their intelligence packaged in ways that were most accessible to them (consider Kennedy's PICL or Reagan's videos of foreign leaders). In the future, however, they are going to expect it.

• For more on this, see Daniel Pink's A Whole New Mind. For examples of how good design makes a real difference, check out some of the projects at the Art Center of LA.

It is our belief that analysts need to think about how to do this in a way that maintains the standards of a good, text-based intelligence product. It is fairly easy to abandon traditonal notions of objectivity and transparency, for example, in favor of a intelligence product that "sells" itself well when using some of these tools and techniques.

Why should analysts have to think about this? Don't most intel agencies have personnel tasked to do this kind of stuff? Maybe, but how many analysts today are required to make their own PowerPoint slides? Back when PowerPoint was new, a specialist made the slides and we analysts fed them the information. Today, everyone makes slides -- and many of them are awful. Grappling with these issues seems appropriate for students trying to better understand how to communicate the results of their analysis to the decisionmakers they support.

To get the students started, I point them towards a number of resources. The first are some general sites about visual thinking or visualizing data such as:

The History Of Visual Communication
Thinking Visually (slideshow of concepts)
The Periodic Table Of Visualizations
Visual Complexity (Examples of how to display data visually)
Information Aesthetics (the last three links are to excellent blogs that focus on visualization)
Eager Eyes
Flowing Data

I also point them towards a number of readily available but under-used tools (such as Windows Movie Maker which comes with every Microsoft Windows machine) as well as some of the more interesting and useful online tools I have run across or which have been used by other students:

Programmable Web (massive list of available tools)
Animoto (automated, sophisiticated slide show production)
Comiqs (See an example here)
Just Leap In (customizable, web-based virtual world)
Community Walk (Customizable mapping software based on Google Earth)
Dipity (amazing timeline generation product)
MapWing (Simple tool for virtual tours)
Issuu (Sleek online document viewer)

These are obviously just a few of the tools and sites available for this kind of project (Don't believe me? Just check out MakeUseOf's List of Tools...). Do not hesitate to add your favorites to the comments!

Jigsaw Demo: A Powerful, New Visual Analytics Tool (YouTube)

Yesterday, I posted my initial reaction to an extraordinary piece of software called Jigsaw I saw demo-ed during my visit to GA Tech for the Workshop on Visual Analytics Education sponsored by NVAC. Today, John Stasko and his team uploaded a video of the software in action that is a must-see.

I understand the video is a little old so some of the functions I saw are not in this video. The good thing about John and his band of software wizards, though, is that they are constantly improving the product.

Amazing Data Visualization Tool From The Geniuses At GA Tech (Jigsaw)

(Note: I am at a visual analysis conference sponsored by the National Visualization And Analytics Center so the last of the Top 5 Methods list will have to wait a day or two. In the meantime, I have found something else that is pretty cool...)

John Stasko and the computer scientists at the Information Interfaces Lab at Georgia Tech may not have found the Holy Grail of visual analysis but they have come pretty darn close with their Jigsaw product.

This extraordinary visualization tool automatically extracts entities (names, places, dates, etc.) from plain text documents. Then, it automatically creates a visualization of the relationships between those entities and the documents containing them. The screenshots below do not do it justice (I hope to have a video of the product in action within a couple of days, though).


The program is fully customizable so you can add or delete data, designate entities or create relationships to modify what the automatic entity extractors come up with.

The real power of the tool comes into play after the data is in the program. You can play with it in a variety of powerful and interesting ways all accessible through a drop dead easy user interface.

The software is continuously improving. On the horizon is the ability to use web input and there is a long analyst generated to-do list that the grad students at GA Tech are cranking through one at a time.

The software runs on a desktop and was developed with a DHS grant so government and academics should reach out to John for a test copy. GA Tech is also home of the Visual Analytics Digital Library and well worth checking out.