The Perfect Mid-term (GDMAG.com)

We are giving mid-term exams here at Mercyhurst this week.  I don't think much of traditional methods for testing student knowledge and was, therefore, intrigued by a quote I recently found in a much longer article on assessment:  

"Good tests make the student feel smart or skilled, if not both.  They make the student feel like they're applying what they've learned.  The puzzle inherent in a test should flow naturally from the existing, well-established content of the course.  While students should feel like they can bring their existing skills to the task at hand, they should also feel like a suitable challenge has been placed in front of them."

I like that.  Imagine a test so well-designed that the student would actually appreciate the challenge the test presented!   The implied lesson here is that tests ought to be about the learning and not about the grade, about mastering a body of knowledge and not about punching a ticket.

Many of you probably agree with me that such a test would be one worth taking.  You probably also think that such tests are impossible to design.  I have to beg to differ.  Such "tests" are routinely developed by a very unique group of people -- game designers.

See, I lied before.  The quote above is not from an educator, it is from Game Developer Magazine.  The article it was taken from is called "Make Better Bosses" by Damion Schubert (For those of you unfamiliar with the concept of a "boss" in games, I would refer you to the Wikipedia entry on bosses).  The original version of Schubert's article read like this:

"Good boss fights make the player feel smart or skilled, if not both.  They make the player feel like they're applying what they've learned.  The puzzle inherent in a boss fight should flow naturally from the existing, well-established mechanics of the game.  While players should feel like they can bring their existing skills to the task at hand, they should also feel like a suitable challenge has been placed in front of them."

All I did was replace "boss" with "test" and "player" with "student".  What really struck me was how obvious the comparison was when I first read it.  Of course, it might just be me (I have spent a good bit of time thinking about games and education over the last year) but I am hardly the first person to make the connection between tests of knowledge in a classroom and the challenges posed by most games  (See the short interview with Prof. Gee below).  It seems like something worth thinking about...

Intelligence Analysts Are Insanely Happy (WSJ.com)

According to a recent Wall Street Journal sponsored study (using info from Payscale.com), Intelligence Analysts are happy -- really happy.

Out of the 82 professions examined, intel analyst came in at number 7, scoring a whopping 73.1 out of 100 points.  Only one job, Aerospace Engineer, scored 100 of 100 and the other 5 happiest jobs were fairly closely grouped (you can get a general idea of the distribution from the image to the right but for the full interactive glory of this infographic, you have to go to the original WSJ site.  In fact, the entire "Paths to Professions" series is worth a look).

This study follows on the heels of the CNN report from last year that indicated that Intel analyst was the 9th best job in the country.  CNN did not look at happiness per se so it is hard to compare the two lists but it is interesting to note that, on both lists, intelligence analyst ranks so highly.

I find this particularly interesting given that intelligence analyst does not even have a Bureau of Labor Statistics Standard Occupational Classification System code, which means that the US Government is not tracking the profession in any meaningful way.  It suggests to me that intel analyst has become a popular and common enough job to earn the attention of both CNN and the Wall Street Journal but that there are few reliable resources for adequately managing the profession.

(Note:  Many thanks to K. for the link!)

Don't Use SWOT Again Until You Have Read This! (Thesis Months)

Image via WikipediaAs Mike Finnegan points out in his recently completed thesis, Evaluating SWOT's Value In Creating Actionable Strategic Intelligence, the strengths-weaknesses-opportunities-threats analytic method is "one of the most popular analytic techniques among competitive intelligence professionals".

Popular, yes.  Effective?  Not so fast.

Mike's survey of over 100 business people with real-world strategic planning responsibilities and experience using SWOT in the workplace paints a very different picture of the value of this technique -- a picture that should have consequences not only for the way it is used but also the way it is taught.

Specifically, Mike found that:

• SWOT adds value only indirectly to the strategic planning process
• SWOT is performed far less often than necessary if it is to achieve even these limited goals (up to four times less often than necessary if I am reading Mike's data correctly).
• SWOT should not be used as a standalone technique under any circumstances

The entire thesis is well worth the read for anyone interested in evaluating analytic methodologies in general or SWOT analysis in particular.  Mike collected a number of comments from his survey participants and they serve to ground the statistical data in the complexities of the real world -- to add qualitative gravitas to his quantitative research.

You can view the embedded file below or go here for the download (Note:  The file was clearly corrupted when Mike uploaded it to Scribd.  All the content is there but some of the pictures and graphs were moved around in weird ways.  Hopefully Mike will be able to fix it.  In the meantime, the data is still there -- it is just not as "pretty" as it was in the original format.)

Evaluating SWOT's Value In Creating Actionable, Strategic Intelligence

Intelligence Studies At US Universities: Who Does It? Who Does It Well? Where Is It Headed? (Dissertation)

One of the questions that seems to pop up with increasing frequency on the fora, blogs and email lists I frequent/subscribe to is "I am interested in a career in intelligence; where can I get a degree?"

William Spracher's recent dissertation, National Security Intelligence Professional Education:   A Map of U.S. Civilian University Programs and Competencies, not only answers this important question but also provides the first comprehensive snapshot of intelligence studies programs in the US.

NOTE:  The full text of the dissertation is embedded below or you can download the dissertation here.

Of particular interest to potential students will be Bill's descriptions of 14 of the most fully developed intel studies programs (beginning on page 136 with a handy summary chart on page 137) and the results of his survey of young intelligence professionals (beginning on page 76). 

Senior leaders within the intel community are probably going to be interested in the entire dissertation but I found the "crosswalk" of course offerings with ICD 610's intel core competencies (at Appendix C on page 235) to be particularly interesting.

Once caveat, though.  This dissertation, as useful as it is, is, in my opinion, just a snapshot of a quickly evolving target.  The dissertation was finalized in 2009 and I am sure that some of the info Bill uses was collected even earlier.  Bill recommends that the DNI stay on top of the trends in this field and, given the changes I have seen in just the last year or two, it is a recommendation with which I heartily concur.

National Security Intelligence Professional Education: A Map of U.S. Civilian University Programs and Com...

Best Companies To Work For -- For Entry-level Intelligence Analysts! (Fortune.com And Original Research)

Every year, Fortune Magazine posts a list of the 100 best companies to work for in the US. This is a very good list but it doesn't quite answer the question my students ask: "What is the best company to work for as an intel analyst?"

Thanks to one of our superb grad students, Nimalan Paul, we now have an answer!

Nimalan started with Fortune's list and made the initial assumption that, out of the 1000's of companies in the US, if you made the list at all you must be a pretty good place to work (Note: Nimalan used the list from 2006 as it was already available as a spreadsheet.  The vast majority of the companies from 2006 are still on the list today and since rank on the Fortune list did not matter in Nimalan's analysis, using the 2006 list seems acceptable).

From there, he thought long and hard about the criteria that would indicate that a company was good for entry level intel analysts.  He settled on  six factors:

• How many intel analyst (or intel analyst equivalent) slots are currently open?
• How many intel analysts appear to be employed by the company?
• At what level are the analysts employed?
• Is there a separate role for intel analysts within the company?
• Is there an internship program for intel analysts?
• Is there an executive level (C-level) position within the company responsible for intelligence?

He looked high and low for information on these six factors and compiled everything he found into the list you see below.  You can click on the second worksheet for his raw observations but he took it another step and actually scored each of the companies based on what he saw (you can see his scoring in the first or currently viewable worksheet).

The scoring is a bit subjective, of course (such that Nimalan indicated to me that the percentage scores are probably best interpreted as + or - 15% or so.  In other words, there is a real difference between a 60% and a 90% but probably not much actual difference between a 90% and a 95%).

Likewise, Nimalan was looking at companies that have intel positions in business exclusively.  He did not count contractual analyst positions provided by any of these companies to the US national security intelligence community.

Finally, we can't consider the list definitive.  Nimalan's ability to gather info on the companies was limited by time and access and we both acknowledge that there are likely some great places for analysts to work that didn't make it to Fortune's list.  If you know of any (and particularly if they are currently hiring...), please leave a note in the comments!