Your New Favorite Analytic Methodology: Structured Role Playing

If you use only one analytic methodology this week, Structured Role Playing (SRP) should be it. 

SRP as a way to improve analysis boasts supportive literature from the fields of law, military, pedagogy and politics. It is also frequently compared - and compared favorably - to other approaches such as Devil's Advocacy, expert opinion judgments and game theory within the relevant academic literature (check out the list of references at the bottom for more info). 

Note: The 2011 study is the "role-thinking" study which,
although yielding better than chance results, was not as effective
as SRP.

Basically, SRP is putting yourself in another person's shoes and looking at situations from different vantage points. 

Sounds pretty straightforward. 

But it's more than just thinking through what another person might be experiencing (a process cleverly named "role-thinking"by Kesten Green and Scott Armstrong in 2011 which has shown to be less effective than SRP). 

It's sitting down with a group of analysts, assigning roles of various actors and playing out situations as if you were those people, acting with the cultural, political and personal motivations of all parties involved. It is spontaneous. And, if done properly, can be extremely revealing. 

In order to implement SRP correctly, there are certain criteria that have to be met.

The two main criteria for effective SRP are that it has to be as realistic as possible and that the scenarios be objective (not decisively biased towards one outcome or another). 

Beyond those structural tenets, other recommendations for effective SRP include:

• Assign roles before presenting the situation (according to Linda Babcock, this makes a big difference).
• Ensure a high level of involvement by participants (also called "active role playing" as opposed to "passive role playing" or "role thinking"). This involves personal and spontaneous interaction between participants.
• Maintain a low degree of response specificity. It is better to let participants improvise responses than to have them select responses from a predetermined list, for example, in a multiple choice format. 

As an aside, it is relevant to mention that the 2011 role-thinking study, which yielded the lowest of the SRP results listed in the chart above, violated the last two of these implementation recommendations, possibly leading to its poor comparative performance.

As is the case with all methodologies, however, there are critics of the technique. 

The main criticism of SRP is its relative ineffectiveness when not correctly implemented. As mentioned above, "role-thinking" does not work very well at all. This point, however, is easily addressed:  Follow the simple guidelines to ensure correct implementation of SRP.

The second, less investigated criticism of SRP is overconfidence resulting from having employed the technique in the first place. This phenomenon, if replicated, is generally true for any analytic methodology.

Ultimately, SRP is an analytic technique that allows analysts the opportunity to interpret situations from distinct vantage points. It's collaborative, it's spontaneous and effective. 

Extended reading list:

• Armstrong, J. S. (2001a). Role playing: A method to forecasting decisions. In Armstrong, J. S. (Ed.), Principles of forecasting: A handbook for researchers and practitioners. Norwell, MA: Kluwer Academic, 15-30.
• Armstrong, J. S. (2001b). Selecting forecasting methods. In Armstrong, J. S. (Ed.), Principles of forecasting: A handbook for researchers and practitioners. Norwell, MA: Kluwer Academic, 365-386.
• Babcock, L., Loewenstein, G., Issacharoff, S., & Camerer, C. (1995). Biased judgments of fairness in bargaining. The American Economic Review, 85, 1337-1343.
• Galinsky, A. D. & Mussweiler, T. (2001). First offers as anchors: The role of perspective-taking and negotiator focus. Journal of Personality and Social Psychology, 81, 657-669.
• Green, K. C. (2002). Forecasting decisions in conflict situations: A comparison of game theory, role-playing, and unaided judgment. International Journal of Forecasting, 18, 321-344.
• Green, K. C. & Armstrong, J. S. (2011). Role thinking: Standing in other people’s shoes to forecast decisions in conflicts. International Journal of Forecasting, 27(1), 69-80.

What Employers Want (AACU)

The American Association of Colleges and Universities recently released a report titled "The Quality Imperative." Central to this report is the chart below on what employers want from colleges (and, by extension, from college graduates):

(Sorry for the poor quality of the graphic. Click on the chart or the link in the first paragraph to get the full report)

What struck me (and what also struck the authors of the report) is the duality of the data: Employers want more emphasis on science and technology and on global issues. They want more emphasis on complex problem solving and on ethical decision making.

The authors of the report seem to be making the case that we no longer live in a world with an ivory tower at one end of the spectrum and a trade union at the other. We now live in a world where theory and application are inextricably bound together.

It is difficult, sometimes, for students to understand this -- that it is a broad appreciation of the world combined with concrete skills that will serve them best in a future where it is difficult to know what is important now, much less what will be important next. It is good to see that employers are well-aware of the way the world has changed and of the broad range of skills necessary to deal with it.