In civilian foreign affairs analysis they have proven to be as effective as in intelligence assessments.
At their core, SATs can be viewed as an assortment of cognitive-enhancing techniques used for organizing and externalizing analytic reasoning.
“A Tradecraft Primer: Structured Analytic Techniques for Intelligence Analysis”[i] identifies 12 base SATs and groups them into three categories, each defined by its chief analytic purpose: to foster diagnostic, contrarian, or imaginative reasoning.
Diagnostic techniques are primarily aimed at making analytic arguments, assumptions, or intelligence gaps more transparent.
Contrarian techniques explicitly challenge current thinking.
Imaginative thinking techniques aim at developing new insights, different perspectives and/or develop alternative outcomes.
A more extensive and detailed taxonomy was proposed by Richards J. Heuer Jr. and Randolph H. Pherson in their seminal work “Structured Analytic Techniques for Intelligence Analysis”[ii].
The cornerstone of this taxonomy, however, was laid some years before by Richards J. Heuer Jr.[iii] who separated SAT from three other analytic domains:
In accordance with the Heuer-Pherson taxonomy it is further broken down eight subdomains.
While our course only aims to teach SAT foundations it currently incorporates some 16 techniques from selected subdomains.
The human mind can efficiently process only so much information. Beyond a certain threshold of complexity reasoning fails to grasp the implications and feedback loops contained in it. Any technique that can visualize analytic reasoning can be of great help in mitigating this shortcoming. In foreign affairs analysis, a sine qua non.
Network Analysis
Politics, by definition, is a team game. One way of getting to understand how it is played is to figure out what makes the teams tick. Network Analysis aims precisely at determining the configuration and operational modalities of groups of actors.
The basis for Network Analysis is laid down by Network Charting. It is a process aimed at identifying network actors, nodes (points of interaction) and links between them. It involves collation of information into Link or Association charts.
Network Analysis is the next step aiming at interpretation of the information contained in, and missing from, such charts. When done correctly it always pursues a specific and clearly defined objective. The point to be made here is that a chart in and of itself is not a self-standing analytical product.
Document Exploitation – can be considered a subcategory of Association Analysis. It is essentially a post-raid technique used to extract information from all kinds of seized evidence and collate it into a relational database.
Threat/Harm Analysis/Risk Assessment in Foreign Affairs Analysis
A technique for setting priorities to guide allocation of scarce resources. It provides an understanding of a threat posed by a problematic phenomenon, harm that could be caused by it, risk that this harm would materialize, and vulnerability of the environment.
Threat is a possibility of damaging action as an outcome of deliberate planning.
Harm is the damage occurring should a threat be realized.
Risk is the probability of materialization of harm.
Vulnerability is an aspect of the environment offering to the threat opportunities to cause harm.
Our treatment of threat assessment is based on the Long Matrix used in conjunction with the expanded T4 matrix.
They were pioneered by the Royal Canadian Mountain Police and over time adopted globally. For details about its origins look up information on the SLEIPNIR project[iv].
This technique allows analysts to rank-order threat agents in an objective, comprehensive and systematic way. The outcome is quantification of threat posed by specific groups of hostile actors to persons, institutions and the society as such.
The focus of threat assessment in foreign affairs analysis is on producing actionable knowledge. It can assist authorities at different levels in setting priorities and choosing pre-emptive or response strategies. Threat assessment techniques can also support the measuring of effectiveness of disruptive actions. Methodology would involve comparing change in threat indices of targeted actors for several periods of time.
Beyond that, the expanded T4 matrix is well-suited for determining the level of success of disruptive measures aimed at degrading specific group capabilities. Last but not least, its use can reveal a group’s ability to recover from disruptions.
Single and Multiple Perspective Utility Analysis
Most problems involve more than one stakeholder. As a rule, these have conflicting interests.
Intuitively choosing among alternative courses of action is often problematic. The added values of multiple perspective utility analysis include both providing an honest merged perspective and allowing to find a way of fixing it – but NOT to play one stakeholder against the others, and NOT to shift the burden of responsibility for the fix.
Alternative Futures Analysis
A common misconception about the Futures Analysis is that it makes predictions. It does not. Instead, it aims to generate possible outcomes based on available information, assess their comparative probability and offer an estimate regarding the most probable path leading into future.
The purpose of futures work in strategic analysis is more specifically to:
The answer sought through futures analysis can include combinations of the following:
Assessing causality offers a different angle for conducting analysis of alternative futures.
Force Field Analysis
The key to FFA is to assess when inhibiting and facilitating drivers are most vulnerable to external pressure and consequently determine where and when own effort is likely to be most effective in achieving the desired outcome. Its specific purpose is thus to suggest when and how to act.
In our course, we show how to use Force Field Analysis in combination with the Futures Wheel technique to identify and rank 16 possible 3rd tier futures and suggest the most probable path leading three tiers of consequences up from the trigger event. Analysis is based on 280 individual data points.
Futures Wheel
The Futures Wheel technique helps to create a formidable collection of possible futures and rank them in accordance with the probability of their occurring.
When generating consequences that add up to futures we teach students to let their wildest fantasies soar free. The Futures Wheel will unequivocally tell whether they are just fantasies. If so, no harm done. If, on the other hand, they turn out to be feasible futures, the better so. In our experience, breath-taking strategic scenarios rarely score high to become the Most Probable Path (MPP).
Being the product of 280 values, the MPP tends to be fairly stable. If tomorrow one‘ll try to review them all and eventually modify some, such tinkering won‘t have a big impact on the MPP. It is likely to switch to a different track only if one gets some new insight, or new piece of key evidence, or changes one’s key assumptions.
Analytic Failure Simulation
Analytic Failure Simulation aims to challenge the accuracy of a futures estimate. The starting point is that one’s judgment proved to be wrong. It is an accomplished fact and one needs to explain why.
The simple logic behind this technique aims to remove the need for an analyst to defend the preferred hypothesis and shield it from criticism by others to prevent discovery of its weaknesses. Such a change of perspective encourages an analyst to shake off ownership in the failed judgment and actively search for its flaws, as if it were someone else’s product.
What if? Analysis
What if? Analysis imagines that an unexpected event has occurred with potential major impact and uses the benefit of simulated hindsight to figure out what could have been its cause.
For the purpose of our course, we assume that the hypothetical trigger event used in the Futures Wheel technique has actually occurred. The objective then becomes to extend the Most Probable Path leading into the future also into an imagined past and suggest, which of the alternative pasts has the highest probability to have triggered the whole chain of consequences.
High Impact/Low Probability Analysis
This technique aims to create a step-by-step scenario for the unfolding of a particularly unlikely and hence previously not considered future with potentially major harm repercussions. An objective is to develop a set of warning indicators that could signal its sudden unfolding.
We present this technique using a case study that explores a scenario for a sea-born invasion and subsequent takeover of the Mediterranean island of Lampedusa (Italy) by IS combatants co-mingled with African migrants.
Let’s start with the issue of information versus noise.[v]
Data is toxic to analytic judgments even in moderate quantity. It has been empirically proven that more data does not result in better estimates. It only raises analyst’s confidence in the accuracy of own conclusions. (For more details, look up Paul Slovic’s study of racetrack handicappers).
Search for more information tends to contaminate analysis with more noise. Continuous updating of hypothesis burns up a tremendous amount of time. While bringing little, if any, increase in accuracy, it seriously risks getting analytic reasoning hijacked by confirmation bias.
Most of data signals surrounding an analyst and continuously assaulting his mind is not information but precisely noise – data that is random, untrue, faked, irrelevant, incomplete and meaningless.
Since we consider own judgments prized possessions, these turn sticky. More random noise will only make one more confident in one’s initial hypothesis formed on the basis of vague and incomplete information.
A typical error of collection is aiming to continuously monitor data sources. The more frequently one looks at data, the lower the ratio of signal to noise tends to become. With time, noise prevails over evidence and skews analytic reasoning. Foreign affairs analysis degenerates into fantasy.
It turns out that standard collection practices can encourage and even REQUIRE analysts to AIM AT CAPTURING NOISE. The result is predictable: junk in – junk out.
In order to arrive at a particular judgment analysts do not need more data. What they need, is to focus and filter out the noise that is hijacking their reasoning.
Humans as a species tend to be demonstrably arrogant about what they think they know. They have a tendency to think that they know a bit more than they actually do. The field of foreign affairs analysis is but one example. While overestimating what they know, people at the same time underestimate uncertainty. Overconfidence can have a toxic effect on analysts. It motivates them to adopt vastly more risky positions than warranted by available information. Analyst overconfidence results in blindness to errors and is a primary reason for the most epic intelligence failures.
Overconfidence is often borne out of disregard for the uncertainty inherent in most analytic problems. Probability is a non-intuitive concept – it cannot be easily derived from everyday experience. We study it in certain depth, but here is a quick primer on the subject.
[i] US Government, March 2009
[ii] CQ Press, 2nd edition 2015
[iii] “Taxonomy of Structured Analytic Techniques” (paper prepared for the International Studies Association 2008 Annual Convention)
[iv] For example, “Sleipnir: The Long Matrix for Organised Crime, an analytical technique for determining relative levels of threat posed by organised crime groups”, Royal Canadian Mounted Police Criminal Analysis Branch, Criminal Intelligence Directorate, 2000.
[v] In the next paragraphs we discuss and offer own illustrations of the ideas expressed by Nassim Taleb in “The Black Swan” and “Antifragility”.