Choosing a Game for Threat Assessment

The difficulty about Threat Assessment is that it requires developers to ‘think out of the box’: to think in ways that do not seem logical. Specifically, instead of thinking like a user of the software, they must think like someone who wants to misuse the software—use it in ways the designers did not intend. That requires what Edward de Bono called ‘lateral thinking’. It needs some way to help developers to think differently from their normal ways of thinking about software.

A good way of doing this is by a game; the frivolity of the game and the randomness that the game can inspire is very useful. A card game works particularly well, and several different security specialists have designed card games to help developers to think up security problems.

In the workshop, we trialled three different games:

• Elevation of Privilege was designed and promoted was by Microsoft, and particularly by Adam Shostak.
• Security and Privacy Threat Discovery Cards was designed by Tamara Denning’s team at Washington University.
• Adversary Personas was designed by Nick Merrill’s team at Berkeley.

 In the workshop, we gave the participants a case study of a software project that required security decisions. The participants, as is typical for SPA conferences, were mainly very experienced software developers and architects from industry. We split them into three teams of about five people each, and asked them to use each game in turn to identify possible security problems. We ignored the more sophisticated rules in the games, and simply used the cards as a prompt for people to think of different possible things that might go wrong. And, to reduce bias, each team did the games in a different order. The workshop was carried out online using a Miro board; you can see the participant instructions and results after the workshop here. 

To analyse the threats they discovered, we asked the teams to use ‘dot voting’ to identify the most interesting, and the most important threats they found. 

As workshop leader, I didn't suggest anything to the participants in advance. In fact, my expectations, given we only provided non-technical overview information about the project, were that the participants would find Elevation of Privilege, which is somewhat security-technical and designed to analyse an existing implementation, hard to use effectively; that Adversary Personas, designed to help identify attackers and their motives rather than technical issues, would probably be most effective in this situation; and that Threat Discovery Cards, which has a good deal of information about threats on the cards, would be somewhere between the two.

What we found was different: actually, all three approaches worked very well; each of the games inspired the developers to think up a respectable number of new threats (Elevation of Privilege included). So, we can conclude that any of the three card games will work well.

The table below shows the numbers of threats found by each game:

So, my hypothesis, that Adversary Personas would generate the largest number of threats, was wrong—though, interestingly, Adversary Personas did generate the largest number of ‘important’ threats. And Elevation of Privilege worked fine despite having neither an existing implementation nor security experts. But the Threat Discovery Cards were decidedly the most effective, generating over half as many threats again as Elevation of Privilege.

These results are hardly statistically significant, so we cannot be sure of all the deductions, but one conclusion is clear: all three games are effective at helping developers find threats. To choose between the three games, we suggest you select the one that appeals most to your team!

– Charles