Playability Heuristics

Next up in the reading stack is Playability Heuristics for Mobile Games.

Stemming from the literature on usability heuristics, the authors (Korhonen and Koivisto) develop a set of playability heuristics for mobile games. In the paper, they present their motivations for developing these heuristics, the heuristics themselves, and the ‘results’ of their ‘validation’ of these characteristics.

Their heuristics are grouped into three categories: gameplay, mobility, and game usability. Their initial list of heuristics was made up of the following:

  • Don’t waste the player’s time
  • Prepare for interruptions
  • Take other persons into account
  • Follow standard conventions
  • Provide gameplay help
  • Differentiation between device UI and the game UI should be evident
  • Use terms that are familiar to the player
  • Status of the characters and the game should be clearly visible
  • The player should have clear goals
  • Support a wide range of players and playing styles
  • Don’t encourage repetitive and boring tasks

In order to validate these heuristics, six evaluators applied them to a selected application and noted all playability problems that were both covered and not covered by the list of heuristics. The evaluators found 61 playability problems, but 16 of these were not adequately described by one of their heuristics. Thus, the authors expanded their initial set of heuristics into three expanded sublists (one for each ‘category’):

  • Game Usability
    • Audio-visual representation supports the game
    • Screen layout is efficient and visually pleasing
    • Device UI and game UI are used for their own purposes
    • Indicators are visible
    • The player understands the terminology
    • Navigation is consistent, logical, and minimalist
    • Control keys are consistent and follow standard conventions
    • Game controls are convenient and flexible
    • The game gives feedback on the player’s actions
    • The player cannot make irreversible errors
    • The player does not have to memorize things unnecessarily
    • The game contains help
  • Mobility
    • The game and play sessions can be started quickly
    • The game accommodates with the surroundings
    • Interruptions are handled reasonably
  • Gameplay
    • The game provides clear goals or supports player- created goals
    • The player sees the progress in the game and can compare the results
    • The players are rewarded and rewards are meaningful
    • The player is in control
    • Challenge, strategy, and pace are in balance
    • The first-time experience is encouraging
    • The game story supports the gameplay and is meaningful
    • There are no repetitive or boring tasks
    • The players can express themselves
    • The game supports different playing styles
    • The game does not stagnate
    • The game is consistent
    • The game uses orthogonal unit differentiation4
    • The player does not lose any hard-won possessions

This expanded set of heuristics was validated using the same process, only now with five different games. Based on this process, the authors draw the following conclusions:

  • Usability problems were both the easiest to identify with their heuristics, as well as the easiest violations to make.
  • More mobility problems were found than expected.
  • Gameplay is the most difficult aspect of playability to evaluate.

Yowza–talk about a scattered paper. I mean, this bad boy is all over the place. It seems as though the authors’ thoughts simply haven’t gelled well at all. Nevertheless, they do present what seem to be reasonable heuristics for the evaluation of playability. I have two primary problems with this paper. First, an the world of smartphones and mobile games has changed dramatically in the last decade. I would imagine an more recent look at playability is both available and more useful. Second, while their heuristics seem reasonable, and they claim to have validated these heuristics, I can’t find any evidence of this. Do Korhonen and Koivisto not understand that just using a set of heuristics doesn’t imply that they are valid? This leads to the bigger question of what it means for a set of heuristics to be valid. Do valid heuristics completely describe all possible problems? Is the ‘most’ valid set of heuristics that which completely describes all possible problems with the fewest heuristics? I’m not sure. I am sure, however, that writing a list of heuristics and then applying them absolutely does not make them valid. The analysis necessary to do so just isn’t present in this paper. Even if the authors claim to have begun to validate their framework of heuristics, they certainly haven’t presented any such results in this paper. While the work shows (showed) promise, I find this both misleading and frustrating.