Surfnet logo

Critic Proofing Games During Development

This is a completed project.


Game reviews play an important role in both the culture and business of games – the words of a reviewer can have an influential effect on the commercial success of a video game. While reviews are currently used by game developers to aid in important decisions such as project financing and employee bonuses, the effect of game reviews on players is not known. Additionally, the use of game reviews to improve evaluation techniques has received little attention. In this project we investigate the effect of game reviews on player experience and perceptions of quality. We show that negative reviews cause a significant effect on how players perceive their in-game experience, and that this effect is a post-play cognitive rationalization of the play experience with the previously-read review text. To address this effect we designed and deployed a new heuristic evaluation technique that specifically uses game reviews to create a fine-grained prioritized list of usability problems based on the frequency, impact, and persistence of each problem. By using our technique we are able to address the most common usability problems identified by game reviews, thus reducing the overall level of negativity found within the review text. Our approach helps to control and eliminate the snowballing effect that can be produced by players reading reviews and subsequently posting their own reviews, and thus improve the commercial success of a game.


Images and Videos



  • Livingston, I.J., Mandryk, R.L., Stanley, K. 2010. Critic-Proofing: How Using Critic Reviews and Game Genres can Refine Heuristic Evaluations. In FuturePlay 2010, Vancouver, Canada. 48-55. DOI=10.1145/1920778.1920786.
  • Livingston, I.J., Nacke, L.E., Mandryk, R.L. 2010. Critic-Proofing: Robust Validation Through Data-Mining. In Playability and player experience: Proceedings of the Fun and Games 2010 Workshop, NHTV Expertise Series 10. Leuven, Belgium. 81-94. Workshop paper.
  • Livingston, I.J., Nacke, L.E., Mandryk, R.L. 2011. The Impact of Negative Game Reviews and User Comments on Player Experience. In Sandbox ’11: The 6th ACM SIGGRAPH conference on video games, Vancouver, Canada. To appear.
  • Livingston, I.J., Nacke, L.E., Mandryk, R.L. 2011. Influencing Experience: The Effects of Reading Game Reviews on Player Experience. In the International Conference on Entertainment Computing, Vancouver, Canada. To appear.


AlienTrap Software – CFINSERC – SurfNet