Developed in the early 1990s, the heuristic evaluation has come to prominence as a usability evaluation method. Developed in a time when web design meant design for desktop screens, it has been a powerful tool largely thanks to its reliance on the 10 heuristics, defined by Jakob Nielsen in 1995, that categorize the issues found by expert evaluators.
Now, with the growth of mobile interfaces, Nielsen’s list has been criticized as outdated and no longer relevant for the different modes of interaction offered by touch devices. But as the need for “discount” or “lightweight” evaluation methods as grown (also noted by Nielsen in a 1997 article), the heuristic evaluation has continued to be an important tool. In efforts to adapt it for mobile, several researchers have defined their own sets of heuristics tailored to mobile. The question is: do these work as replacements for Nielsen’s elegant list?
The original 10 heuristics have persisted in use because of their simplicity, as it is relatively easy to remember 10 objects, and their mapping, with each heuristic clearly communicating its role. Any user of the system can quickly appreciate the breadth of description contained in the list. Any replacement list will have to equal these features, matching them to the demands of touch interfaces.
In looking at 5 proposed sets of heuristics it quickly becomes clear that this is much easier to achieve in theory rather than practice. One proposed set includes 21 points, broken into 5 larger categories. Not only is this difficult to understand, some of the proposed heuristics, such as “Predictability” and “Consistency” seem largely to overlap (Ji, 2006). Another heuristic “checklist” proposes 22 points, some of which are specific as an item stating that a design must include a button or link to allow users to bookmark the page (Tay, 2011). On the other side of the spectrum, heuristics have been proposed with as few as 6 points (Baker, 2014). While this heuristics do tend to focus on important issues, such as the optimization of button sizes to avoid mistyping on touch devices, these simply seem to be too broad and do not provide enough guidance to evaluators as to how they should seek to address the issues they identify.
Another issue these heuristics must address are the specific design concerns of mobile devices. One possible set of 13 heuristics provides for this, but only in 230 separately defined sub-heuristics (Gomez, 2014). This then simply replicates the issue discussed above of complexity.
Finally, another set of 8 heuristics, designed for mobile devices in general does appear to address this adequately (Bertini, 2008). Put forward in 2008, these heuristics cover such issues as screen readability, ergonomics and reachability of items on the screen, and place a strong emphasis on system status, which is key for phones that rely on often patchy connections to provide services that demand constant internet access. Unlike the heuristics described above, these 8 criteria seem to attempt to match Nielsen’s general ideas with the demands of mobile devices and emphasize the most common areas of difficulty.
What is currently unanswered is if any of these heuristics, or any other newly proposed schemes, can identify mobile usability problems as well or better than Nielsen’s original 10. There seems to have been little testing conducted directly comparing heuristic sets in how they perform against specific mobile applications and webpages. I would suggest further exploration along these lines to ensure that mobile usability is being fully tested.
References
Baker, Rebecca & Sun, Xiaoning. (2014). Empirical Development of Heuristics for Touch Interfaces. Available at http://www.uxmatters.com/mt/archives/2014/06/empirical-development-of-heuristics-for-touch-interfaces.php
Bertini, E. et al. (2008). “Appropriating Heuristic Evaluation for Mobile Computing.” International Journal of Mobile Human Computer Interaction. January-March 2008: 20-41.
Gomez, Rosa Yanez, Caballero, Daniel Cascado & Sevillano, Jose-Luis. (2014). “Heuristic Evaluation on Mobile Devices: A New Checklist.” The Scientific World Journal Sept. 14.
Ji, Yong Guo et al. (2006). “A Usability Checklist for Usability Evaluation of Mobile Phone Interface.” International Journal of Human Computer Interaction 20.3: 207-231.
Nielsen, Jakob. (1995). 10 Usability Heuristics for User Interface Design. Available at http://www.nngroup.com/articles/ten-usability-heuristics/
Nielsen, Jakob. (1997). Discount Usability for the Web. Available at http://www.nngroup.com/articles/web-discount-usability/
Travis, Tiffany & Tay, Aaron. (2011). “Designing Low-Cost Mobile Websites for Libraries.” Bulletin of the American Society for Information Science & Technology 38.1: 24-29.