Quantitive Usability

In the field of user experience, there is a wide range of research available, researchers mapped 20 methods across 3 dimensions: “Qualitative vs. Quantitative”, “Attitudinal vs Behavioral” and “Context of Use”. This article will focus on introducing the “Quantitative research” method, and in order to make it more clear, it will also compare it with another direction in this dimension—“Qualitative”.

(figure 1)

What is Quantitive usability?

“Quantitative usability testing (or benchmarking) is based on a large number of participants (often more than 30); (quant) data, in form of one or more metrics (such as task completion rates or task times) that reflect whether the tasks were easy to perform.” (When to Use Which User-Experience Research Methods, 2014) In most cases, quantitive data provide a more indirect assessment of the usability.

It can reflect results about task-completion times, success rates, the perception of users (figure 2) or a number of errors occurred in participants’ user journeys.

(figure 2)

Its characteristics can be summarized as below:

What?—Focus on “how many?” and how much?”. The outcomes are statistically meaningful results that are likely to be replicated in a different study.

Why?—its goals are to evaluate the usability of an existing site; to track usability overtime; to compare the site with competitors or to compute ROI;

When?— You can use it when you have a working product, either at the beginning or end of a design cycle.

The methodology: Involve many participants, the test should be well-defined, strictly controlled study conditions, and don’t need a think-aloud protocol.

Pros and Cons of Quantitative Usability


It is statistical significance. Quant data can be very supportive when delivering design, In addition, quantitative methods do a much better job answering “how many and how much” types of questions. The numbers can help prioritize works, for example, to focus on issues with the biggest impact. It’s also regarded as an essential instrument in calculating return on investment. (14 Usability Testing Tools Matrix and Comprehensive Reviews, 2014)



Since a large amount of data is required, it’s often very time-consuming. In emergency circumstances, such as when an action is required urgently, this method is not recommended.

-Can be hard to interpret in the absence of a reference point. For example, in a test, if 60% of participants were successful, it is hard to say it’s successful or not. Experts said it is the reason why this method is rather to describe the usability of a site, it is often used to compare the new design with a previous one.

-It doesn’t point out “What problem”users encountered.

According to a quant usability test, you can know data such as “only 20% of participants completed the tasks”, but you won’t know what difficulties users met and how to make it easier.

What needs to be careful when doing “Quantitative Usability”

  • For quant studies, tasks need to have a single well-defined answer.

It means the answer cannot be open, for example, a task like “Find the address and phone number of MET”, if the user only fount its address but didn’t find it’s phone number, it’s hard to define if the task is a success or a failure.

  • need to be stringently controlled from session to session.

That is, you need to make sure that your tests should be run under a similar environment as possible, which means: you cannot run one session in person but get the rest done remotely.


When trying to get general insights of usability of one site, Quantitive usability can be an appropriate choice, such as “how many people are satisfied with their experience here?” Or “How many clicks went to the wrong direction?” It can also help you compare one design version with another. It is a statistical type of method which needs relatively strict condition control. It often is used when there’s an existing system already.



Design Critique: Uniqlo APP (IOS application)


Uniqlo is a popular casual apparel brand, they recently developed an APP to support their online retailing market.  As most functions of the app directly linked to their responsive web so that it can complete their basic goals–purchase items online, generate guests’ own digital barcode of their member accounts, get coupons and alerts, etc.

However, the APP still needs some improvements regarding the user experience. According to concepts and principles in Donald Norman’s book “Design of Everyday Things”, I will give some critiques of this app from both positive and negative sides.



constraints of item selecting function

As Norman said–“Forcing functions are a form of physical constraint: situations in which the actions are constrained so that failure at one stage prevents the next step from happening.”, the APP can automatically enable certain “size”,”color” or “quantity” by turning them into grey color and unclickable when selected items are out of stock, it is a widely used function applied in online retailing stores, it can effectively prevent people from selecting items that they want but actually out of stock, and then be realising those items are unavailable when they try to purchase at last step– it could cause anger and frustration.  It’s a smart and necessary “constraint” which can prevent inappropriate operation.


1)Discoverability of “Feedback of web/product”&”view module” 


  • Feedback on products and web

According to Norman’s concept, the “Discoverability” means if It is possible to determine what actions are possible and the current state of the device. 

In this case, people might be confused when they see this pop-up window at the first time–especially if they click it by accident (It is placed at the side of the screen and easy to be clicked). In my experience, I realized it was a feedback window until I actually selected one of the buttons and saw the rating bars. According to Norman’s seven fundamental principles of design: Effective use of signifiers can unsure discoverability. I suggest using some signifiers such as 1. Replace “Select a category” with “How is your experience?” 2.Add “Review the” in front of each category to remind people that they can write their reviews here thus to improve the discoverability.

  • View Module

The discoverability of the “View Module” was relatively low, one reason is the use of Icon can hardly indicate the layout of the pictures. It looks more like a “book” and a “screen”. The other reason is the spaces were too small compared to the item number. The item number may be important but it is not operatable, it does not need such a larger space than “view module” and “filter” which allow people to click and actually operate.

Above all, I redesigned the “view module” icon which more related to pictures’ actual layout and also rearranged the spaces for three items to enhance the discoverability of too operatable functions.

2)Mapping of Text corresponding to the image.

“Mapping is an important concept in the design and layout of controls and displays.” A good use of mapping can make it easy to determine how to operate it. In this seasonal picks’ section, Although the staggered layout makes it looks beautiful and reduced the boring, unfortunately, it increased the confusion– when consumers tried to select one item, she/he might hesitate, especially when there are two items have similar features, such as “tops” and “shirts”.

I suggest to divide these items by card element, put each image and their corresponding words into an individual card, so people can clearly understand which label belongs to which image.



Norman said, “Good design requires stepping back from competitive pressures and ensuring that the entire product is consistent, coherent, and understandable.” As I was a former designer of Uniqlo’s product, I certainly know that the difficulty of designing an online store APP for a brand which has such countless items to sell and replaces their items so fast. Considering it’s relatively bad user experience, besides the drawbacks I mentioned above, the APP still needs a lot of improvement, designers need to consider more from users’ side but not only designers’ taste,  make it more user-friendly by using good signifiers, mapping, and consistent layout, I believe unlike now–it has only two stars rated on the APP store, with more and more improvement, Uniqlo APP can be a popular and easier APP to use, thus to attract more people to shop their items on mobile phone.