Introduction: As a user experience design student, quantitative research is mentioned far less than qualitative research. My brief encounter with quantitative usability data was during a class assignment where we were supposed to perform tree test through Optimal Workshop, but I was not fully aware why and when we need to quantitative data in usability research. Quantitative usability research is the unsung hero that points out the problem with solid statistic backup if done well, while qualitative research adds context to the problem and informs researchers how the problem can be solved.
With qualitative research, such as interview, survey, and remote usability testing, we get very interesting user feedback that brings close-up perspective to look at a problem. However, how do we identify a problem first? That is where quantitative research comes into play. With statistics, quantitative research gives a “Yes” or “No” answer to questions like, “is the event section on the website homepage can draw enough users to click on it?”
Quantitative usability research measures interface through 3 aspects: effectiveness, efficiency, and satisfaction. Effectiveness is indicated by how users correctly performed a task. Efficiency is measured by whether a task is completed and how long it takes to complete the task. Satisfaction is measured through questionnaires to indicate user’s experience with the interface.
The greatest benefit of quantitative research is that the results it delivers is very unlikely to be a fluke, which could be a problem in qualitative research when the test participants change. This benefit is a result of strictly controlled consistency of quantitative research task questions, test settings, and participants’ familiarity with the interface.
Despite of its obvious benefit, quantitative research is usually very expensive, takes a long time to complete, and needs many participants (ie. over 30 participants). Due to its expensive nature, many company are hesitate to use such research method. When companies consider whether a decision is expensive, it considers the actual cost, the cost–benefit ratio, and the available budget. The main concern for investing in quantitative research is that it may not generate result that worth the investment, or its actual cost may exceed the existing research budget. This is a common phenomenon among companies where the company is not aware of the value of quantitative research. Without quantitative research, the company’s UX team could risk the time and money working on a less significant problem which is tangible to the change of test participants’ population. More importantly, when combining with qualitative research, quantitative research is a very valuable tool to test the hypothesis problem qualitative research brings up.
When company doesn’t give quantitative research credit, such research becomes difficult to be carried out. However, a even more severe challenge lies in performing quantitative research. A recent survey conducted by Nielson Norman Group among 429 UX professionals indicates UX team lack the knowledge on how to conduct, run, analyze and report quantitative research. It seems quantitative research is an intimidating subject among UX designers.
Quantitative research should go hand in hand with qualitative research to better identify the problem, create informed design solutions, and verify the design solution. As companies’ UX culture grows, the budget on quantitative research is increasing. The future for implementing quantitative research without too much resistance within companies seems not too far, but the how and why quantitative research should be used needs to be thoroughly addressed in future UX professionals’ training.