QUANT: The “Other” Analysis.

In todays world, Digital products are interwoven into the functions of daily life whether for business, social or personal purposes. The quality of these digital platforms or products go along way in the total experience for the intended audience. User centered designed digital products are needed and are growing in demand, hence a measure of the usability is an important but sometimes overlooked or dreaded activity for the caretakers of these platforms. However, for a platform to continue to engage its users and lure in new users, it is imperative timely and efficient quantifiable measures of the usability need to be taken into consideration for the insightful upkeep and continual use of any such digital product.


In life when problem solving, there are usually two parts to every problems equation, the question being one part and the answer the other. Both parts to the equation are relatively important as for some, the more important part is what the question is and others what the final answer is. This is somewhat the unspoken conversation in Usability research. For User Experience experts there are two common types of analysis done in the field namely, qualitative which is more subjective and the other quantitative, the boring yet other important fact grounding part. Depending on what is more important to the UX expert or organization attention is paid to one or the other, with qualitative being the more favored most of the time. With increasing methodologies and theories popping up in the growing User Experience field, it is important to be able to accurately identify and quantify any assumptions made about platforms and its user. Many leaders in the field propose that qualitative data alone lacks substantial validation without being applied in a quantitative manner, and quantitative data cannot exist without qualitative data. In recent times, there are growing number of platforms and books dedicated to such studies, like Quantifying the User Experience by Jeff Sauro, a thought leader in the UX Quantitative analysis field. The idea is for these conversations to help in making the case for quantitative analysis during usability research amongst audiences.


So what is Quantitative or Quant analysis in UX design and why is it relevant in digital platform usability? According to JAKOB NIELSEN,  Nielsen Norman group, “Quantitative analysis is a process that gives an indirect assessment of the usability of the design”(Nielsen, 2006).It can be based on users’ performance on a given task, for example task-completion times, success rates, number of errors or can reflect participants’ perception of usability, satisfaction ratings (Nielsen, 2006).While quants can factor and give us an idea of the level of usability of a site, it doesn’t go into the details of what makes it so usable. It is like the yin to the yang of qualitative or qual analysis. quant studies usually focus on the  numbers or “how many” types of questions focusing on the users performance rate and when properly gathered from a sizeable amount of a sample, such research can then be converted into qualitative.


So is Quant analysis really necessary and is it really time consuming? There are a lot of misconceptions and studies associated with quant usabilities which contributes to the reason most people avoid it or are intimidated by it. One of the more popular misconceptions is that you need a super larger set of data to sample. Quant is not that complex and not always time consuming, however there are a few concepts that need to be taken into consideration during Quant usability that past research/ers recommend.  According to Sauro, concepts to be mindful of are the power, the confidence level of detecting a difference, the probability value, sample size and the confidence intervals(Sauro, 2012). Confidence intervals is the one area where most User Experience experts get overwhelmed during quant usability research but there is hope as more efficient methodologies are being introduced to the field. For example methodologies like the Adjusted-Wald Binomial confidence interval, a promising emerging trend. The adjusted Wald Binomial interval is different from the traditional Wald interval, because in the equation x and n values are adjusted to get an adjusted value of P. According to Jeff Sauro, the amount of inaccuracy of the analysis really depends on the extent to which the sample fails to represent the population, so exercising care when selecting samples is important(Sauro, 2012). 


In summary, Quantitative analysis is important because it helps ground research and recommendations in facts. While quantitative usability is important, it is not important because of the numbers associated with it but rather the insights that are garnered from overall usage. This quantitative analysis statistical significance helps in the process of making more informed decisions, which reinforces and ultimately contribute to great design.


JAKOB NIELSEN, Quantitative Studies: How Many Users to Test? ,June 26, 2006. Retrieved from: https://www.nngroup.com/articles/quantitative-studies-how-many-users/
Jeff Sauro, Jim Lewis, A New Formula for Quantitative UX Decision Making. August 7, 2012. Retrieved from http://uxmag.com/articles/a-new-formula-for-quantitative-ux-decision-making
Jeff Sauro, Measuring U, FIVE CRITICAL QUANTITATIVE UX CONCEPTS . September 25 2012.  Retrieved from https://measuringu.com/five-hard-quant/
Jenna Erickson, Usability Geek, UX Designers: Do Not Be Scared Of Quantitative Data. April 10, 2017. Retreived from: https://usabilitygeek.com/ux-designers-quantitative-data/RALUCA BUDIU.  
Quantitative vs. Qualitative Usability Testing, October 1, 2017. Retrieved from https://www.nngroup.com/articles/quant-vs-qual/