Would Designers Find Their Way to Statistics?

——Yes, They So Much Love To.


To be a great designer of user experience, it matters that the design work is brilliant. Nevertheless, to be a opinion leader of great designers, evaluation metrics of designs and effective communication of works are of considerable necessity. In this post , this post and this post, We will discuss about Quantifying User Experience, the relation between designers and Statistics, types of user testing and sub-classification, A|B Testing as a most common technique of comparing designs and how designers tell stories and make use of statistics through data visualization.


When I first looked for related information on internet, I cannot find a systematic way of discussing the topic “Quantitative Usability”. I then turn to Library for help, luckily I found several related physical books such as “Quantifying the User Experience”, “Eyetracking the User Experience”, Thanks to Librarian Nicholas Dease from Pratt Library who provide instruction about partial keywords search, such as inputing “Quan*”,you will get all related resources of keywords: quantitative\quantify\quantity. In the following paragraphs, several questions are provided to guide the thinking process in detail.

What UE Disigners Think of Statistics?

Initially, “The last thing many designers and researchers in this field of user experience think of is Statistics.” quoted from “Quantifying the User Experience”,Jeff Sauro / James R.lewis, 2012 P1. In this phase, Statistics are similar to giant strong devil to designers. Designers try not to work with numbers and they fear and are not confident to think about Statistics, not to mention make good value of it.

Increasinglyusability practitioners and user researchers are expected to quantify the benefits of their efforts. If they don’t, someone else will- unfortunately that someone else might not use the right metrics of methods.” quoted from “Quantifying the User Experience,Jeff Sauro / James R.lewis, 2012 P1. After seeing the good application of statistics and also the persuasion through data, designers are more likely to cope with quantitative methods, especially proving in numbers why their designs worth favoring.

How Useful is Quantitative UE Research?

In the next phase, designers are confident, they not only can see Statistics as a tool, but also, they are trained with more mastery. Designers are smiling, with a smirk face (smirk means to smile in a way that expresses satisfaction with yourself or pleasure about having done something or knowing something that is not known by someone else.) “The user of statistics method does not guarantee 100% correct decisions, but it doesn’t have to be perfect to be effective.” (Quantifying the User Experience,Jeff Sauro / James R.lewis, 2012, P272 ) Statistics provides “the world’s most specific way of saying maybe” (Searles, 1978, P110) 


While making statistics useful, there are a few steps to take. Let’s start with some concepts.There are two types of user testing: Formative and Summative

Similar as tests of student learning, these terms come from education(Scriven, 1967)

Formative: providing immediate feedback to improve learning.
Summative: evaluating what was learnt.

When we try to Quantify User Experience, we see more specific definitions.

Formative: finding and fixing usability problems 
  • A small-sample qualitative activity
  • Data form-problem descriptions and design recommendations.
  • Quantify the problems in terms of frequency and severity
  • Track which users encountered which problems
  • Measure how long it took them to complete tasks
  • Determine whether they completed the tasks successfully.
Summative: describing the usability of an application using metrics
  • Two types of Summative tests: Benchmark and  comparative.

Benchmark tests are similar to comparative tests, when one of the comparative object is set as a fixed requirements/standards. We will talk about comparative tests more this time.

What if we want to know

if there is a Statistical Difference between Designs?

“Many researchers first realize the need for statistics when they have to compare two designs or products in an A|B Test or competitive analysis.” (Quantifying the User Experience,Jeff Sauro / James R.lewis, 2012,P272 )

Comparative: (A|B testing)It can be a comparison of a current with a prior version of a product or comparison of competing products. Please refer to The Post about A|BTesting, you will find details about this widely useful technique. Moreover, after mastering skills, designs find that they not only mastered statistics, but also make it fun while creating data visualizations. Thus, designers can play with statistics when that happens.






Design Critique: streeteasy.com (website / iPhone app)



For a fresh New Yorker, the first goal to achieve might be looking for an  apartment. Streeteasy.com is a marketplace provider in New York City, which links buyers, sellers and renters with access to comprehensive sales and rental listings, and information about all buildings and neighborhoods. The following critique is based on intensive usage over a time period of two weeks, with three attributes covered from consistency, feedback to constraints.


Consistency ***

Consistency refers to designing interfaces to have similar operations and use similar elements for achieving similar tasks. However, there is a frequent button on both website and IOS App, used for contacting with the operator of the apartment, which actually brought about confusion. I was using website and I told my future roommate that I have sent messages to those available apartments, how about you. And she was confused because all she can see is CONTACT AGENT instead of SEND MESSAGE. At first I was thinking maybe the product manager changed the name of the same button because there is not enough space, but it turns out that the one on App is one letter longer than the label on website.

As a result, we both contacted agents and sent messages without knowing that we are under the same action to the same building, until we both received emails of notifications that those emails are sent through both buttons from website and IOS App with the same feedback.


Feedback ****

Feedback is about sending back information about what action has been done and what has been accomplished, allowing the person to continue with the activity.

In the page of a specific rental unit, if the user want to “bookmark” an apartment, it is visible on the right side by the photo of units which are the most focused area. When save button is clicked, the label SAVE turns to SAVED, and also, a yellow label will star the profile as a recognition for user to remember this is a apartment they have viewed and shown interest. Moreover, there is another hidden button ADD NOTE, so the users are not only given feedback of the action, saving the profile, but also provided with the following process of an action.


Constraints ****

Constraints-The design concept of constraining refers to determining ways of restricting the kind of user interaction that can take place at a given moment. There are various ways this can be achieved.


Please refer to the above two screen shots.


The ADD NOTE button was hidden also provides a good example of restricting possible user interactions. It is said that, the best number of options that you can give to a boss is 2. So the third button was hidden  unless SAVE button is turned to SAVED.


Affordance ***

As Norman mentioned in 1988,At a very simple level, to afford means to give a clue. When the affordances of a physical object are perceptually obvious it is easy to know how to interact with it.

By providing some visual icon, button and clickable color and bold character, the website is of a high affordance level. First of all, words in blue shows a clickable attributes, just as clickable as buttons with blue or white colors. the show wheel of pictures and two pointing buttons imply the easy transition of photos. What’s more, the message box was designed to be similar with a real message on a paper, thus, they are all recognized without much effort. Nevertheless, the only confusion here is the envelope icon standing for SHARE, envelope icon should be used to send message instead of copying link and sending to other platforms.



StreetEasy is a website with users ranging from various background, and they have respective conceptual models. The website has followed most of the design principles of Don Norman, except for a couple of confusion points as I mentioned above.