Quantitative Usability in the Corporate World: Data-Driven Design

By / / Tools & Techniques

Summary

This review briefly explores how quantitative and qualitative usability methods are leveraged by product managers as data-driven design to develop and deliver memorable user experiences.

Quantitative versus Qualitative

What is quantitative usability? According to the Nielsen Norman Group, it’s important to note the difference between quantitative and qualitative usability and what is required to carry out each. Specifically, the two methods are complementary to each other and serve different purposes (Budiu, NNG, 2017). Budiu states that quantitative or “qual” consists of “one or more metrics (such as task completion rates or task times) that reflect whether the tasks were easy to perform” (Budiu, NNG, 2017). Qualitative or “qual” consists of “observational findings that identify design features easy or hard to use” (Budiu, NNG, 2017).

While Budiu distinguishes the two with clear inputs, outputs, and use cases, as mentioned, there are there opportunities to conduct a study which leverages both. In doing so provides robust insights not just for UX/UI specialists but also for other data-driven specialists whose career is centered on leveraging data for how users interact with their product and reiterating based on those insights. One notable professional is the product manager.

Data Analysis Infographic

Behold, The Product Manager!

According to O’Reilly Design, as a product manager, “much of your job comes down to translating between the needs, perspectives, and skill sets of your stakeholders and customers” (O’Reilly Design, 2017). This role unifies all parts of the organization to conceptualize products according to business needs, competitive analysis, and user data; they must engage in cross-function collaboration to unify the work of UX/UI researchers and designers, data analysts and scientists, developers, and company executives. This exciting role allows individuals to wear many hats and apply many skills, however, with that comes a price of requiring a strong emotional state to not only manage the work but colleagues’ varying skillsets, expectations, and feedback (O’Reilly, 2017). According to the Harvard Business Review, some key characteristics of a successful product manager are:

  • *Team player*
  • Emotional intelligence
  • Agile project management
  • UI/UX concepts
  • Design abilities
  • Technical knowledge
  • Analytic abilities
  • Strong writing skills
  • Cross-functional business acumen

Despite the level of intensity, the product manager is truly an admirable individual. A key characteristic of this role is that they unify data and design–a rather difficult yet fruitful marriage of often conflicting ideologies. Product managers collaborate with the intent to design products and experiences with data in mind. This practice acknowledges that “data capture, management, and analysis is the best way to bridge between design, user experience, and business relevance” (King, Churchill, & Tan, O’Reilly, 2017). King, Churchill, & Tan acknowledge that “that design intent and evaluation are often poorly matched to the data capture and analysis because designers with a desire to understand user experience have not been in effective dialogue with data scientists and machine-learning experts” (Designing with Data, 2017).

Product Design

The product manager’s role and responsibilities will differ across industries and organization size. Product managers at smaller organizations may find themselves leveraging more out-of-the-box technologies for less complex workflows, but with less team support. Wearing of many hats will go beyond the general expectations of the role and expand to satisfying the requirements of other roles due to the lack of team support. However, corporate product managers, depending on the industry, may use an array of tools for a more complex workflow consisting of many team members. Either way, the product manager’s toolbox will generally consist of the following:

Design/Code Version Control

  • GitHub
  • Sketch
  • Visio

Analysis

  • Heap/Segment/Google Analytics
  • Optimizely/VWO/Hotjar
  • Typeform/SurveyMonkey

Project/Product/Feature Management

  • Invision
  • JIRA/Confluence/Aha!
  • Mural/Trello/Asana/Monday

Technology

  • SQL
  • Tableau/Looker/Airtable
  • Heroku/Rails/Frontend/Backend

Many of the above tools will support the product manager’s workflow in addition to the analysis they conduct to ensure new products, redesigns, or iterations are successful. The analysis a product manager conducts could range depending on the features or products they support. Sample metrics or key performance indicators (KPIs) to guide a product manager’s strategy are:

  • Testing the success of marketing messages, email/web/platform features.
  • Assessing the performance of a marketing campaign and which high-performing campaign should receive funding.
  • Constructing synthetic versus treated groups to predict the interest and identify core impact.
  • Measuring unengaged users or ‘drop-off’ usage of specific user types over time (predictive/time windows).

References: