Content is King: Enhancing Met Perspectives through Eye-Tracking & Behavioral Analytics

Client Overview

The Metropolitan Museum of Art (Met) is one of New York’s leading cultural institutions, boasting over 5,000 years of art from across the world. Their mission statement is as follows:

“The Metropolitan Museum of Art collects, studies, conserves, and presents significant works of art across time and cultures in order to connect all people to creativity, knowledge, ideas, and one another.”

Metropolitan Museum of Art Mission Statement

As part of this mission, the Met has a strong digital presence that helps to expose visitors to art in a variety of forms and modalities. Key to this digital strategy is the Met ‘Perspectives’ webpage, which aims to surface deep dives on pieces of art across multiple themes and content types. My team was tasked to help the Met understand the current user experience and expectations of the Met ‘Perspectives’ mobile webpage to help inform the Met’s future product roadmap through user research.

The Research Team

The research team consisted of four graduate students at Pratt in an advanced UX Research Methods course. Within my team, I led the research project management, data analysis and content writing work streams. All of us participated in moderating eye-tracking research sessions, conducting web analytics and synthesizing findings with relevant recommendations.

Our Process:

Typically the research team would determine the appropriate UX research method based on an initial client consultation and development of research goals. For this project however, our client reached out to specifically leverage our Eye-Tracking and Analytics expertise as this does not exist within their organization. This distinction was important as our research goals were developed with a pre-determined methodology.

Research Goals: Layout, Motivation & Engagement

Based on our broad objective of understanding the user experience and expectations of the Met ‘Perspectives’ mobile webpage, we identified three distinct research questions with our client:

Participant Recruitment

Our participant recruitment criteria prioritized researching users that were:

  • Young adults
  • Comfortable using mobile technology
  • Non-Professionals in Art History

Given the usage of eye-tracking as one of our methodologies, we also screened participants for an eye or vision problems, as eye-tracking software does not work effectively for users with abnormal eye shapes. Ultimately, 9 representative users were recruited via email invitations and our personal networks (visual demographic breakdown below).

Research Methodology: Eye-Tracking, Surveys & Behavior Analytics

This study was a mixed-methods study, meaning that multiple research methods were used to triangulate findings. This approach combined qualitative and quantitative research insights which increases the validity of findings and the level of insight that a researcher could provide.

Eye-Tracking

Eye-tracking was conducted using the Tobii Pro Nano software. Tobii Pro records a participant’s eye movements while they look at a digital interface and allows the researcher to view a user’s gaze patterns (specifically fixations and saccades).

The moderator must calibrate the participants gaze at the onset of an eye-tracking study using Tobii Pro Nano in order to capture their gaze patterns.

The Eye-Tracking method works by having participants perform a series of user tasks with minimal interruption. Eye-Tracking

List of User Tasks for Met Perspectives Mobile Eye-Tracking Study

Eye-Tracking differs from a standard usability test in that user feedback and reactions are collected after task completion, rather than during the tasks. The moderator plays back the participants gaze pattern recordings and prompts them for additional context / information. This method is known as retroactive think aloud, or RTA.

Retroactive Think Aloud (RTA) with Participant 8 reviewing their eye movements for Task 4

Survey

All participants completed a survey questionnaire via Google Forms following completion of the user tasks. There were two parts to the survey:

  1. System Usability Scale: Participants completed the System Usability Scale (SUS) survey which asks a set of 10 usability-focused questions to benchmark an interface’s usability on a scale from 0 – 100 (with the average score being 68).
  2. Sentiment Questions: Participants also answered questions about their expectations for using the website and likelihood to engage with the Met virtually and physically in the future.
SUS Survey Data Calculations in Google Sheets
Post-Study Questionnaire Google Form

Behavior Analytics: Google Analytics & Hotjar

To supplement our eye-tracking and survey findings, we utilized the Met’s Google Analytics and Hotjar pages to develop a stronger understanding of user behavior.

Google Analytics provided quantitative data on how user’s engaged with the Met Perspectives page and Met website overall. We were specifically interested inn figures around content engagement, page views and user paths.

Met Perspectives Mobile Page Views analytics from Google Analytics

Hotjar provided heat maps of user engagement with different pages as well as recordings of user interactions with content on the site. Due to relatively low mobile user traffic on the Perspectives page, we had limited Hotjar data but leveraged what was available.

Hotjar Met Perspectives mobile heat map displaying taps

Data Analysis: FigJam

Quantitative metrics and qualitative observations were initially captured using an Excel spreadsheet organized by task and participant. Following completion of all of our eye-tracking studies data was migrated to Figma and thematically analyzed using the affinity diagramming approach in Figma. Behavioral Analytics were brought in to reinforce findings as appropriate.

Observations and key findings for Task 3 in our shared FigJam board

Findings & Recommendations

Our final slide deck report is linked below. The following is a summary of the key findings and recommendations we developed as a research team.

Overview

Our users found the Met Perspectives mobile webpage to be a modern and aesthetically pleasing interface. Survey results indicated that most participants would recommend the webpage to a friend and they felt more inclined to visit the Met after interacting with the Perspectives page. However, the SUS questions indicated overall usability challenges with the site which were reinforced by our findings.

Met Perspectives mobile performs below the SUS benchmark with a score of 61.4
Participant responses to survey questions around expectations and experience with Met Perspectives.

The Met Perspectives mobile webpage received a score of 61.4, almost 7 points below the average SUS benchmark. Specifically, questions focused around usability scored 56.9, underscoring the opportunity to make meaningful usability improvements to the website. We identified 3 primary findings with accompanying recommendations based on our research.

1. Navigation on Perspectives page

Our research demonstrated that users faced challenges navigating both to the Perspectives page as well as within the page. Specifically, we observed that:

  • Page labels hindered wayfinding
  • Secondary navigation bar was ineffective
  • Pages hosting media type repositories were difficult to find
Page labels hindered wayfinding
Secondary navigation bar was ineffective
Pages hosting media type repositories were difficult to find

Fortunately, we were able to identify opportunities to mitigate some of the navigation challenges outlined above. Our recommendations were as follows:

  • Switch to a mobile-friendly drop-down menu & add content type filters to the top of the page
  • Improve user location understanding by highlighting page titles.
  • Introduce a refined search for Perspectives content.
Drop-down menu + dynamic content type filters
Highlighted page titles for improved user location
Refined content search capabilities within Perspectives

We believe that the implementation of these navigation recommendations (summarized below) would address the user’s existing wayfinding difficulties and improve their ability to orient themselves within the webpage.

Summary of navigation recommendations

2. Engagement with Different Content Types

While content engagement is the primary purpose of the Perspectives webpage, users faced challenges with identifying their desired content types quickly and accurately. Specifically, we observed that:

  • Users are not seeing or understanding icons by simply skimming.
  • Users are searching through text for content clues.
Users are not seeing or understanding icons when skimming
Users are searching through text for context clues

Icons were being used throughout the Perspectives page to help users identify the content type but it was not working as expected. Icons were applied inconsistently and there was nowhere for the user to confirm the icon’s purpose. Our recommendations for this finding were grounded in increasing consistency and clarity of labels:

  • Increase visibility & understandability of icons.
  • Consistency in usage of icons.
Add icons and their meaning to headers
Use icons consistently

By making these small changes to define icons upon the user’s first interaction with Perspectives and applying them across all content types consistently, we believe that user’s ability to engage with content effectively and identify their desired content type will improve.

Summary of Content Engagement recommendations

3. User Interaction with Close Look Articles

Task 5 directed users to read through a ‘Close Look’ article on the Met Perspectives page. These articles provide an image-centric deep-dive into a particular piece of art or artist collection. Our participants read the article, ‘The Visual Games of Juan Gris‘ and surfaced multiple usability challenges during their eye-tracking sessions:

  • Lack of embedded wayfinding tools can lead to user frustration with Close Look articles.
  • Long text chunks detract from the intended user experience.
Lack of embedded wayfinding tools can lead to user frustration with Close Look articles
Long text chunks detract from the intended user experience

The issues identified within the Close Look articles largely centered around a user’s desire to know where they were in the article and their ability to engage with the imagery. Our recommendations aimed to address these concerns with simple additions / typographic choices:

  • Highlight article length and pinpoint the location of the reader.
  • Limit all chunks of article text to 4 or 5 lines where possible.
  • Affix a “view whole image” icon to the screen.
Highlight article length and pin point reader location
Limit length of text chunks and affix ‘view whole image’ icon

The inclusion of average time of read recommendations and a progress bar would help to address the user wayfinding challenges identified within the Close Look articles. Limiting length of text chunks and providing the ability for mobile users to view the entire image should improve image engagement and balance gaze patterns between text and imagery.

Summary of Close Look Article Recommendations

Conclusion & Future Direction: Content in Context

Our findings and recommendations were well-received by the Met. Specifically, the focus on content organization and media types aligned with existing hypotheses held by the Met and provided additional data to support further investment in exploring these changes.

In terms of next steps, we recommended the following:

  1. Simple A/B Test: Given the findings around content organization, an A/B Test updating the secondary navigation labels and hyperlinks to reflect media types would be low-lift and further validate our research findings. Our team pre-programmed a test in Google Optimize on behalf of the client, with a primary emphasis on bounce rate.
  2. Scoping of Dynamic Filter: The dynamic content-type filters were a strong recommendation that we believe would meaningfully improve the usability and findability of content on the Perspectives page. However, we recognize the current sites structure does not support dynamic filtering of content. Getting an estimate of effort required for the change would help to determine whether a wholesale change is feasible and whether to focus on small labeling updates throughout or a redesign.
  3. Expanded Testing: Our sample size was relatively small for both the eye-tracking study and the behavior analytics. To increase confidence in scroll maps / heat maps and further generalize findings, we’d recommend recruiting additional users to repeat the study and strengthen our findings.

A One-Page Summary is available below summarizing our findings: