Ethics in Usability Research: Invasive Tracking

Shadowing, monitoring, and recording of users has become increasingly sophisticated. As these services become more robust, providing increasingly granular and specific information, they can raise the risk level for users. They tout many advantages for researchers but their activities are often non-consensual with respect to users. The user is unaware of their depth or just how much they are divulging while simply browsing.

Why monitor your users? There are several benefits to monitoring users on your site for the researcher and user alike. Not only will your customers be provided with a more personalized experience, but keeping tabs on them can allow a researcher to view the entire customer journey and identify areas of frustration. Using a service that runs in the background also has the advantage of reducing the need to arrange usability tests. These services provide plenty of data that can easily be segmented. And real users mean higher quality data and, in return, better design solutions. (It can also helpful for replicating bugs for Q&A since many of these services record console errors.)

Ethical concerns. Users expect that their page visits and clicks will be recorded, but most don’t realize that their keystrokes and mouse movements can easily be recorded as well. This data can then be matched up with their identity. The mental model for most users leads them to believe that all their actions are private until they take an action such as clicking a link or pressing submit. They are unaware of how much they are sharing and if it is being saved by another party. Sometimes the data that is recorded is very sensitive.

I have listed three cases below that highlight these concerns:

Case1: FullStory and Walgreens. Walgreens used FullStory, a popular “session replay” company, with hopes of improving their online customer experience. FullStory is able to generate user sessions by recording their actions. This could sometimes include sensitive information such as credit card details, prescriptions, and even medical conditions. Although there are redaction tools available in the FullStory software, and Walgreens used them heavily, personal information was still showing up in replay sessions.

FullStory "Replay Session"

The FullStory dashboard plays a session while showing all user actions listed on right. The individual user is also identified.

FullStory Segment Tools

FullStory offers its clients a robust set of options for segmenting their visitors.


Case 2: Facebook Shadow Profiles. Website owners that run Facebook ads can install “Facebook Pixel” in order to get more advertising and tracking capabilities. This bit of code also has another function. When anyone visits a site that is using Pixel, FB will link that visit to their existing FB profile. Facebook also retains this data and builds “shadow profiles” of users that do not have accounts. These profiles can become active if the user joins Facebook. The fact that FB retains profiles of people that did not sign-up for their service is disturbing.

Facebook Pixel

Installed into web pages, Facebook Pixel is intended to provide tracking information for their ads, but it also helps build “shadow” profiles of non-Facebook users.


Case 3: Customer Chat Windows. The conceptual model of a chat window that includes a send button is that until you finish crafting your message and hit submit, your message is not yet shared. For most customer service chats, this is not the case. The service person can see as a user types. This is done so that the service person has more time to develop a response and get a read on the person’s mood. The user is unaware they are being monitored. And the fact that the user needs to press submit to move ahead in the experience, leads us to believe they are knowingly deceptive.

Putting Your Users At Risk.  Data collected by these services far exceeds user expectations. One would not expect to have a pervasive profile built by browsing or to have precise mouse movements recorded while moving through a page. These activities are clearly non-consensual and go beyond the general implicit consent user feel they are giving by visiting a site. Anything above page history and data they submit is unexpected. A researcher should take this into consideration when choosing how to track users activity.

Also to be considered is the handling of information. Any data that is being collected should be weighed against whether or not it is necessary for reaching your research goals and if the user expects it to be collected. Be transparent. Be careful and protective of user information.

Putting Your Users At Risk

Possible Solutions: All Users to Opt-In. The default for any user should be that no data is collected without consent. Generally, we work under the assumption of “implied consent” by the user for common tracking such as page history. Ideally, a user should be able to opt-in to different levels of data sharing. These could include the type of activities, retention time and how the data will be used. A modal or message that can be closed out is not enough. The user must be required to activate more invasive measures.

There are benefits to being tracked for users. They will want to have certain items tracked to improve their experiences. They may also be willing to provide data if asked to help improve the experience overall.

Shadow profiles are the biggest flaw in Facebook’s privacy defense.
The Dark Side Of ‘replay Sessions’ That Record Your Every Move Online.
No, you’re not being paranoid. Sites really are watching your every move.
Online customer agents can see what you’re typing even before you hit that ‘send’ button.
About Facebook Pixel. Facebook business.

Design Critique: My Disney Experience (Web)


My Disney Experience is both a web app and mobile app for navigating and planning your visit to Disney. Having Disney Annual Passes, everyone in my family uses this app constantly. Before this app, we had to make reservations by phone, or in person, and schedule fast passes at a kiosk upon arriving at the park. Although we knew the parks well, we would occasionally need a map to navigate. This app was a game changer for us and really enhanced our trips to Disney.

For my critique, I chose to focus on the web app due to the fact that the mobile app works best when on location.


Using the URL will take you to the My Disney Experience (“MDE”) landing page within the Disney World site. This page also corresponds to (1) “My Plans” from the MDE dropdown menu and works as a dashboard. It provides a quick overview of all your current plans and possible actions.

The app, in general, does a great job of providing feedforward through good discoverability that allows a user to easily identify possible actions. Using clear signifiers, the user is first presented (2) with five buttons that utilize both visuals, title and short descriptions to communicate. The visuals leverage common images or knowledge in your head, so the function of each button is easily understood.

Another level of information is communicated through whether the button icon is grey or blue. The darker color means you have made plans in this area. Not as obvious as the button signifier, but something the user quickly picks up on. Upon mouse rollover, users are presented with buttons for possible actions.  Once again, providing signifiers.

Treatment of the FastPass+ button indicates status.

Good: Easy discoverability of possible actions (creating groups, getting tickets, making reservations and fast passes) that along with signifiers (buttons/icons) provide the feedforward to bridge the Gulf of Execution.


There are several “My” themed links available through the My Disney Experience dropdown menu. The conceptual model for the user would expect that if a user chooses these links, they are navigating within this section.  The varying navigations on these pages do not follow expectations and can be confusing.

On the “My Photos” page, (3) the link at the top takes the user out to the Disney World landing page instead of back to My Disney Experience.

My MagicBands & Cards page (4) makes a clear relationship between the page and MDE.

And the page with a where you link current reservations to your MDE (5) has no such mapping.

Poor: Navigation is not consistent and in some cases does not match the conceptual model. The mapping in the navigation between Disney World content and what is specific to MDE could be more clear.

Possible Solution:  Mapping could be helped by adding an element to the main navigation that delineates or separates MDE from the rest of the navigation items. Also, a consistent breadcrumb on “my” pages would help with the conceptual model. See solution below.




The most used function of the My Disney Experience is the FastPass+. If you have ever been to the park, this is a logo and term that you will certainly remember.

This functionality of the app provides constant visual updates as feedback that allows a user to quickly detect if there has been an error, or slip, and make a correction.

Clicking on a selection results in an instant visual update.

The above image which combines the before and after screens while adding another guest to a FastPass+ reservation (the goal) is a good example of the Seven Stages of Action. The left side clearly shows the stages of execution (plan, specify, perform) and the right the evaluation stages (perceive, interpret and compare).

The screenshot below for choosing your date makes good use of a commonly known conceptual model.

The list of available FastPass+ times for attractions is mapped in a way that most people will understand with (6) the earlier choices on the left. There are also (7) clear messages explaining why some choices are not available which is appreciated feedback.

Although… I like the hazard signifier from the mobile app for these messages better 🙂

The use of constraints is used to prevent slips, such as a Loss-of-Activation, by greying out buttons or not displaying the next action until all selections are made. There are also (8) several cues for consequences of actions in case of other slips, such as Capture Error during a series of selections.

The constant feedback loops work to ensure there is no Gulf of Evaluation and the user is clear on the results of there actions.

Conclusion: The app makes good use of many of Norman’s design principles to deliver an enjoyable and empowering user experience that we find fun to use.