The Ethical Issues of A/B testing in the Tech World

In recent years, ethics in usability research has increasingly become a concern among users. Large corporations track, monitor, and manipulate interfaces to evoke an intended reaction from a user. An example of this would be with A/B testing, which is a standard UX research method that can often yield some ethical issues.

A/B testing is a method in which different variations of a webpage or app exist and are shown to different users with the purpose of evaluating which one will perform better (Optipedia). With A/B testing, an example of a variation could be a change in color or mapping of an item. Better performance is defined in regard to a given task. More often than not, this task is with the intent of getting users to click on some type of call to action. This can be in the form of purchasing an item, signing up for a newsletter, simply clicking on an intended location, or many other options.

There are a few ethical concerns that come into play with A/B testing. For example, users are often unaware of the fact that they are participating in these large-scale studies. In 2012, Facebook conducted a study on almost 700,000 users in which they showed the newsfeed of half of the participants “positive” posts for a week and the other half “negative” posts for a week. This resulted in positive newsfeed users posting positive words and negative newsfeed users posting negative words (McNeal, 2014). The purpose of this study was ultimately to examine if the emotions of one user would be affected by the emotions of another. Facebook later published their findings and underwent scrutiny for unethical research practices. This study in particular can be deemed unethical due to lack of informed consent and potential emotional impact.

Additionally, in 2013 and 2014, OkCupid conducted three experiments on their users with questionable ethical research approaches. A description of all three can be found here. However, the final experiment was significantly more controversial than the others. OkCupid matches users by assigning a rating percentage, which informs users on whether or not they are compatible. For their experiment, they attempted to examine the likelihood of users conversing if they were shown a higher match rate (e.g. 90%) verses their actual match rate which was much lower (e.g. 30%). Their hypothesis was that the myth of compatibility will be just as strong as the truth, which turned out to be true. Users that were shown a higher match rate were more likely to converse, even though it was a false rating. This experiment may also be deemed unethical due to lack of informed consent, manipulation, and wasting participants time.

The findings from the OkCupid studies were later published and also under went scrutiny. However, one of the co-founders and data scientists responded by stating “if you use the internet, you’re the subject of hundreds of given experiments at any given time, on every website. That’s how websites work.” This response elicited rage and anger by many users, blogs, and media outlets. It also left the impression that large tech companies do not value their users or see ethical issues, simply because “everyone does it” (Kolowich, 2014).

Most internet services use tools to monitor user activity, length of time of time on a page, and click rates (Farrell, 2017). In addition to those metrics, A/B testing can be an excellent method for researchers to determine the effects of different variations of pages. However, organizations should be more transparent about testing, even if it may seem harmless. In these examples, both companies lacked informed consent and could have potentially impacted users’ psychological states. With OkCupid, they even falsified match rating, which in turn manipulated their users. By simply alerting users on landing pages that they may be subject to A/B testing, many of these ethical issues would be addressed.

References:

Optipedia, A/B Testing. Available online: https://www.optimizely.com/optimization-glossary/ab-testing/  [Accessed on March 18, 2019]

Farrell, J. (2017, December 07). Why OKCupid’s ‘Experiments’ Were Worse Than Facebook’s. Retrieved March 20, 2019, from https://www.huffingtonpost.com/joseph-farrell/why-okcupids-experiments-_b_5655217.html

Hern, A. (2014, July 29). OKCupid: We experiment on users. Everyone does. Retrieved March 20, 2019, from https://www.theguardian.com/technology/2014/jul/29/okcupid-experiment-human-beings-dating

Kolowich, L. (2014, August 1). Facebook, OkCupid, and the Ethics of Online Social Experiments. Retrieved March 20, 2019, from https://blog.hubspot.com/marketing/ethics-experimentation-ab-testing

McNeal, G. S. (2014, July 01). Facebook Manipulated User News Feeds To Create Emotional Responses. Retrieved March 20, 2019, from https://www.forbes.com/sites/gregorymcneal/2014/06/28/facebook-manipulated-user-news-feeds-to-create-emotional-contagion/#22800ada39dc

Design Critique: Lyft (iOS App)

Introduction:

Lyft is a transportation company that provides users with online ride-hailing services through a mobile application. The application matches on-demand riders with drivers and also allows payments through the app. Lyft operates in approximately 300 U.S. cities and provides over 1 million riders per day.

Critique 1: Home Screen

When opening the Lyft app, you are immediately brought to the home screen. This screen is a virtual map which shows nearby drivers in real-time as car icons. This home screen is a compelling conceptual model of how this application works. This is because the car icons moving in real-time, coupled with the clarity of the map represent the idea of how one may perceive traveling in a vehicle.

The home screen is well designed due to good discoverability, mapping, and signifiers. The overall mapping of attributes on the home screen allows for easy discoverability in regard to:

  1. Your current location signified by a blue dot with a white outline and a triangle pointing in the direction you are facing. I’m aware that the triangle that points out in the direction that I’m facing due toknowledge in the head. I’ve learned from other GPS apps that it is meant to face in the direction that I am facing.
  2. The location of nearby drivers which are signified by car icons.
  3. The magnifying glass which signifies that you may use this bar to search. This also follows the principal of knowledge in the world in that one associates a magnifying glass with searching.
  4. The label “Search destination” is another signifier indicating where the search should take place. Additionally, it allows affords searching for intended destinations.
  5. Displaying a previous destination with an icon.

Overall, the signifiers, mapping, and discoverability are well designed. However, when displaying a previous location, there is no clear signifier informing the user that this has been a previous destination. The current icon being used as a signifier may not be effective since it requires knowledge in the head, acquired through repetitive usage of the application to learn that these have been previous destinations.

Solution:

A suggested solution for this would be to include an additional signifier, a label that says “Previous,” therefore users know that this location was a previous destination.

Critique 2: Hamburger Menu

The home screen is designed for users to request cars to travel to intended destinations but there are several other options available as well. Some of these options are things such as ride history or access to payment information. Due the physical constraints of a small screen, the hamburger menu was implemented with clear feedforward intent as a secondary navigation to get to the other options. The mapping of the hamburger menu in the upper left-hand corner makes its easily discoverable. Additionally, it’s further discoverable by the appropriate icon as a signifier as well as being grounded in a white circle, increasing visibility and affording other options.

Moreover, the hamburger menu further offers immediate feedback as all options are shown once tapped on. The hamburger menu has physical constraints in that it cannot be pulled out any further than it currently does. It also does not allow for scrolling further than the options listed, and if one does then they will be automatically scrolled back to the list of options. This automatic scroll back could be interpreted as immediate feedback.

However, there is one design flaw that comes to mind when I use the hamburger menu. After clicking into on the options, I was no too sure how return the previous screen. The signifier that is currently used three dots in the upper left-hand corner. I feel that this could pose a problem for users that do not have knowledge in the head, due to previous experience with the app.

Solution:

Replace the three-dot signifier with a back arrow instead. A back-arrow signifier uses intuitive feedforwarding for users to return to the previous page.

Critique 3: Ordering a Lyft

There are clear signifiers with immediate feedback informing the user that they are making the appropriate steps to order.

After clicking “Search destination” you are able to type your destination in. Clear mapping of labels such as “start” and “end” in the both text boxes act as signifiers, affording the user to input their destination. Additionally, there is a pink bar in the text box that further acts a signifier. Once you input your destination, you receive immediate feedback, in the form of a screen that maps the route to your destination.

You’re then prompted with appropriate signifiers to choose whichever type of Lyft you’d prefer. Finally, once you’ve tapped the “Select Lyft,” you once again receive immediate feedback informing you that the app is searching for a driver to accept then you will be pick up.