What is the Hawthorne Effect?

Introduction

During usability research, there could be many cognitive biases happened that finally violate the accuracy of the data. Both researchers and users can have biases, such as framing effect, confirmation bias, social desirability bias, etc (Subramanian, 2018). Here I would like to introduce one of the cognitive bias, called the Hawthorne effect.

What is the Hawthorne effect?

The Hawthorne effect was first discovered through a set of study in the 1920s by the Hawthorne plant of the Western Electric Company, Chicago(Stand, 2000). The studies were originally focusing on the relationship between the changing of environmental factors and the productivity of workers(Macefield, 2007). The Hawthorne effect was accidentally discovered through the studies of the lighting level(Snow, 1927).

In one study, workers were asked to repeat their tasks under three different lighting levels, and their productivity was evaluated. In another study, the workers in the control group were working under stable lighting and the experimental group under increasing lighting. The final result was surprising, under all these conditions the productivity was increased.

There are many interpretations of the result. The most widespread one was given by Mayo(1933). He argued that the increase in productivity was due to the test participants believed that the changes would improve their performance,  and they were reinforced by the fact that they were being observed during the study.

However, other researchers such as Parsons(1974) questioned this interpretation. He believed that learning and feedback could better explain the phenomena. The workers used to have no feedback on how well they were doing, in this study, however, they could possibly set themselves higher targets and try to achieve them, and finally increase the productivity(Macefield, 2007).

Despite many other interpretations on the Hawthorne effect, Mayo’s opinion keeps a dominant position, although Parsons’s opinion is more recognized in the field(Macefield, 2007).

How might we overcome the potential Hawthorne effect?

It is hard for us to review the study and get a systematic conclusion right now since the environmental factors were not strictly monitored by the researchers(Stand, 2000). But we can still find out some methods to build a better relationship with tested users, relax them and improve the accuracy of the collected data. Oswald(2014) and other researchers introduced a way called “six-stage protocol” as a practical way. According to Oswald(2014), after the first five stages are finished before going into a conversation, the overall research result could be more reasonable. The six stages are shown in figure 1(Oswald, 2014, p.59).

 

 

Figure 1. Six stage protocol for Hawthorne effect mitigation

 

Stage 1. Gauge the person

In many cases, researchers are often recognized as strangers. It is hard for strangers to gauge a user, but it is important to do so. We need to gauge the type of users and the circumstance they are under. People tend to alter their behavior based on different circumstance, therefore, it could be a good idea to communicate with them under different circumstances.

Stage 2. Create a non-threatening perception

To dress in a casual way and look non-threatening could be helpful in creating a non-threatening perception. Additionally, it is also practical to adapt your behavior to the circumstance. For example, if the circumstance is more social, do not act too professionally. What’s more, figuring out a more proper way to observe the users is also helpful.

Stage 3. Introductions

For both researchers and users, it could be stressful to meet someone new, therefore, if the research could keep calm and relaxed, and introduce him/herself, it is more likely to build up a non-threatening environment.

Stage 4. Establishing rapport

Rapport happens between individuals when there is a harmonious understanding between them. Building rapport with your users is necessary to improve communication, build trust, and more importantly, improve the accuracy of data. We can achieve that when we:

  • keep an open and accepting body language;
  • maintain eye contact;
  • nod and show an interested smile;
  • try to agree with the user;
  • call them by their name early in the conversation;
  • be praising if appropriate.

Stage 5. Relaxed signal

It is important to find out the point when users are comfortable communicating with you. Typically, a great signal of getting to the point is a light-hearted comment or joke at the beginning of the section.

Stage 6.  Link to the conversation area

Once the user is relaxed, it is time for the researcher to start the research.

 

Resources

Macefield, R. (2007). Usability studies and the Hawthorne Effect. Journal of usability studies, 2(3), 145-154.

Mayo, E. (1933). The human problems of an industrial civilization. New York: MacMillan.

Oswald, D., Sherratt, F. R. E. D., & Smith, S. I. M. O. N. (2014). Handling the Hawthorne effect: The challenges surrounding a participant observer. Review of social studies, 1(1), 53-73.

Snow, C. E. (1927). Research on industrial illumination: A discussion of the relation of illumination intensity to productive efficiency. The Tech Engineering News, 257-82.

Stand, J. (2000). The” Hawthorne effect”-what did the original Hawthorne studies actually show. Scand J Work Environ Health, 26(4), 363-367.

Subramanian, S.(2018, Jun 7) 10 cognitive biases to avoid in User Research (and how to avoid them). Retrieved from https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6

 

 

Design Critique: Path Vending Machine

 

Introduction

The path vending machines are used to add value or time to a MetroCard or SmartLink by walking users through steps. Users need to interact with a touch screen and multiple parts to get to their goals. We can easily find one on a path train station.

 

Critique 1: Different colors are used to distinguish between different functional areas

Before using the machine, users can find different areas separated by colors. The touch screen is placed in the black area, cash will be inserted in the green area, MetroCards go into the yellow area, ATM/credit card should be inserted in the blue area, and receipts or changes will be received from the red area. Colors are helpful to distinguish different functions from a visual aspect, users can easily discover which area to interact with during processing.

 

 

Critique 2: The start button on the screen is big enough to be discovered

When users first interact with the machine, a start button on the upper right is easy to be discovered on the screen. A signifier above the bright color bar saying “Touch start to begin” is visually clear enough and also indicate to tap the button. What’s more, touching a button on the screen is also in line with the users conceptual model. They will not get confused on how to use the touch screen during the next steps.

 

 

Critique 3: Using physical constraints, signifiers, and feedback when paying a bill

If users want to add value to their cards by cash, they will not be upset when inserting coins or paper bills, because there are physical constraints for different entrances- coins can only be inserted to the upper left entrance, and paper bills to the lower right one. There are also signifiers displayed to tell what the entrance for and the what values are accepted, which is clear enough and easy to understand. After inserting a paper bill, the light is on for a while to give users feedback, that a paper bill is inserted successfully.

 

Critique 4: No mappings or feedbacks when typing a zip code

If users want to pay by card, they will be asked to dip their card and type the zip code to finish the process. However, there are no feedbacks when typing on the keypad, therefore users would be confused if they have tapped a key or not. There are also no mappings between the keypad and the screen, sometimes users may forget how many digits they have tapped. I suggest that there should be a sound or vibration after each typing, to give users feedback. Also, there is a need to display on the screen, so that users can map between their actions and the result.

 

 

Critique 5: It is hard to discover the touch area for the SmartLink

When adding time or value to a SmartLink, there is a signifier on the screen saying that to “touch the SmartLink to continue”, and an arrow is pointing to the lower right corner. But due to the long distance between the screen and the touch area, it is really hard for users to discover it. I would suggest placing the touching area closer to the screen so that it will be easier to discover.