Introduction
During usability research, there could be many cognitive biases happened that finally violate the accuracy of the data. Both researchers and users can have biases, such as framing effect, confirmation bias, social desirability bias, etc (Subramanian, 2018). Here I would like to introduce one of the cognitive bias, called the Hawthorne effect.
What is the Hawthorne effect?
The Hawthorne effect was first discovered through a set of study in the 1920s by the Hawthorne plant of the Western Electric Company, Chicago(Stand, 2000). The studies were originally focusing on the relationship between the changing of environmental factors and the productivity of workers(Macefield, 2007). The Hawthorne effect was accidentally discovered through the studies of the lighting level(Snow, 1927).
In one study, workers were asked to repeat their tasks under three different lighting levels, and their productivity was evaluated. In another study, the workers in the control group were working under stable lighting and the experimental group under increasing lighting. The final result was surprising, under all these conditions the productivity was increased.
There are many interpretations of the result. The most widespread one was given by Mayo(1933). He argued that the increase in productivity was due to the test participants believed that the changes would improve their performance, and they were reinforced by the fact that they were being observed during the study.
However, other researchers such as Parsons(1974) questioned this interpretation. He believed that learning and feedback could better explain the phenomena. The workers used to have no feedback on how well they were doing, in this study, however, they could possibly set themselves higher targets and try to achieve them, and finally increase the productivity(Macefield, 2007).
Despite many other interpretations on the Hawthorne effect, Mayo’s opinion keeps a dominant position, although Parsons’s opinion is more recognized in the field(Macefield, 2007).
How might we overcome the potential Hawthorne effect?
It is hard for us to review the study and get a systematic conclusion right now since the environmental factors were not strictly monitored by the researchers(Stand, 2000). But we can still find out some methods to build a better relationship with tested users, relax them and improve the accuracy of the collected data. Oswald(2014) and other researchers introduced a way called “six-stage protocol” as a practical way. According to Oswald(2014), after the first five stages are finished before going into a conversation, the overall research result could be more reasonable. The six stages are shown in figure 1(Oswald, 2014, p.59).
Figure 1. Six stage protocol for Hawthorne effect mitigation
Stage 1. Gauge the person
In many cases, researchers are often recognized as strangers. It is hard for strangers to gauge a user, but it is important to do so. We need to gauge the type of users and the circumstance they are under. People tend to alter their behavior based on different circumstance, therefore, it could be a good idea to communicate with them under different circumstances.
Stage 2. Create a non-threatening perception
To dress in a casual way and look non-threatening could be helpful in creating a non-threatening perception. Additionally, it is also practical to adapt your behavior to the circumstance. For example, if the circumstance is more social, do not act too professionally. What’s more, figuring out a more proper way to observe the users is also helpful.
Stage 3. Introductions
For both researchers and users, it could be stressful to meet someone new, therefore, if the research could keep calm and relaxed, and introduce him/herself, it is more likely to build up a non-threatening environment.
Stage 4. Establishing rapport
Rapport happens between individuals when there is a harmonious understanding between them. Building rapport with your users is necessary to improve communication, build trust, and more importantly, improve the accuracy of data. We can achieve that when we:
- keep an open and accepting body language;
- maintain eye contact;
- nod and show an interested smile;
- try to agree with the user;
- call them by their name early in the conversation;
- be praising if appropriate.
Stage 5. Relaxed signal
It is important to find out the point when users are comfortable communicating with you. Typically, a great signal of getting to the point is a light-hearted comment or joke at the beginning of the section.
Stage 6. Link to the conversation area
Once the user is relaxed, it is time for the researcher to start the research.
Resources
Macefield, R. (2007). Usability studies and the Hawthorne Effect. Journal of usability studies, 2(3), 145-154.
Mayo, E. (1933). The human problems of an industrial civilization. New York: MacMillan.
Oswald, D., Sherratt, F. R. E. D., & Smith, S. I. M. O. N. (2014). Handling the Hawthorne effect: The challenges surrounding a participant observer. Review of social studies, 1(1), 53-73.
Snow, C. E. (1927). Research on industrial illumination: A discussion of the relation of illumination intensity to productive efficiency. The Tech Engineering News, 257-82.
Stand, J. (2000). The” Hawthorne effect”-what did the original Hawthorne studies actually show. Scand J Work Environ Health, 26(4), 363-367.
Subramanian, S.(2018, Jun 7) 10 cognitive biases to avoid in User Research (and how to avoid them). Retrieved from https://uxdesign.cc/10-cognitive-biases-to-avoid-in-user-research-and-how-to-avoid-them-993aa397c8c6