Guerrilla testing is one of the most cost-effective form of UX testing method to improve the quality of your product, and make sure people can use it for its intended purpose. Using guerrilla testing usually means going into a public place (e.g. coffee shop, train station, etc.) where you can find a relevant audience to validate critical ideas or assumptions about your product.
What is Guerrilla Testing?
You are going to your users, rather than having them come to you
American Civil War
Not a typical everyday word, “Guerrilla” was originally used in military context to describe a form of warfare strategy which confronts the enemy using a small mobile force to perform ambushes and hit-and-run tactics. In the world of UX, nothing lethal happens – but we do have enemies: time, money and resources. This concept was coined by designer Martin Belam a decade ago to describe a UX research technique as “the art of pouncing on lone people in cafes and public spaces, [then] quickly filming them whilst they use a website for a couple of minutes.” In essence, you are going to your users, rather than having them come to you. Lean and agile, this technique allows you to get insights in a little as a couple of hours, with as few as five users. It is usually identified with the following characteristics:
- Participants are not recruited but are approached by you
- Sessions are short (typically 10-15 minutes) and are structured around particular key research objectives
- Output is typically qualitative rather than quantitative. Testing helps to quickly validate how efficient design is on its intended audience or whether specific functionality works in the way it is supposed to
How do We Use It?
Where do we test
In general coffee shops and shopping centers are popular places because we can often find people from different age ranges and backgrounds. However, this should be really driven by what we are testing and the context we need. For instance, we might want to consider going to a retail store if you are testing a retail chain mobile app.
Who will Go
Usually one or two people from our team. More than three people might make our participants feel uncomfortable. Everyone involved in the project should go at some point, not only designers and researcher, but also developer, product manager, executives, etc.
what the process looks like
- Before actually approaching people in a coffee shop, we need to have a firm understanding of the scope and objectives of the user research with an organized test plan. It is critical to be specific on what we are looking for, because we know poorly planned and executed sessions would not provide reliable insights
- Next, instead of waiting in a random cafe with a sign, we pick a location where our target audience would spends their time,
- Run five sessions of test. The great news about guerrilla testing is, we only need to talk to as few as five users. Extensive research have arrived at this magic five users number which can help us uncover 85% of the core usability issues (See more about why 5 users are enough). The session itself is pretty straightforward:
- Set up and test the tools we are going to use. E.g. filming devices or screen recording softwares
- Approach a person, kindly ask for consent
- Give them a couple of tasks to do. Two or three tasks is about right for 10 to 15 minutes. Have participants to think aloud as they perform tasks, and always start with open-ended, non-leading questions (e.g. What do you make of this? How would you do that?). Let participants tell a story in which they explain how they perceive the product because we want to focus on their overall comprehension rather than basic task completion
- Thank our participant with a nice reward like a gift card, or buy them a drink or dessert
- After the sessions, capture important notes immediately, analyze the data and find patterns emerged
- Review and implement ASAP. Discovered issues should be improved immediately and validated during the next iteration
- Come back to the cafe to validate that users no longer face these issues
Why should We Go Guerrilla?
“There are no facts inside your building — get outside.”
— Steve Blank
Cost-effective and quick turnaround, especially for answering small-scale design questions
While traditional user research such as formal usability testing could break a bank in months of planning and execution, guerrilla tests are quick to set up and easy to perform. It is also relatively inexpensive so we can probably afford multiple tests on a regular basis, making it a habit. However, even clients who are not on a tight budgets should also consider performing guerrilla style research, because it’s “not just about saving money, it’s about saving valuable time (Levy, 2015)”.
People: Team lead on left; participant faces the wall; client on the right
Involve the wider team for greater immediacy and transparency
When we put key stakeholders in front of a real user, they will learn the good or bad news sooner rather than later, directly from the source. This helps whoever takes part in the project to get a better idea of the value of talking to users, and also potentially challenge some presumptions about users.
Provide high-level insights while testing hypotheses/assumptions during sprints
Whereas quantitative data collected from formal testing is likely to have more statistical validity in formal testing, the qualitative nature of guerrilla testing can provide improved design insight. Timely recap of feedbacks and insights can be completed on the same day and results can fed back into next iteration with minimal loss of signals.
Are there Limitations?
We might not get the right target audience
In guerrilla testing, it could be difficult to get the right cross-section of participants. We don’t get to curate who we talk to, ensure we are talking to the right people – for example making sure they are at the right age range, proficiency of technology, and interested in our product or service. We are largely lacking the context of people to know if they are our intended users.
People have limited time – we are competing for their attention as they go about their day
People are busy. There’s chance we might catch them in-between events, on their way from a meeting or to meet a friend. When we approach someone in the coffee shop, it will probably be rushed and we’re not going to get as much time from them as we planned. In such case people are probably going to be quick with their answers or not able to stick around for follow-up questions.
Better fit for validating ideas rather than creating or developing them
People’s attention wander quickly, and it will be even harder to sustain if the product or prototype being tested are lacking enough realistic details. To get people mentally involved, we usually need an well articulated design to have people feel that it’s “real” and not an experiment, even though we want to get out of the building to validate design assumptions as early as possible in the project’s life cycle.
Might not be appropriate for certain types of products
When domain-specific knowledge is required to use a product, such as completing specific tasks in financial or medical apps, guerrilla testing might provide less value because not everyone we talk to will have the corresponding knowledge and skills. Comparably, we wouldn’t expect it to work great if a specific environment is required to conduct testing (e.g., testing can be done only in a certain location).
Tips
1. Bring someone else with you
There a several reasons why you should pair up. You might want to split up responsibilities, so you don’t have to do it all on your own. For example, your partner(s) can recruit participants, take notes or record while you are engaging with the user. An additional person also means you won’t be worried about looking after your things (e.g. coat, bag, etc.). It will also be easier to recap and discuss the results with a partner immediately after the test sessions.
2. Build rapport and trust
Explain to people what’s going on and the implication of their participation. Be honest about who you are, what and why you are testing – you can prepare a “patter”, such as 3 minute formula introduction, to get it all out upfront as you approach people. Preparing a consent form will also help participants understand what they are doing and help establish more trust.
3. Take it with a grain of salt
Guerrilla is not detailed and rigorous enough to be the sole source of data. You don’t know how rush they are, you don’t know what their days been like, so you probably don’t want to take everything verbatim, and talk exclusively to people in one space (e.g. coffee shops). Instead, use it as a way to get a broad opinion and then commit to following up with other people who are more qualified and closer to the type of person that would use your product. Don’t just start making changes here in the coffee shop.
4. Test with new participants
While friends, families or colleagues are often the easiest way to find participants, they might be too close to you or your product to have fresh eyes, and the most important problems may not be as easy to spot for them. Therefore it is more valuable to test with new participants with minimal insider knowledge about you and your product.
5. Use quantitative data to inform planning
You may want to use Google Analytics, heat maps of A/B testing to help you spot trends – what is working well for you product and where improvements can be potentially made.
Reference
Levy, J. (2015). UX strategy : how to devise innovative digital products that people want (pp. 175–204). Sebastopol, Ca: O’reilly.
Marsh, S. (2018). User research: a practical guide to designing better products and services (pp.191-194). London, UK: Kogan Page.
Simon, D. P. (2017). The art of guerrilla usability testing. UX Booth. Retrieved from https://www.uxbooth.com/articles/the-art-of-guerrilla-usability-testing/?source=post_page—–c9a9fdc25a1e———————-
Belam, M. (2010). 10 tips for ‘ambush guerilla user testing’. Currybet. Retrieved from http://www.currybet.net/cbet_blog/2010/06/10-tips-for-ambush-guerilla-us.php
Pirker, M. (2016). 7 step guide to guerrilla usability testing: DIY usability testing method. Userbrain. Retrieved from https://userbrain.net/blog/7-step-guide-guerrilla-usability-testing-diy-usability-testing-method?source=post_page—–c9a9fdc25a1e———————-
Nielsen, J. (2000). Why you only need to test with 5 users. Nielsen Norman Group. Retrieved from https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
Babich, N. (2017). A guide to the art of guerrilla UX testing. Medium. Retrieved from https://medium.springboard.com/a-guide-to-the-art-of-guerrilla-ux-testing-69a1411d34fb
Adiseshiah, E.G. (2018). Guerrilla usability testing: how to introduce it in your next UX project. Usability Geek. Retrieved from https://usabilitygeek.com/guerrilla-usability-testing-how-to/
Budd, A. (n.d.). Guerrilla Usability Testing. UX London. Retrieved from http://uxlondon.s3.amazonaws.com/slides/UsabilityTesting.pdf