A Comprehensive Guide to Unmoderated User Research

Unmoderated user research, or self-testing, is a remote research tool that helps product teams gain valuable user feedback on product concepts, prototypes or websites.

Unmoderated sessions are qualitative, since you’re asking people their feelings towards a prototype, product, concept or idea. This type of research should be performed often throughout the product lifecycle – from the product discovery phase to post-launch feature enhancements.

When conducting an unmoderated research session, you should ask participants to review your prototype or website while thinking out loud. In order to conduct an unmoderated session, you will need to utilize a software that will send participants a link to your website or prototype. The software will instruct participants to complete a set of tasks or questions and record their responses. At PlaybookUX, we have this software feature and an artificial intelligence engine that automates the analysis of the participant videos so you can view the high level results rather than spend hours watching videos.

Unmoderated vs. Moderated User Research

Unlike moderated research, there isn’t a user researcher present in unmoderated studies. Moderated sessions are basically interviews, where the researcher will utilize a discussion guide to conduct the session. The research has the freedom to go off topic and ask follow up questions. With an unmoderated session, all of the questions, are submitted prior to the session and all participants answer the exact same questions. However, unmoderated sessions are less time consuming for researchers, so you can perform more of them.

Here’s what you need to know about preparing for an unmoderated study

As always, the first step is to determine what you want to test and understand your testing goals. The more specific the better.

Instead of testing the usability of your entire product in one session, test one specific part of the experience. Participants will get fatigued, so keep it condensed so they give you their full attention for the duration of the study. Also, by focusing on specific parts of your product or website, you can get very granular and uncover helpful details.

After, you determined what you are going to test, you need to recruit and screen participants. First, decide how many participants you need. We recommend 5 participants since it will be sufficient enough to uncover the majority of usability issues.

Next, select a user testing software to conduct the unmoderated sessions. Following that, you should create a list of tasks. Make sure to set the context of the study, which is setting the scene for your participants. For example, you may say, “imagine you’re in the market for new accounting software, and after searching, you arrive on the Quickbooks website.”

After you outline the context, it’s time to create the tasks. In our Quickbooks example, a good task to ask participants would be “What is your initial reaction to this pricing page?” Once prompted with this question, the participant will speak their thoughts out loud as they view the Quickbooks pricing page.

When the results of the unmoderated videos come back, you’ll have to synthesize your research. While analyzing your research, identify patterns at the individual and the group level. If you’re performing a usability study, see how people interacted with your prototype.

– Did participants feel that it was easy to interact with?

– Did they find your design visually appealing?

– Were they able to quickly find what they are looking for?

If three participants independently mentioned that they had a hard time finding the pricing page, then you should consider updating that design.

Lastly, for usability tasks, tally up the success and failures for usability tasks.

Analyzing the user research sessions is the most time-consuming part of performing unmoderated research.

At PlaybookUX we provide you with AI-generated user insights so you spend less time synthesizing research.

Advantages of unmoderated research

– You can gain a lot of valuable feedback in less time and with less effort exerted than a moderated or ethnographic research studies.
– It’s a great method for usability testing since participants will speak their thoughts out loud while walking through a prototype.
– It’s less expensive than moderated sessions because they’re shorter, typically lasting around 10-15 minutes.
– Since it’s performed remotely online, you have a wider geographic reach than in-person research

Limitations of unmoderated research

– Researchers don’t have as much control over the session. You’ll add your tasks at the beginning of the session but won’t be able to answer questions as they come up during the session.
– If the participant is confused, they can’t ask follow up questions as you can in other forms of qualitative research.
– If the participant is shy or isn’t very articulate, the user’s feedback will not be as valuable.
– The observer effect also comes into play, which states that people behave differently when they are being watched. Since you’re observing participants while they are completing tasks, they might not perform certain actions for fear of embarrassment.

General FAQ

What is unmoderated user research?

Unmoderated user research is the process of setting tasks for testers to walk through your website, while speaking their thoughts out loud. It doesn’t just stop at websites – it includes products, prototypes and concepts.

How do you conduct unmoderated user research?

Use a software, like PlaybookUX, that has the software to conduct unmoderated sessions. Then, write tasks for participants to walk through while viewing your prototype, website or product.