11 Tips for Creating Effective Screener Questions
You may have heard how screener questions are integral to finding the right audience for user testing and gathering user insights that help you build habit-forming products. After spending a significant amount of time recruiting and reviewing candidates for user research sessions, you may be wondering, what goes into crafting a good screener and how can I find the right participants every time?
What are screener questions and why do you need them?
A screener question is a way to weed out participants who are not qualified for your study. You should screen participants when you’re looking to gain valuable user insights for all types of qualitative and quantitative research including moderated interviews, unmoderated sessions, surveys, contextual studies etc. It’s important to ask the right questions because asking bad screener questions can qualify the wrong participant and having an unqualified participant in your study will skew the outcome and produce poor user insights.
7 Types of Screener Questions
1. Demographics: age, gender, ethnicity, location
2. Firmographics: job role, company size, employment status
3. Experience level with specific software & tools: An example question is: Do you have experience with any of these tools? Sketch, Invision, Omnigraffle, Figma, Framer.
4. The frequency of use: An example question is: How often do you clean your clothes? Once a day, Once a week, Once a month, Other, Never.
5. Tech Savviness: An example question is: What is your web expertise? Beginner, Average or Advanced.
6. Device Types: What type of computer, tablet or phone do you use?
Screeners in B2B and B2C Research
When conducting research for B2C services or products, you should ask different types of screener questions than if you are conducting research for B2B services or products since you’ll gather very different user insights. If you are working on a B2C product, your screener questions will typically be focused on your user’s demographics, their level of tech savviness, decision maker status, and how often they use certain products.
If you are conducting research for B2B products, your screener questions will be focused on firmographic questions, software usage and decision-making authority within their company.
11 Tips for Writing Screener Questions
1. Keep it short
Don’t write screeners with more than 5 questions. Participants will be disengaged and quit in the middle of it. When recruiting niche demographics, you should make sure users aren’t fatigued with the initial screener. You may accidentally screen out a great candidate if you are asking too many difficult questions. Also, if a participant is disqualified after a long screener they’ll be frustrated because they invested a significant amount of time answering questions but they never received an incentive for it.
2. Start broad and get narrower
Your first question should be broad, with the following questions becoming increasingly more specific.
3. Stay away from the binary Yes / No questions
Instead, use multiple choice questions. With binary questions, participants have a 50% chance of qualifying, which decreases the credibility of the study. Instead of asking: Do you work in market research? You should ask: Do you work in any of the following fields? Finance, Market Research, Technology, Healthcare, Food & Beverage, None of the above.
4. Do not ask leading questions
Answers chosen from a list are more reliable. Instead of asking: On a scale of 1-5, how much do you love using Windex? You should ask: On a scale of 1-5, how do you feel about the Windex?
5. Ask qualifying questions at the beginning of the survey
However, if your budget allows, use a robust recruiting software where you can filter down by demographics first. At PlaybookUX, we have a matching algorithm that narrows down to the target population first, so you don’t need to send participants unnecessary demographic & firmographic screener questions and you can be sure you are matched with the right participants.
6. Stay away from jargon and use simple language
We don’t want to confuse or trick participants. Often companies use acronyms as if they are commonplace. Try avoiding any words above a 9th-grade reading level, to avoid confusion. When you send questions to a group of participants, you don’t have an opportunity to clarify, so it’s important to write clear and concise questions.
7. Be very specific when defining frequencies
Asking questions with answer choices such as often, rarely, and sometimes is confusing. This is incredibly subjective. Someone may think “often” is defined as once a week, but you may think “often” is every day. Instead of asking: How frequently do you use Slack? Often, Rarely, Sometimes. Try asking: How frequently do you use Slack? Daily, Weekly, Monthly, Never .
8. Utilize Other, I don’t know and/or None of the above
Make sure to include these as answer options when a list isn’t all-encompassing. This will yield better results since participants won’t select inaccurate answers to proceed to the next question.
9. Provide clear distinct answers that don’t overlap
This will prevent confusion. Instead of asking: What’s your age? 20-40, 40-60, 60+. If this is the case and you are 40 years old or 60 years old, you don’t have a clear answer choice. Should you select the first answer range or the second one? Try asking: What’s your age? 20-39, 40-59, 60+.
10. Limit the number of open-ended questions you ask
Open-ended questions cause participant fatigue. The researcher has to read hundreds of long-form questions and manually determine who qualifies or disqualifies for a study. If you ask multiple open-ended questions, you will have a lot of participant responses to review prior to even starting your research. However, if you use a comprehensive user testing software that automates the qualify and disqualifies process based on the participant’s answers, it will save a lot of time.
11. Include a non-disclosure agreement (NDA)
It’s a good idea to add this as the last screener question. If you are working on a project that is confidential or hasn’t been released to the public yet, you should include an NDA to ensure the participant doesn’t reveal any exciting new features before you are able to make the launch announcement.
Screeners are just one of the many steps in selecting qualified participants who will give you valuable user insights.
Start getting user feedback today
