8 Common User Testing Myths that Prevent You from UX Testing

Although user testing has existed for a while, there are still several user testing myths around it. Many companies still believe that you can conduct user research after developing a product. Some feel that they can conduct a user test only after developing a final product because it is more important to get the product on the market. Others think that you need lots of cash to hire a separate department to take care of UX testing. 

These types of UX testing myths stem from a time when the digital world was seen as cumbersome. Picture the internet’s early days. You couldn’t easily collect or showcase data. So, usability testing as a digital process turned into a baffling concept that many did not want to try out. However, as technology continued to become more agile, these misconceptions are now outdated. So, let’s expose the top UX myths so that you start UX user testing.

Myth #1. User testing is artificial

Myth description: Many businesses think that UX testing methods encourage participants to perform unrealistic tasks in a lab. Since these are tasks that the participants would not normally do and the environment is artificial, you cannot apply your findings to real life.

Truth: You can make user testing more natural

Truth description: It is true that moderated or unmoderated user tests do not always take place in a natural setting. But this type of user testing myth can be discarded as you can make the test results more meaningful by giving users tasks and questions which are more personal to them. For example, if you want to discover whether a woman who earns over a certain amount would like to fill out a journal every day, you recruit that type of user for your study. UX testing helps you observe how people will try to use your product. 

In contrast, a field study can help you gather more information about an individual’s behaviour. It can also help you collect more information about a customer’s real-life duties and the environment in which they will use your product. For instance, if you wanted to observe how a participant used your shopping app in their life, you could set them tasks over a period of a month. These tasks could involve detailing when they shopped, the number of individuals shopped for and what they usually bought. You could ask them to write down their reasoning behind buying certain products i.e. did they buy some products because they were cheaper? 

Remember to check-in at the end of every week with them to check that the study is still on track. At the end of the month, schedule a call with them so you can discuss any questions you have after reading through their record. It is also worth focusing on fewer tasks and questions so that the user spends an ample amount of time on all you require without trying to rush through the study. Follow this guide if you need more assistance with your user scenarios.

 Myth #2. UX testing doesn’t bring much value

Myth description: Another one of the user testing myths examples is that UX testing cannot bring much value if a business has a leadership team who have years of expertise in our product or service area. This question is one that many brands will ask as they don’t see the value in user testing. Furthermore, do customers know what they want?

Truth: Participants will reveal the faults of your product that internal users will not see

Truth description: If you are a superuser of a product or service area at your company, you are unlikely to realise a feature is difficult to use or does not make sense. In other words, this type of bias means that you will fail to see any issues with your product. Also, other individuals in your company might also be prone to bias. When asked for feedback, they could follow what their other senior colleagues said due to the fear of being judged. 

On the other hand, participants won’t be afraid of telling you what works and what does not. Their UX testing feedback will show you the problems with your product or service. If you’re lucky, some users might also tell you how to improve the usability of your product. For example, if you conduct a brand perception unmoderated or moderated test, see where you fall short by asking them how they would improve your business. In this way, you can focus on eradicating these issues before wasting time and money on coding and marketing an impractical product. In other words, a more useful product gives you a higher ROI. 

Myth #3. You need at least 50 users to test on

Myth description: Another user testing myth involves the theory that there is a magic formula for the right number of participants per test. Others believe that the results from a user test will only be useful with a higher number of participants. 

Truth: The number of participants depends on the type of test

Truth description: The number of participants you need to recruit for UX testing depends on the type of user test you’re doing. Need to test your design to find any issues? Choose five or six participants at a time over several rounds of testing. But, if you are more interested in the metrics collected from analysing the usefulness of a website or app interface, opt for a more significant number of participants.

In addition, when you are first testing out an idea or a concept, you might want a larger number of participants for a user test as you don’t know who your target audience is yet. A wider range of participants will help you determine your audience once the final product or service is developed. However, when you are testing the usability of a product or a service, five or six participants who meet your target users are sufficient. 

Myth #4. User testing is expensive

Myth description: Brands can see conducting user research as an additional expense. Some might not see it as a necessary part of a project if they are confident that a product will sell. Many also feel that you need to carry out user testing in a lab with a moderator which can also be expensive.

Truth: The cost of not meeting your user’s needs is higher

Truth description: This user testing myth can be banished very easily with the right question. Rather than asking how expensive user research will be, focus on whether this money is spent well. What are the consequences of designing a product without user knowledge? Ask yourself whether you are designing a product or service that a user needs. Higher costs and other adverse effects as a result of a poorly designed solution. 

Additionally, if your budget is low, try remote or unmoderated user testing, this involves getting your participants to think aloud as they use your product or service. Take a look here for guidance on choosing the right remote user testing tool for your needs.

Myth #5. UX testing takes too much time

Myth description: Businesses often want to launch a product or service. In this way, they can still get their return on the investment into a product or service quicker. The user testing myth here revolves around the idea that user research is an optional step. It can waste time if brands are confident that their ideas are profitable. 

Truth: Usability testing makes the process of going live faster and smoother

Truth description: When used well, user testing can provide you with valuable insights that you cannot get in any other way. It will not be difficult to do and will improve the launch process. UX testing will tell you what users find confusing about your product so that you can fix it before release. Not doing user research will cost you in the long term as once a user has experienced a wrong product, it will be hard to get them back on your side. 

Furthermore, gone are the days where user testing could take up the whole day or weeks. Remote testing tools can save you a lot of time when UX testing by helping you recruit your target users, analysing the data received and offering your customer support should you get stuck.   

Myth #6. You must be a professional to conduct UX testing

Myth description: It’s easy for companies to put off user testing if they feel that only a specialist can conduct UX testing. When you think like this, it can be challenging to prioritise user testing at the early stages of your business, especially if you feel you lack the budget for it. This is another example of a user testing myth as a business feels that they can only conduct user testing once they have the money to hire a UX professional. 

Truth: There are a lot of tools that simplify the UX testing process so that anyone can conduct user testing

Truth description: While you should read articles like ours to understand user testing, remote user testing tools can also help. These were built to cater for a wide range of users, not just user testing professionals. And if you make a mistake initially, you can try again. 

For instance, PlaybookUX is an international UX tool that allows you to conduct moderated or unmoderated tests with A.I. analytics and verified participants. Other remote tools like Usability Hub also allow you to conduct a five-second test of your design so you can quickly analyse a user’s first impression of your design. For a broader list of UX tools and its benefits like screener questions and video annotations, click here.

Myth #7. You should conduct UX testing at the end of a project

Myth description: Some businesses leave user testing until the end as they feel that you need a fully-fledged product or service to test. The idea behind this user testing myth is that you can’t test a prototype until you can check all its uses. Once a final product is ready, you can get a more realistic reaction from participants.

Truth: Test early and as many times as possible

Truth description: Testing should happen at various stages during the design process. Designers follow this process so know that they are going in the right direction when creating a paper prototype, low-fidelity prototype and then a high-fidelity prototype. Additionally, testing right before launch does not give you adequate time to fix any problems. 

Moreover, user testing can double your conversion rates for your product or service. This process can show you which parts of your design frustrates your target users, what confuses them and what prevents them from purchasing anything. Brands should also want a great experience for their customers, not just another product or service. User testing helps you get fresh, personalised perspectives from customers who know what they need and aren’t afraid to tell you!    

Myth #8. UX testing is all about the metrics

Myth description: Many consider user testing to be valuable only if they gather quantitative metrics that impress stakeholders. These include time on task, system usability score and user satisfaction ratings. This type of user testing myths values quantitative over qualitative testing. 

Truth: Metrics only show you the how not the why

Truth description: While quantitative data can show you that users had difficulty completing a task associated with your product or service, it cannot show you why. For example, a quantitative-based metric such as time-on-task could tell you how much time a user spent on completing a task during a study. But, other than this number, the metric could not tell you why a user spent this amount of time on a task. Only qualitative UX testing could get you the correct reasoning behind this number from the user. 

In other words, you cannot improve your product only with metrics-based data. It would help if you combined metrics with qualitative testing that asks participants what issues they have with your product and why. In some cases, you can also ask them how they would improve your product. 

Key Takeaways

Banish your beliefs in user testing myths and start putting your customers first. Connect with them before, during and after launch, so you always understand their needs. User testing is the key to becoming more customers centric and so set aside money and resources for it. Then share these UX testing insights with the rest of your team to give customers what they want.