Usability Metrics You Need To Look Out For

There are many ways to judge a design. Its aesthetic qualities, its innovation, and other factors are all important. However, for user experience (UX) designers, few are as important as usability metrics. After all, UX design is all about what your users get out of it. Therefore, usability is the most important metric of all.

That said, there are multiple ways to gauge usability. In this guide, we’ll cover seven of the most important metrics and how to measure them. First, though, what are usability metrics?

What Are Usability Metrics?

A usability metric is a system of measurement to judge the efficiency, effectiveness, and satisfaction of users with a product. Depending on the exact metric, it can help you:

• Understand your product better.
• Make a Stop/Go decision before launching a new digital product.
• Assess your competitive position in the market.
• Track progress between releases to fine-tune your product.
• Create bonus plans for executives and stakeholders. 

Generally, you can calculate these metrics by running usability tests. This involves giving a pool of real users a task to complete. Meanwhile, researchers observe their behaviors and measure their actions. 

The resulting data are your usability metrics.

How To Measure Usability Metrics

Next, you need to know how to measure usability metrics. The first step is to know what you’re looking for.

Every usability test has goals, and you need to know what yours are before you start. Think about the features and user flows you want to test, then consider how to measure them.

Then, you can decide on the specific metrics you want to measure. Some of these are broad, like the types of metrics outlined in the section below. Others are more specific, like the percentage of time that users follow the optimal navigation path.

You can collect usability metrics for experienced and new users separately depending on the research. Generally, UX researchers use new users. This is because, as per Jakob’s Law, people don’t often spend enough time on one site to become an expert. There are, of course, exceptions to this rule. Particularly, software applications might have more experienced users.

Regardless, you’ll need a pool of users to test with. The Nielsen-Norman Group recommends 3-5 users for qualitative testing and a larger pool of 5-20 users for quantitative testing. Usability metrics are numerical data points, so you’ll need to go with the larger pool.

Then, you can narrow down the type of tests you want to carry out.

Types of Usability Metrics

There are four categories of usability testing metrics or four areas of your product you can test. You might collect data on more than one type, but organizing them this way can help you keep track.

1. Completion rates: These measure whether users can complete tasks effectively. The design can fail, succeed, or indirectly succeed.
2. Duration metrics: These measure the average time it takes a user to perform a task. Or they measure how long the user spends on a screen. This can tell you how complex your design is.
3. Error rates: These measure how often users make mistakes while using your product. They highlight places where users become confused.
4. Satisfaction metrics: These measure how satisfied users feel with your product.

Below, we’ll cover different metrics that fall into these categories.

7 Usability Metrics To Measure

There are tons of ways to test usability. Here are seven common metrics you can use to score your product and make improvements.

1. Success Score

In terms of completion rates, the success score is a key metric. In a nutshell, it tells you if your design works and if users can successfully complete a task. 

The basic formula is this:

Success score = number of completed tasks ÷ number of attempts

Task success refers to the user fully completing a task. Partial completion must go down as a failure. For example, let’s say the task is to buy a medium white T-shirt. If the user gets the wrong color, it’s a failure.

That said, collecting data on these failures is still useful. It provides a more nuanced picture of where users are going wrong with your design. By collecting ‘partial success’ as a separate group, you can start to understand where the problem lies. 

Using the above formula, you can calculate a percentage between 0 and 100%. For reference, a good success rate is around 78%. 

2. Error Rate

Conversely, you can test the error rates for a particular task. An error is any wrong action during the completion of a task. You can divide these into:

• Slips: Errors made with the right goal, such as a typo when entering your name into a form
• Mistakes: Errors made with the wrong goal, such as entering your first name into a form that requests your surname

You can measure errors in two ways, too. The first is the error rate, which measures all of the errors. This involves tallying up the total number of errors (including repetitive ones) and dividing it by attempts. The formula is:

Error rate = number of errors ÷ number of attempts

The second is the error occurrence rate, which focuses on a specific error. The formula for that is as follows:

Error occurrence rate = total number of errors ÷ total number of possible errors

For this, you need to define all possible slips and mistakes that you’ll check for in the task. That’s the second number in the formula, which you’ll use to divide the number of errors the user makes.

Measuring errors is a common form of user testing. However, it’s important to note that no digital product will be entirely error-free. After all, your users are human, and humans make mistakes.

3. Task Time

With great UX, users should be able to perform a task pretty quickly. Tracking the task time is one way to determine this. While this sounds like a simple metric, you need to know how to use it properly.

The formula you’ll need is this:

Task time = Time of the first user + Time of the Second User (etc.) ÷ Total number of users

This calculates an average time to complete the task. But, with this number, how do you know if the result is good? After all, it depends on the complexity of the task.

You can use Keystroke-Level Modeling (KLM) to add up the average times for each action taken. This includes small actions such as pointing the mouse and clicking. That way, you can calculate the time precisely. When you see hesitations, that means there are points of friction.

Usually, time metrics compare newer versions of digital products with their older counterparts or with competitors. 

The difference might seem miniscule, but every millisecond counts for your users. After all, according to data, one in four users would leave a site that failed to load after four seconds. 

4. Efficiency

There are a lot of ways to measure efficiency. Time-based efficiency is the most common, combining task time and success score. However, even this formula is more complicated than anything we’ve seen so far!

Essentially, this metric combines the task time and success score. You can see the formula in the image above, where:

• R = number of users
• N = number of tasks
• n_ij = result of coming through scenario i by respondent j (1 or 0)
• t_ij = time spent by respondent j to come through scenario I

To dive into statistics like this, it’s worth reading up on metrics and how to calculate them.

5. Single Ease Question

A single ease question (SEQ) is a more opinion-based metric. However, it’s startlingly simple. The researcher assigns the user a single question following the test. This usually involves a numbered scale from 1-7, where the user rates the task difficulty.

The SEQ is subjective, of course. But, since UX comes down to your users, it’s good to collect these subjective opinions, too. Combine these with your other data points to get a really robust answer to your research question.

For example, the user might have taken a longer time to complete a task. However, if they didn’t perceive it that way, does it really matter?

6. Task Level Satisfaction

Task-level satisfaction involves asking users directly how satisfied they are with the task. This involves a questionnaire of 3-5 questions. Like the above examples, they should not be open-ended questions and should provide a scale for users to answer.

You can use some standardized questions, such as:

• After-scenario questionnaire (ASQ): Three questions that measure satisfaction with time, support, and ease of use.
• Usability magnitude estimation (UME): Users assign ratings to tasks in just one question.

You can expand this, too, by assessing test-level satisfaction.

7. System Usability Scale

The System Usability Scale (SUS) is similar to the SEQ, except with more questions. Here, you ask 10 questions, and the user rates them on a scale of 1-5 (Strong Disagree to Strongly Agree). Each question is worth 10 points. Based on this, the product receives a score out of 100.

Here are the standard questions:

1. I think that I would like to use this product frequently.
2. I found the product unnecessarily complex.
3. I thought the product was easy to use.
4. I think that I would need technical support to be able to use this product.
5. I found the various functions in this product were well-integrated.
6. I thought there was too much inconsistency in the product.
7. I imagine that most people would learn to use this product very quickly.
8. I found the product awkward to use.
9. I felt very confident using the product.
10. I needed to learn a lot of things before I could get going with this product.

The average SUS score is 68 points, while results over 80 are ‘excellent.’ You can use this quite effectively to compare with other products.

Why Are Usability Metrics Important?

Usability testing is important for many reasons. Primarily, it can provide actionable insights to help you build great products.

Here are some of the ways that these metrics can help you.

Improve User Experience

First and foremost, user experience testing helps you to improve your product. The end goal is to make something usable—and enjoyable—for the user.

Usability metrics like efficiency can help you identify points of friction. That way, you can hone your product until you have something that’s satisfying for your users. Data-driven design is extremely helpful in this way.

Get Past the Aesthetic-Usability Effect

Usability testing itself is important. However, users can fall into a trap of assuming a product is good just because it looks good. The aesthetic-usability effect is a cognitive bias that happens in these cases.

Specific metrics can help you remain more objective, preventing these cognitive biases and subjectivity.

Business-Focused Product Improvements

Although your users are at the heart of everything you do, you still have a business to run. Usability metrics can help you see where you’re at with your business goals. 

For example, let’s say you’re trying to increase conversion rates. You conduct a usability test, and find that users spent too long searching for the ‘sign-up’ button. You also found that a high percentage of users abandoned the task.

When you know this, you can work on fixing it.

Involving Stakeholders

Many people participate in creating a digital product, and not all of them grasp UX. Hard data, like usability metrics, can help you explain things to non-UX stakeholders.

You can effectively show what is going wrong with statistics to back it up. That way, you can secure the necessary time and funding you need to rectify the problem. 

Tracking Progress

UX is iterative, and that’s a fact. You’ll often need to work on improving your product. Usability metrics mean that you can easily compare a product’s previous design to its new one. 

Conducting frequent tests throughout construction means that you can track your progress all the way. Again, this is useful for demonstrating your work to stakeholders. However, it’s also useful for making sure you’re on the right track.

Gaining Competitive Advantage

According to Maze, 83% of respondents to a survey think testing should happen at every stage of the product lifecycle. However, 78% said their companies didn’t research enough.

In other words, there’s a fantastic opportunity here to test more. The results could be astounding, giving you an incredible edge over your competitors. 

Make Great Products With Page Flows

Hopefully, you now have a better idea of how usability metrics work. However, you can’t just rely on numbers. UX design is also about creativity and innovation. So, why not learn from proven products?

Page Flows is a helpful resource for finding UX design ideas. Get started today to access our growing library of user flow recordings and finally stay up-to-date with current design trends.