But how to analyze the results of these tests? How to know in which criteria and heuristics the system needs to be improved? What if the team is unable to turn the results obtained into concrete data? How to convey user feedback clearly to the customer, stakeholders and the rest of the team? To answer this and other questions that may arise in the process, we use the System Usability Scale.
Created in 1986 by John Brooke, the System Usability Scale is a scale that measures the usability of an interface. It is recommended that it be applied together with the usability test, regardless of the adopted format. In this way, in addition to obtaining impressions about the interface, analyzing whether the tasks were carried out with ease and efficiency, detecting whether mistakes were made, among other factors, one also obtains quantitative data that, when analyzed together with the results of the test of usability, result in a complete and quantifiable research. The System Usability Scale was built with the purpose of measuring 3 main aspects of a system.
✅ Effectiveness: Are users successful in using the product? Can you complete your goals?
⌛ Efficiency: How much effort does it take to use the product?
😃 Satisfaction: Was the user experience satisfactory?
According to Brooke, the usability of an application is a very particular factor, which must be adequate to the context in which it is found, for this reason it can be difficult to measure it directly. For him, it is impossible to really define the usability of a system without having well defined the users for whom the system is intended, the tasks that these users will perform and the environment in which the system will be used. Likewise, the way to measure the usability of a system also depends on a specific context. Therefore, the System Usability Scale (abbreviated as SUS), shows itself as a tool to more assertively measure the subjective perceptions obtained in the usability test, being a means that "translates" to a quantitative scale what is already known about the interface. One of the main goals of this questionnaire is to be practical and quick. Still, each step was planned with care. In the article 'SUS: A retrospective (2013)', John Brooke points out that over its 25 years of existence, the System Usability Scale has been referenced about 1200 times in scientific articles as being a suitable method for the industry, as well as has proven effective for a wide range of systems, from the simplest to the most complex applications, when used correctly. There are case studies that use SUS to evaluate the user experience in interfaces such as Microsoft Teams, Gitlab, and even programming languages such as Scratch. With these examples, we also perceive the impact of SUS on a global scale, as the application of this questionnaire is explored in several countries by researchers and specialists in the area. The questionnaire consists of 10 statements. For each of them, the user must respond on a Likert scale that has parameters numbered from 1 to 5 — 1 means Strongly Disagree and 5 means Strongly Agree.
The statements have an alternation between positive and negative, preventing the user from responding in a biased way. In addition, we can point out relationships between Nielsen's Heuristics and SUS questions, emphasizing the strategic importance of applying the questionnaire. Nielsen's Heuristics are 10 general principles that guide interface and interaction design, being considered a set of quality criteria developed by Jakob Nielsen, computer scientist and founder of the Nielsen Norman Group.
Below we list the Heuristics and their respective SUS questions:
How to calculate the results:
Something that should be clarified is that, despite the appearance, the final result is not a percentage, but an absolute number. This is because for Brooke, the scale from 0 to 100 is more expressive, catching the attention and facilitating the perception of project managers, the product team and engineers. In addition, he wanted to make sure that the differences in the results of each test were noticeable, which would not be the case if a smaller scale was used. The results can, finally, be grouped into intervals, where scores below 60 are considered severe, and above 70 acceptable.
The average of SUS points is 68, if your results are lower it is an indication that your product may have usability problems. The advantage is that there are already many options that automatically apply and calculate the result of the SUS, such as the SUS Calculator. Note that this method will not point out or say what is wrong with the interface, but rather support the impressions obtained in the usability test, making them more tangible.
We can say that a user did not notice that a certain button was clickable, or that another user thought that the system colors were too saturated, so they were unmotivated to complete the tasks proposed in the test. Using the support SUS, it is possible to state that the usability of this system is around 70 points, which suggests that improvements must be made to improve the user experience. Some of the advantages of using System Usability Scale are: It's an easy-to-manage scale You can use it with a small or large sample of users Provides an overall measure of satisfaction with the system Makes the level of usability of a system tangible and measurable It's cheap, you don't need a lot of resources to do it Is fast It has theoretical and practical support in the market In summary, this type of scale is useful because it is a method capable of measuring the usability of an interface leaving the abstract and uncertain field. For example, when a screen is considered “beautiful” or “pleasant”, we use subjective concepts, which can vary between individuals.
However, if a screen was considered easy to learn or satisfactory by 93% of users, we have measurable data that is comparable with other interfaces or systems. Therefore, using SUS is a great aid to support usability tests, interviews and other evaluations carried out with users. At Vetta, design is one of the pillars of digital transformation in technology solutions. Do you want our help to design your next solutions and innovations? Get in touch with us.
