Usability Metrics
Measuring usability provides critical feedback on the effectiveness of a new web design, indicating whether it has truly enhanced the service or if the old design requires updates.
What is usability and why should it be measured?
Understanding usability is essential. It's often misconceived in its definition, given that it varies based on the user's specific needs and circumstances. Hence, usability should be viewed as a situational assessment, encompassing various characteristics (qualitative indicators). These characteristics are known as the dimensions of usability.
Measuring usability is crucial for assessing whether a website, online store, or any other information system meets its intended objectives. By identifying key activities to monitor and knowing where to look for insights, we can gain a comprehensive understanding of usability. Importantly, this approach offers a clear indicator of improvement, demonstrating whether changes in design have indeed resulted in enhanced usability.
Is measuring usability expensive?
There's a lot of talk around usability measurement focusing on the need for usability testing to observe how end-users actually interact with the platform. This is extremely crucial, but it can become quite expensive, especially if done frequently for a large-scale website.
On the other hand, statistical measurement occurs automatically. This allows for the convenient and cost-effective monitoring of user behavior and the early identification of potential issues.
Why isn't Google Analytics alone sufficient?
Traditional web analytics, like Google Analytics or Piwik, do provide information on the volume of searches and the keywords used, but further analysis tends to be limited.
Our solution, however, offers the following:
- A clear overview of the issues
- Guidelines for solving these problems
- Real-time measurement of the impact of changes
Do you truly understand how your users are experiencing your site?
There aren't many possible reasons why usability might be poor, but one thing is certain: poor usability almost always leaves its mark. Perhaps you'll recognize some of these signs immediately?
Negative feedback
Users are providing a lot of negative feedback
Incomplete processes
Users often abandon processes (e.g., form submissions) midway
Dead ends
Users frequently land on 404 error pages
Abandoned shopping carts
Users frequently abandon their purchases midway
Need for help
Users seek for help
Unconventional Journey
Users employ unusual methods to achieve their goals
How do we measure usability at Trinidad Wiseman?
Traditional web analytics (such as Google Analytics or Piwik) do provide information on the volume of searches and the keywords, but further analysis remains limited.
Our solution delivers a clear overview of site traffic, including specifics like the frequency of visits to a 404 error page. We delve into identifying usability issues and provide the opportunity to measure the effects of changes in real-time. This allows for a more dynamic and effective approach to enhancing user experience, focusing on targeted improvements based on concrete data.
To facilitate an easier overview, we have compiled essential indicators into a dashboard that provides a percentage-based summary of the usability issues encountered.
Our specially developed solution is configured on platforms like Google Tag Manager or Piwik Tag Manager and presents the results as a Power BI dashboard. This integration allows for a more streamlined and visual approach to understanding and addressing usability challenges.
On our dashboard, we track a variety of metrics that signify a poor user experience, showing the percentage of user sessions affected by any of these issues.
You can view the data on the dashboard over time and filter it based on language and search keywords. This functionality allows for a nuanced understanding of the user experience across different dimensions.
The dashboard is a valuable tool for website administrators, helping them to better tag and improve content, making it more easily discoverable and user-friendly. It also enables the comparison of enhancements over time, providing a clear perspective on how changes are influencing user experience. This assists in making informed decisions for optimizing the site's interface and overall usability.
Figure 1: Example of a Search Usability Dashboard 1
Figure 2: Example of a Search Usability Dashboard 2
Examples of Possible Usability Metrics
The metrics can be further developed and configured as needed. There's always the possibility to delve into more detail and customize the dashboard to suit the client's specific requirements.
- % of no search results
- % need for help
- % of broken pages
- % of unsubmitted forms
- % of page refreshes
- % of 404 page visits
- % of forgotten shopping carts
- shopping cart abandonment %
- % of useful content
- search filter usability %
How much does it cost to measure usability?
Currently, we offer four distinct modules for usability measurement. These modules are designed to identify user issues in several key areas: using search-related functions, navigating the website, interacting with various forms, and utilizing the shopping cart.
The setup for each module involves three stages: configuring tags, creating an API interface, and constructing the dashboard. Additionally, there is a monthly maintenance fee of 100 euros. As the scope of setup varies for each module, and each project has its unique requirements, we provide bespoke quotations tailored to the specific needs of each project.
Just as each module configuration is different in scope (e.g. the number of tags varies), each project is also different (e.g. the needs of the client, the complexity of the website) - therefore we can provide an accurate quote on a project-by-project basis. To give you an idea of how much a usability measurement project could cost, here is an example of an ongoing project.
Cost of a sample project
- We offered 3 different modules according to the content of the client's website: search, navigation and forms. The first 2 of these were selected by the client.
- The client's website is related to the tourism sector and visitors come there primarily to find information. Therefore, the user experience related to search and navigation is at the heart of their site.
- The search module was priced at € 5940 and the navigation module at € 6900. The difference in price was influenced by both the number of tags and their complexity - tags often need to be created by hand and require more work.
- Their monthly maintenance fee after setting up the modules is €100, which is a minimum, as the site does not have large amounts of data and is not a more complex site than average. There are no purchase options and no complex forms.
If you wish to discuss your project or have questions about the work being done at Trinidad Wiseman, please feel free to get in touch with us.
Thank you!
We’ve received your submission and we’ll get in touch with you as soon as possible.
Contact
Looking to measure the usability of your service? Contact our Chief Innovation Officer.
Hegle Sarapuu-Johanson
Chief Innovation Officer, Partner
Hegle is a co-founder of TWN and serves as our innovation manager, overseeing the development of our services and organization. With over 20 years of experience in user experience, service design, and information technology, she is one of the pioneers of UX in Estonia and possesses comprehensive skills and expertise across the entire project life cycle. Hegle's responsibilities include managing TWN's innovation process, exploring new service business models, adopting emerging technologies, and implementing new work methodologies.
