Introduction Data-informed design is the process of using usage
data to evaluate the effectiveness of user experience (UX) changes made to a
website. This process is also referred to as data-driven design, and it involves
using metrics to measure the impact of UX changes made to a website. While basic
traffic metrics such as overall page views or the number of unique users are
easy to track, they are often not useful for evaluating the impact of UX
changes.
The Importance of Using the Right Metrics To make the data-informed design work,
it is essential to use the right metrics. Basic traffic metrics such as overall
page views or a number of unique users are not suitable for evaluating the
impact of UX changes. These metrics are too general and do not relate directly
to the quality of the user experience or to the goals of a project. Therefore,
designers need to use more specific metrics that reflect the user experience and
the goals of the project.
HEART Framework A group of quantitative UX researchers at Google has
developed a couple of useful methods for choosing and defining appropriate
metrics that reflect the quality of user experience and the goals of a project.
One of these methods is the HEART framework, which suggests five categories of
metrics for evaluating UX changes. These categories are:
Happiness:
Measures user attitudes and is often collected via surveys. Examples of happiness
metrics include satisfaction, perceived ease of use, and net promoter score.
Engagement:
Measures the level of user involvement and is typically measured via behavioral
proxies such as frequency, intensity, or depth of interaction over some time
period. Examples of engagement metrics include the number of visits per user per
week or the number of photos uploaded per user per day.
Adoption:
Measures new users of a product or feature. Examples of adoption metrics include
the number of accounts created in the last seven days or the percentage of Gmail
users who use labels.
Retention:
Measures the rate at which existing users are returning. Examples of retention
metrics include how many active users from a given time period are still present
in some later time period.
Task Success:
Includes traditional behavioral metrics of user experience, such as efficiency,
effectiveness, and error rate. This category is most applicable to areas of a
product that are very task-focused, such as search or an upload flow.
How the HEART Framework Helps The HEART framework can
help in deciding which category of metrics to choose for a particular project.
For example, engagement may not be meaningful in an enterprise context where
users are expected to use the product every day as part of their work. In such a
case, a team may choose to focus more on happiness or task success. However, it
may still be meaningful to consider engagement with specific features of the
product as an indicator of their utility.
Goals-Signals-Metrics Process:
The Goals-Signals-Metrics process is a method that can be used to move from the
HEART categories to metrics that can be implemented and tracked. To use this
process, a small set of key metrics that everyone on the team cares about needs
to be identified. To figure out what these metrics are, the goals of the project
need to be identified. It can be difficult to articulate the goals of a project,
and the HEART categories are particularly useful in the discussion.
For example, at YouTube, one of the most important goals is in the engagement
category: to make users enjoy the videos they watch and keep discovering
more videos and channels they want to watch. Different projects or features may
have different goals than the product as a whole. For example, a key goal
for YouTube Search is in the task-success category: to make it easy for
users to find the videos or channels that are most relevant.
Common Pitfalls:
A common pitfall is to define goals in terms of existing metrics. For example, a
team may define its goal as increasing traffic to its site. However, increasing
traffic may not necessarily lead to an improved user experience or achieve the
project’s goals. It is important to focus on the metrics that are directly
related to the quality of the user experience and on the goals of the projects.
Another common pitfall is choosing too many metrics to track, which can lead to
information overload and difficulty identifying actionable insights. It is
important to choose a small set of key metrics that everyone on the team cares
about and can help to be tracked consistently over time.
Conclusion In summary, "data-informed design" is a process that uses usage data
to evaluate the effectiveness of user experience changes made to a website. To
make this procedure work, suitable metrics that represent the quality of the
user experience and the project’s goals should be used. The HEART framework and
the Goals-Signals-Metrics process are useful methods for choosing and defining
appropriate metrics.
The HEART framework suggests five categories of metrics for evaluating UX
changes:
Happiness
Engagement
Adoption
Retention
Task success
The Goals-Signals-Metrics process is a method that can be used to move from the
HEART categories to metrics that can be implemented and tracked. It is important to
avoid common pitfalls such as defining goals in terms of existing metrics and
choosing too many metrics to track. By using data-informed design, designers can
make data-driven decisions that can lead to a better user-experience and can help to
achieve the project’s goals.
