Presentation of the project

Initial overview of the project before presenting our design documents.

What are heatmaps ? 

Heatmaps are a set of visual tools that help understand where users are focusing their attention on a web page, there are several types of Heatmaps, this project exclusively focuses on Clickmaps, which track users clicks on a given page.

What do people use it for ?

Heatmaps are usually used by product teams to help inform product decisions, for example, Heatmaps can help best understand where to place their main call to action on their landing page to improve conversion.

Why did we want to build this product ? 

Heatmaps are part of Datadog's Real User Monitoring (RUM) product, the RUM product was built for developers. As RUM grew, we wanted to diversify and allow product teams to exploit RUM data. These less tech-savvy users need an easier way to access data without needing to learn the Datadog query system or the structure of their data. We thought that Heatmaps would be a good tool to fit to for these personas. 

What's the overall strategy this fit into ? 

Heatmaps are just part of a Product Analytics tool set that Datadog is developing (as of jan 2023) to open its platform to new personas which are much more product focused rather than engineering focused.

Our design plans

A brief overview of all the documents and research that we ran before starting work on prototypes, it is very important for us to make sure that we know what problem we want to solve and who we are solving it for.

All projects will have an MVP defined and a design/product brief written based on their specific needs. Design exercises will be conducted accordingly, with a focus on competitive analysis, user stories, and target persona definition in the case of Heatmaps.

Defining our Minimum viable product (MVP)

This MVP takes into account the fact that Datadog already has a session replay product, which means that we already collect mouse position, clicks. scrolls of our users on a given webpage, the features that need to be aded to this already existing product to build a heatmap are the following:

• Data Collection: The ability to collect click target data (i.e. where the user clicked)

• Heatmap Generation: The ability to aggregate click data to visualize the user behavior patterns on a given page.

• The ability for the user to filter the data based on parameters such as screen size, device type, browser, etc…

• The ability to link aggregated click data to specific user sessions.

Competitive analysis

All the resources that we review for this research are public, we do not create accounts on our competitors' platform to gain this information. The main players in therms of heatmaps that we selected were the following: 

• HotJar

• Full Story

• Crazy Egg

• Content Square

• Mouseflow

• Lucky Orange

• Ptengine

The analysis was based on videos of the products, as well as documentation, to understand the features and functionalities that were covered by these products. After documenting our findings, we used these to help with brainstorming as well as understanding the needs of the users using these heatmaps products.

Personas and Users stories

We combined the insights from Datadog's product managers with our own research to develop user stories and personas. These two resources help us focus on the key users need while designing the product, we notably exploit them to define our MVP."

Product managers

The Product manager understands customer needs, brings products to market, communicates benefits and measure the performance of the product.

User story

As a product manager, I aim to make data-driven decisions to enhance my website. Heat maps help me achieve this as I can identify which parts of my website are receiving the most attention and which ones need improvement. They also help me save time by analyzing data visually and presenting engagement data graphically.

These actions will enable me to make better data-driven product decisions. Sharing a heat map can also help effectively communicate engagement information to key stakeholders.

Persona

User goals: The primary goal of product managers is to identify new opportunities for growth and to drive the product roadmap towards that direction. They need to prioritize shipping new features that bring value to their customers, which helps to improve customer buy-in and satisfaction with their products.

User needs:

• Extract actionable data from product usage.

• Independently research the market and competition.

• Understand the impact that changes have on their customer's experience.

• Have the Ability to share data about product usage.

• Get in touch with customers and extract valuable product feedback.

Pain points

• Unclear requirement from key stakeholders and customers.

• Lack of actionable product usage data to justify business/design decisions.

• Unclear Forecasting and delivery timelines.

• Changing priorities during spring.

• Overwhelming product backlog. 

• Decision fatigue.

Tools usually needed by this role

• Workspaces (Gdocs, Notion, Confluence)

• Project Management Tools (Jira, Asana)

• Presentation Tools (Gsheets, Powerpoint)

UX Researchers / Designers

Understands the customer needs, thinks new features, designs such features, handoff to developers.

User story

As a Product designer, I aim to understand user behavior and improve the overall usability of my applications. To achieve this heat maps help me see changes in user behavior across different device types, compare different versions of my site, and understand where users are focusing the most of their attention or getting frustrated. This information helps me understand if a change improved for the user experience, and helps me ensure that design patterns are being understood and used as expected.

Persona

Goals: Product designers help with product planning as well as defining product specifications. They also Conduct research to gain a deep understanding of the target user's needs. They work closely with both the product team and engineering teams, and aim to design interfaces that are appealing, consistent, and usable.

User Needs

• Understand how their customers are using the product, in practice this means: 

• Qualitative data, to understand specific user interactions with their product.

• Quantitative data to help understand product wide usage patterns.

Pain points

• Users don’t always know what they need to fix a problem (might be different from what they want).

• It’s hard to get visibility on what is causing frustration.

• No easy way to translate user behavior into actionable data.

• Lack of quantitative usage data.

• Decision fatigue

Other Tools

• Prototyping tools (Figma, sketch)

• Whiteboarding tools (Figjam, Mural, Miro)

• Project Management Tools (Jira,Asana)

• Workspaces (Gdocs, Notion, Confluence)

Frontend engineers

Their role is build and maintain the frontend part of their product, the interface, they are very skilled on the technical side.

User story

As a frontend engineer, I want to find bugs that are impacting customers and causing them to lose attention. Additionally, I want to understand the product's pain points from a technical perspective, and see if the frustration of users is linked to technical limitations or load.

Heat maps will help me make better decisions regarding prioritization and bug fixing, also see the impact of technical changes on the customer experience.

Persona

Goals: General goals for software engineers, no matter their specialization:

• Translate product needs into technical requirements.

• Estimate the time it will take to build such product, and prioritize.

• Develop and maintain the products that they own.

Goals: Goals specific to frontend engineers :

• Implement compelling, usable UIs, Contribute to their design.

• Improve website performance and address interface scalability limits.

User Needs

• Test features/user flows in the interfaces that they developed.

• Observability over the parts of the frontend app that they own over different key areas:

  • Application Performance

  • Application Usage

  • Errors generated by the application

• Be alerted if a part of a website suddenly becomes inaccessible to users.

• Have the ability to reproduce and troubleshoot errors, gather context around such errors

Paint Points

• Lack of visibility, complex site architecture can makes it hard to find root causes.

• Poorly organized and maintained projects, scope creep.

• Lack of requirements.

Tools usually needed by this role

• Version control systems (GitHub, GitLab)

• Chrome Developer Tools

• Integrated development environments (VSC)

Prototyping

Once we had defined who was our target and why heatmaps would help them achieve their goals we started working on low-fi and then high-fi prototypes.

In this section we will present what we believe is a mature solution to help our customers, the content that you see below is pulled from a design review to help align all major stakeholders.

Please fine below a video commentary of the prototype, each component is presented in more details in the following paragraphs

Heatmap header

The Heatmap header is split in two areas, the top section is used to select what data that we want to display, the bottom section is used to filter out this data, you can see a full overview below:

The bottom section contains filters that modify the data displayed on the heatmap. We have created individual dropdows for Browser type, Country, and Device size filters as all of our competitors use these. We think our customers will want to access them easily without needing to open the full filter menu.

Filters menu

The filters section contains what we call facets (essentially metadata that we use to filter events) to keep consistency with the rest of the platform. this also still allow customers to filter data on any of the facets they might be interested in.  We will organize facets to display the most relevant filters first based on our competitive analysis. The pill on the filter button allows users to know at a glance if facets are actively filtering out data.

Side panel content

These panels works hand in hand with the map itself, when an hovering an element on the map we will highlight in the panels, the opposite is true as well. this behaviour helps build a clear visual connexion between the panels and the map. The Page Side Panel will display page-wide data, while the Action Side Panel allows customers to drill down into a specific event that was registered on the page.

The Page Side Panel (left)

With this panel we want to put heatmap data in context to help our target personas make decisions. The to section of the panel enables our customers to understand at a glance the relevance of this page compared to the rest of the application, They should be able to quickly find: 

• The Importance of the page, i.e. how much is the page visited compared to other pages.

• The percentage of their users that are visiting this page.

• Whether their users are getting frustrated when visiting this page

The table also helps support the heatmap content: Users can narrow down the list by displaying only frustration signals or searching for an event name.

Action Side Panel (right)

The action side panel's goal is to clearly define the specificities of an event. The top level graph displays the occurrences of an action over the selected timeline, this should help our customers find out if a specific action is happening constantly or at specific intervals.

The "Action details"  tile is here to explain in natural language the importance of this action relative to the page, This should remove the mental load from users that are trying to understand the relevance/importance of the action on this page.

The "User details"  tile Explains in natural language the importance of this action relative to our customers user base, This Gives a good idea of the user impact of a frustration signal.

The links below these two tiles allows our customers to investigate the selected action further with RUM analytics or Funnels. finally the The replay list helps to gain qualitative understanding of user behaviour linked to this action.

Heatmap empty state

It is very important for us to gracefully handle Heatmaps empty states, this is a rather complex dataviz product so we want to make sure that we are always transparent with our customers, we don't want them to get the feeling that the product is not working as expected.

In the example below the user selected a timeframe that does not contain any heatmaps data, there are several reason this might happen:

• Customer is not sending RUM data data from the specified application

• Customer is sending RUM data but Session replay is not enabled

• Customer is sending RUM and Session replay data but the selected timeframe does not contain any, the data might have been sampled out or just not present in the first place due to low traffic.

We take into account each of these reasons and display an empty state that helps the user find a quick solution to their problem.

Usability tests

After showcasing the prototype to the entire team, we conducted both internal and external usability tests to validate its usefulness amongst our customers.

Objectives

Defining objectives for usability tests is essential was we need them to validate wether the test was successful or not, here are some of the key points we wanted to shed light on:

• We are looking to see if users understand what they are looking at when landing on the page.

• Vanity Metrics: are these useful ? Is there any other data that would be better suited for the main panel ? 

• Focus on Popovers, do these lead to where they expect ? Are there other information that that would see in these 

• Filters: do we like this new way of filtering ? Does it make sense for users that are already familiar with Datadog ?

• Action list: Does it make sense for our users to isolate frustration signals ? Do users understand how to filter for a specific frustration or for all frustrated actions?

• Action side panel, Do users see the need for the top graph, is it clear that it can be used for selecting a timeframe ? Vanity Sentences: do they find these useful ? Is there any other information that we could put there ?

Assumptions

Few key points that we used as a basis to come up with the design, we want to verify if these are correct:

1. Our target audience is less technical than usual for datadog, we expect to provide a lot of context to help make sense of the data.

2.  Integration with the rest of the platform as well as sharing are really important for such personas.

Tasks

These are a snippets of some of the script that we used to run the usability testing. this was ran on a group of internal and external users, we interviews 15 potentials users in total.

Main page tasks:

• Now from this page I'd like you to  show the heatmap from users that come from (France) that use the  (Chrome) browser ? and use an ultra wide display (wider than 1200px < *)

• Now Let's say that you also need a very specific facet for your use case, you have a "campaign name" attribute on your session that targets a specific Google ads campaign. Could you show me where you would find that attribute ? 

• Now regarding sharing the heatmap ? How would you go ahead and share it ? In what format would you want to share this and with whom ? 

Platform Integration

• What part of Platform do you think you would want to jump to from this page ?  Why would you want to go there ? What do you think you would do there? 

• Do you think you would want to jump to the RUM explorer from this page? What kind of visualization would you expect in the RUM explorer when jumping there ?

• Regarding these metrics blocks on the top panel: Can you describe to me what those mean - what would you do with the information that's shown on there ?  Would you say these are useful to you ? 
  Would you expect to see other information in this place ? Did you expect them to be clickable?

• Can you show me how you would filter out the list on the bottom of the panel to only display frustration signals. What else do you think would happen when filtering it ? 

• Now could you show me how you'd find the most common action on the page - not exclusively a frustrated one - and then dive into that specific action to get more details ? 

Action Panel

• Do you expect to be able to share this panel, how would you do so ? What do you think of the copy button, what do you expect this to copy ? 

• If you were to select a different time frame to focus on that spike we see in that graph, how would you do so ? Please describe what you’re seeing in the action details and the user details boxes - element by element.

• Do you find the information in the action details and user details boxes to be useful ? Would you expect something else there ? Do you expect these to be clickable ? 

• Could you show me how you would jump to a replay that contains the action we are looking at from this screen ?

• Could you show me how you would see all replays that happened on firefox from here. 

Key findings for our research

[Coming soon]

Technical Challenges

A brief overview of the issues we encountered during this project and how we worked around them to still provide a good user experience [Coming soon]

Future updates

What comes next after the first version of Heatmaps is GA

Better site navigation

[coming soon]

Better integration with the platform

[coming soon]

Conclusion

[coming soon]