How To Measure the Value of Internal Tools

What Are Internal Tools?

At Square, we use all kinds of internal tools to help improve our productivity and efficiency. In our org (Seller Experience), we use tools to effectively manage our data and provide engineering and analytical support to a variety of internal users. Here are some examples of our internal tools:

• Communications Platform is a centralized messaging platform, mediating messages between senders, publishers, and recipients across multiple channels. It helps our marketing teams create message templates and send out messages to customers.
• The Customer Data Platform (CDP) combines data from various different sources into a single centralized data hub, making it easy for our internal users to access data.
• The Amplitude data platform that helps data scientists create dashboards for data analysis.

All these tools are used across product, engineering and analytical teams within Square. Even though these tools are internal, they are an important part of our work process and help us achieve the goals of providing the best experience for our external customers.

Importance of Tracking Internal Tool Metrics

Our internal tools can be developed by engineers within Square or by a third party. Nonetheless, it takes us time and effort to build, implement, maintain, and use these tools. Due to the time, cost and effort spent to build and maintain the tools, it is important to understand how effective the internal tools are and how much value they are creating. By using the right data and metrics we can

• Measure the internal product’s impact
• Improve internal product features through data
• Identify anomalies and improve the experience for our internal stakeholders and users

How to Track Internal Metrics

Unlike external facing tools that generally can be linked directly to sales and revenue, internal tools may not have those direct external impacts and may have challenges in measuring their return on investment. I’ll introduce some basic methodologies for tracking internal metrics, and how we should design our data collection processes to ensure we collect the right data.

Key Metrics

In our org we categorize our products and tools based on the similarity of the metrics that will be tracked. Since different types of tools will need different metrics and tracking mechanisms, it’s important to have standardized metrics design logic while providing flexibility in measuring individual products based on their features/characteristics. In our case we have two types of metrics:

Product metrics are metrics relating to usage/impact of the product and how users are using the product. The product could be a communication platform, a machine learning algorithm, a service/data infrastructure tool like CDP, an analytical dashboard, etc. Here are some examples:

• User adoption - number of new users/teams using the product
• User engagement - number of daily/weekly/monthly active users
• User satisfaction - qualitative and quantitative data from CSAT survey
• Business impact - this may or may not be available based on the nature of the internal product. But we could assess the internal impact based on above metrics

Operational metrics are metrics to evaluate the performance and reliability of the product/service provider. Rather than looking at it from the users’ perspective, we track the metrics from the provider perspective. Some examples include:

• Internal impact - number of features delivered per quarter
• Service level - percentage of tickets/requests resolved within SLA
• Reliability - number of open vulnerabilities detected per week

Data Structure

Not all metrics will apply to every team and we may need different ways to measure for different teams. We often don’t have the right data logged because we didn't consider measuring internal product impact when we designed the product. It’s helpful having the metrics in mind when drafting the product PRD. Here are some tips for designing the data structure:

• Identify the key functions and steps that represent the user activity, and log the data. For example, the number of user clicks for the “create” button could indicate the adoption level of users.
• Standardize column names, table structure and naming conventions. A standardized naming convention ensures that all data elements have a consistent and uniform format. This makes it easier to identify and locate specific data elements within a database, reducing the risk of errors and confusion.
• Have a data table that maps the user name with team name. Users could switch teams within an organization, it would be helpful to keep the data table with the latest information.

It’s All About Collaboration

To effectively build internal product metrics and implement the measurement effort, we need to collaborate with product teams to understand the product and engineering teams that provide the data infrastructure. Our Go-to-Market analytics (GTM analytics) team aims to bring a centralized effort to measure the effectiveness of our internal products. We create the metrics in a way that is automated, scalable and requires the minimum amount of effort for maintenance. We provide a standard guideline for all products, yet are able to create customized metrics based on the nature of each product and identify the key metrics. We constantly seek feedback from our internal customers in order to help them gather meaningful insights from the data.

About the Author

Chang is a Business Intelligence Analyst at Square. She has a passion for discovering insights hidden within data and believes data can be transformed into actionable insights that can drive business decisions. Through this blog, she hopes to share insights and experiences as a BI analyst at Square, as well as thoughts on various data-related topics. Hope you enjoy the read!