Metrics for Internal Product teams

September 2021

A few thoughts on explaining the benefits of internal product investment.

++

Most companies are full of people - the one I work for has over 2,000 employees. These people together build and manage the company, so it follows that to ensure the organisation is in good health the people that work for them need the right tools and capabilities at their disposal to be effective and impactful.

I work as a Product Director for a team called Internal Products at a company called Financial Times. Internal Products is a group that partners with other departments to work out what tools are needed, what they need to do, help build them, and introduce them to peoples working lives.

We take the principles from outwards-facing product teams, mix them with some unique aspects of development that we’ve found internal-facing products require, and apply them to opportunities across the company.

As with our colleagues in customer facing product roles, internal products need metrics to describe the outcomes, justify their existence, guide their development, and ascertain their success. Metrics also make it clear on why we are doing something, and why we are not, or why we are pursing one opportunity over another.

While the projects we work have different objectives we do maintain an informal shared vocabulary for describing project benefits based on three common themes - impact, efficiency, and satisfaction.

Impact (qualitative)

Some people who work at your company will have roles that directly impact the organization's bottom line ("profit centres" in cold, finance-speak) so we can fairly easily track their activities back to some sort of financial benefit. So ironically for an inwards facing team, this first metric, impact, is about attribution of tools we provide to a outwards-facing commercial goal.

More often than not we won’t express this in terms of £ or $ but in one of the many proxy metrics for money or revenue - engagement, visit frequency, conversion rate, and so on. A sales person will have a quarterly target to track, a marketeer will have some funnel numbers to hit, and improvements to the tooling, data, or organisation processes can help (or even hinder) these objectives.

For example, a typical key result might be as follows, ”Increase reader engagement by 3% (via improved Editorial curation).”

Expressing internal product metrics in these terms is not always possible, nor is it always wanted. Often people don’t think of their jobs purely as a return on investment, which is fair. We can provide great customer care or write brilliant news headlines without attempting to jump through hoops to tie it directly back to a revenue line. It’s important to understand the motivations of the teams you are working with when designing the metrics.

Efficiency (qualitative)

The most frequent metric we encounter is efficiency. In the world of Enterprise™ software we don’t have to look very far for inefficient software or complex processes. Often people will offer their opinions on the state of things without much prompting, which is helpful. It’s safe to say demand to fix things outstrips our supply of resources to help with solutions.

In white collar organisations efficiency is often just a proxy for time. The time it takes to pay a bill, time it takes to glean an insight, time it takes to work through some process you have to undertake. In other industries it might be some net reduction of resources used (“we will use 1% less fuel for every 500kg the aircraft weighs”) but that’s not so much my area.

Time is simple to measure and something that is as interesting to organisations today as it was to 19th Century factory owners, and overhauling processes or automation are the routine solutions as much today as they were in 1830.

I like to quantify the time saved across a year by multiplying the number of hours saved by an approximation of the cost of an individual employee. For example, eliminating a daily 30 minute process for 200 people equates to £1m of employee time a year. Conversion to a monetary figure this way means we can easily compare different opportunities across the department and, for example, write key results that look like this, “Save 180 sales staff 45 mins a day doing data-entry (~£1.3m p/annum @ £70k cost p/employee).”

One temptation is to go a step further and marry time saved into other benefits. For example, if we could save Team X 50 hours a week they could spend their time on other higher-value activities. Productivity, in other words. While this is certainly a goal I’ve found it somewhat tenuous as a product metric. There’s things we can control, such as automating a process, and there’s things slightly beyond our reach or responsibility, such as what happens with the time now freed up from the automated process. We might well have an opinion, or make a recommendation, but we don’t run the Sales or Finance team. Efficiency as an end goal is a clearer, controllable measure.

Satisfaction (qualitative, quantitative)

Can you run a company with error prone, old, inefficient tools? Of course you can, plenty do, though not something to aspire to. Bad tools will dent the progress of your company. They will make your colleagues frustrated and stressed.

We want folk to have a high opinion of the technology they use to do their jobs, or at least not see them as a hinderance. Tools should remove friction and enable people to do their jobs more effectively and therefore engender a positive sentiment.

So as we do with customer-facing product development we can measure the satisfaction levels of people using the different tools. Typically this is done as an Net Promoter style survey, one for each department or group of users using a particular bit of software or service.

With such a benchmark taken we can then create a key result and measure at intervals (once every 6 months we find enough), attributing any changes to the efforts of the product team working with that department or software to improve things. Our satisfaction key results look like this, ”Improve Sales NPS sentiment by +25 points.”

The best thing about Internal Products as a Product Manager is the amount of time people are willing to donate to explaining their problems in detail so we tend to collect a lot of qualitative sentiment via these conversations, either via an survey or structured face-to-face interviews. This is a good source of information and can translate into key results such as, “fix the top 3 problems with Newsletters.”

One interesting aspect of satisfaction is that is can vary wildly depending on your perspective. For example, a Finance department will have finance professionals as users of, say, some software to process employee expenses and who may be extremely satisfied with it, but asking a busy, expense-filing employee might yield the opposite response. Likewise, a manager's view of a tool might be to standardise a process, which may be very different to the motivation of people using it day to day who want more creative freedom. Aggregating the responses without understanding these roles might mean you miss problem areas.

++

Individual projects will certainly have other goals - data quality, cost savings or consolidation, technical reliability or security - but these sorts of things tend to be local to individual areas. Impact, efficiency and satisfaction tend to be universal. The goal is to run a product team with credible, believable metrics and use them to focus their efforts on things that will create the most value for the people we work with.


Thanks to Laura Carvajal for the valuable feedback!