A few months back, delegates from Ayima attended the annual Digital Analytics Hub (DA Hub) 2018 in Austin, Texas, a conference that brings together top analytics professionals for in-depth exchanges of ideas (and in this case, to also eat some great BBQ!).
The conference is structured around small group discussions on a specific topic called “huddles” . The incredible quality of the attendees allowed us to freely share insights about the latest developments, challenges, and opportunities in the industry. It was one of the best conferences I’ve personally ever been to, mostly due to the high level of engagement and the extremely well-curated topics.
One of the most interesting huddles I attended was on the “North Star Metric”. It was led by Andrea Mestriner, Head of BI Tech at Just Eat, a food delivery tech company based in the UK.
What is a North Star Metric?
Before meeting, we were challenged to think about what metrics our companies track and report on at the various levels of the chain of command, why these metrics are used, and if we thought we could really influence them.
At the start of our discussion, we learned that the North Star Metric (NSM), sometimes called One Metric That Matters (OMTM), is a fairly new concept predominantly used by startups. Simply put, it is a single number designed to capture the core value that your product delivers to customers.
Optimizing your efforts to grow this metric is key to driving sustainable growth across your entire customer base. If used properly, a good NSM helps teams move from driving short-term growth to focusing on the generation of long-term retained customer growth. Coming from the world of complex, granular reporting, most of us were not familiar (or comfortable!) with the idea that one metric would be enough.
A Common (But Often Misguided!) North Star Metric: Net Promoter Score (NPS)
One existing metric that is often used as a North Star Metric by businesses is the Net Promoter Score (NPS). This controversial barometer aims to measure customer satisfaction and loyalty to a brand using a score between -100 and 100. Data is gathered from survey responses to a single question based on a scale from 0 to 10: “How likely is it that you would recommend our company/product/service to a friend or colleague?”. Rather than focusing on something more traditional for a single metric, like orders or revenue, NPS reflects a desire to move to a more experience-based metric as a company grows.
Our group concluded that NPS is misused most, if not all, of the time so marketers should be cautious when weighing it too heavily when making business decisions. Studies have shown that NPS is not necessarily a good predictor of future performance, and does no better (perhaps worse) than other surveys with a wider range of questions. On the flip side, even though NPS may not be a good indicator of how a website is doing, it should not be ignored entirely. We suggested using it not as a NSM but as a basic company health check. Being aware of what it can do and how it’s calculated is important in the modern marketing landscape.
How to Incorporate a North Star Metric into SEO Reporting
The discussion got me thinking about how we report on SEO and if there is room for a NSM in our industry.
As an SEO agency, clients often hold us responsible for many metrics at the site level. One of the most common is simply the overall SEO traffic to their site. If the client’s industry is seasonal, performance is measured Y/Y; otherwise, a M/M comparison is used. This number usually appears at the top of most SEO monthly reports.
However, because there are so many factors outside of our control as an agency given the unpredictable nature of Google’s algorithm (and often the client), perhaps this is not the best reflection of our performance.
Site-level metrics by channel that aren’t segmented down can be problematic for a few reasons. Firstly, overall SEO visits do not differentiate between brand and non-brand sources. As an example, traffic to the Ayima website driven by keywords containing “ayima” (like “ayima seo”) is considered branded, whereas traffic from generic keywords (like “how to improve seo”) are non-branded. Branded traffic is generally boosted by factors like advertising spend and marketing efforts to increase awareness and brand demand.
Companies should generally expect to rank very well (position 1) for brand queries. If that is the case, and they also make up a large percentage of total SEO traffic, it will be harder to see the impact of any optimizations we make with the goal of improving non-brand rankings. I think we can all agree that agencies should not be held to numbers they cannot fully influence.
So is there a single metric that is easy to understand and report, and that supports the results of all product, technology, marketing, experimentation and development effort? The solution we often suggest here is to report on non-brand traffic as the main KPI. Importantly, the metric passes all the criteria we kept returning to in the huddle that a NSM must meet at a basic level:
- Simple to understand
- Clear scale and direction
- Easy for all employees to relate to
- Creates accountability for management
Now that Google Search Console provides a full 16 months of data rather than the previous limit of 90 days, we can see long-term trends and compare monthly performance Y/Y. With a simple exclude filter (or a fancier regular expression formula) to remove branded terms, a truer measure of SEO performance can be measured. And by using grouping functions in Google Data Studio or pulling raw data from the Search Console API, we can aggregate it by week or month so trends are easier to see.
How to Deal with Revenue
If a client’s company culture is particularly geared toward the bottom line and justifying ROI, we may be commonly held to revenue that has been generated from the SEO channel as a NSM. The idea is that by following SEO best practices, creating quality content, and increasing links, better rankings lead to more money:
Tech changes / Content / Links > Rankings > Traffic > Conversions > Revenue
However, the dots are hard to connect all the way through because analytics platforms for a while now have not shown which keywords actually convert (the dreaded “keyword not provided” problem). Thus, we don’t see the actual monetary value of individual keywords in Google Analytics or Adobe Analytics.
It's also worth mentioning that SEO agencies should not necessarily be held responsible for actual orders since our main responsibility is driving traffic. Once a visitor gets to the site, the UX department is in charge of getting them to spend money through conversion rate optimization.
Compromises with the North Star Metric
One of our main conclusions is that a single metric is not enough to cover all the complexities of a mature company. Unless you are running a lean startup and growth is your only goal, it’s better to have one NSM for each part of your company. Or, better yet, you could have a short list of metrics (up to five) structured in a hierarchy. Each level needs to be checked and reported on at different times, in different spaces, and by different teams at various levels along the stage of command.
An iterative process could include simulation techniques to prioritize metrics and discover how they connect. By showing how movement impacts your initial selection of metrics, it becomes easy to spot those that don’t reflect true improvement.
We also proposed breaking metrics down in other ways (financial, strategic and tactical/diagnostic) to further improve the organizational hierarchy. Additional considerations should be made for leading and lagging indicators (metrics that respond immediately vs those that take time to respond) and how to weigh their importance against each other. In any case, these metrics are not set in stone and they need to be constantly checked and reevaluated.
Selection of these metrics is very important. It’s easy to fall into the trap of tracking and managing metrics that lead you in the wrong direction. Ultimately, you need to find metrics that predict your company’s success and help it grow and survive. Maybe this requires some reflection on the purpose of your company and why it exists. Every situation is different, but if you are in SEO, we would highly suggest including non-brand traffic in your reports.
Suggestion for further reading:
- Good to Great by Jim Collins
- Reforge: Don’t Let Your North Star Metric Deceive You https://www.reforge.com/blog/north-star-metric-growth