New feature: set high/low performers based off one North Star KPI such as CPP

‍TLDR: we released an important change in what classifies a high performer, making our data more actionable and proving that we're borderline addicted to perfecting how to compare data across companies

Previously, each metric was evaluated separately. But given the interconnectedness of metrics, a low CPM might not mean a high ROAS. 

In the new method, cohorts of companies are selected by how they rank in your chosen metric above. The median of that same cohort of companies are then used to calculate the rest of the metrics.

For example, if you chose CPP as your "Rank By" metric and companies A, B and C have the strongest CPP, they would be classified as High Performers. Then the CPM shown would be the median of those same companies (A, B and C)

Check out the updated ranking method on the top right of each of your Varos dashboards. Feel free to revert to the old method if desired. 

‍

White paper on the importance and data behind this change

Fixed Cohorts in Benchmarking

At Varos, we're borderline addicted to perfecting how to compare data across companies. One aspect of benchmarking that we have been obsessing about lately is making sure that our customers know what metrics they need to hit in order to outperform competitors (measured by ROAS or CPP).

Looking at the benchmarks on our platform, brands can easily identify weak spots on various KPIs compared to high performers. CPM, CPC, CTR, CVR, and so on all represent different parts of the buying journey, and while it's important to focus on the health of each one, they're highly interconnected. The reality is, you don't necessarily need to be a high performer on every single step of the funnel to be a high performer in what really matters: efficient and effective growth.

A simple example of this is looking at two interrelated metrics, CPM and CTR. As any marketer understands, a high CTR usually requires a high CPM. If we look at high performers for both metrics independently (as presented today in our platform), we can see that not a single company is a high performer in both. If you use high-performers from our platform as a target, you would be setting an unattainable goal. Worst case scenario, you would try to optimize both without knowing which one is more relevant. Because the truth is that some of these companies are doing better than on more important metrics, and we are not answering key questions like: is this driven by high CTR or low CPM?

After extensive discussions, we reached the conclusion that the metrics our customers look at should all be benchmarked against a key North Star metric such as ROAS or CPP. This way, looking at the numbers of high performers will tell you what really matters: what CTR and other funnel metrics you need to get to achieve overall high performance. And if the CPM or CTR of high performers is not that high, you will know if high CPM or high CTR matter the most.

We arrived at a clear conviction on how things should be done, but this implied a significant change to our platform, which was based on per-metric benchmarking. Because our mission is to become the best benchmarking platform in the world, we had no choice but to go forward with this change no matter the complexity. And today we are proud to announce this new way of benchmarking is generally available. 

Let’s look at how the new benchmark generation process works. 

We first rank companies with the common outcome metric. While ROAS was the obvious choice as the outcome metric, our research shows that it’s very sensitive to AOV variations and thus a better outcome metric for benchmarking would be CPP which we have set as the default ranking metric (you are still able to customize this by clicking on “Rank by”).

Once we have ranked the companies, we create the three performance segments based on this metric:

‍

Low performers: companies in the bottom 33th percentile of the ranking metric

Median: companies between the 33th and 66th percentile of the ranking metric

High performers: companies in the top 33th percentile of the ranking metric

‍

With the companies segmented based on the common ranking metric, we look at the metric values in each segment and take the median to give a good representation of the segment. In the example illustrated below, the x-axis represents the ranking metric and in the y-axis the metric for which we are calculating low, median and high performers. The horizontal lines are the values presented in our platform, e.g. the CPM for high performers is the median CPM companies with best CPP values. 

Note that this change also implies that a metric for high performers is worse than for median or low performers. While this can sound confusing at first, it reveals a very important insight: high performers are not optimizing this metric and the incremental value of optimizing it is likely low. 

This feature, despite being nuanced and somewhat complex, is the kind of features we love to build at Varos. We obsess over these so that you don’t have to, and you can focus your efforts on the highest leverage areas.

‍

About the Author

Yarden Shaked

Co-founder & CEO at Varos

LinkedIn