Stop Measuring Everything
We're addicted to data, but rarely derive insights or activate what we measure. Here are five recommendations based on my work with both early-stage startups and mature products.
Hi! It’s been a while. I got a little wrapped up in the momentum of starting Velocity Ave, a research and product strategy agency, these past 5-6 months. I’m getting back to writing and look forward to continuing to share and discuss with you all.
Don’t be a stranger!
Maya
In our data-driven age, it might seem paradoxical to hear “stop measuring everything.” A cursory search on product measurement will give you dozens of results with lists that tout 10-20 metrics. Yet, the genuine power of data comes not from its abundance but its relevance and applicability.
Pick Wisely: Tailored Instrumentation over Exhaustiveness
Recently, my agency worked with a client who had their eyes on the instrumentation prize. They wanted to measure the whole funnel. Idealistic or perhaps unconsidered. Measurement, especially in-product, is not easy to engineer. It’s costly. You need to build the scaffolding, prioritize, and create a culture around measurement before you try to adopt the gamut.
Make it Meaningful: Context over Volume
Yes you could measure NPS. Yes you could measure CSAT. However, a closer look reveals that these “snapshot” metrics often miss the holistic portrait of the user experience and product value. Personally if I’m going to solicit consumer or customer feedback at scale, I prefer more contextual measurement to assess product or feature integrity. Metrics are a compass, but they won’t give you the path to your destination.
Peel Back the Layers: Inquiry over Vanity
Metrics have many purposes: vanity, sales pitches, competitive positioning, storytelling, momentum, advocacy etc. Often, in my experience, it’s rare that a metric informed decision-making. Typically, I’ve seen them used optically. Totally ok! But, on the very best teams, a metric is a signal that can point to an opportunity to dive more deeply into the why. Enter: UXR. In these rare cases, there’s an opportunity to create a flywheel between product experience, research, product decisions and strategy.
Do it Well: Integrity over Hastiness
I’ve sat through hundreds of reviews, sometimes CAC, retention, and active users were literally FUDGED. If you are going to bother measuring something, talk to an expert, read a book, consider measurement strategies and craft a plan with the xFN team to ensure there’s commitment and accountability.
Tell a Metrics Story: Connection over Isolation
Behind and in front of every metric is a story. In an ideal situation it connects to a market problem, the human need, and how your product creates value. Metrics can be an open window or a closed door depending on your approach.
In Conclusion: Why over What
In our pursuit of data-richness, let’s not fall prey to the illusion that more metrics equate to more insights. It’s not about measuring coverage, but measuring thoughtfully. Why is your organization committed to or considering a specific metric? If it doesn’t make sense or feel connected to the product experience, you may want to lead the team through an assumption mapping exercise.
Metrics can be a lens through which you discern the impact and potential of your product. Metrics can guide you towards intelligent, insight-driven action, but it requires intention, alignment, and accountability.
Please reach out if you have a success story about metrics or if you are working on a product that needs a research partner in 2024 to support the measurement journey.