Data driven product management is a fallacy

Why “Data-Driven” Product Management Is a Fallacy

"Data-driven product management" has become a widely accepted mantra. Many product leaders use it as shorthand for rigor, objectivity, and efficiency.

But I think it's a fallacy—or at least a dangerously incomplete idea.

Relying solely on data—those cold, quantitative metrics—can trap teams in a loop of chasing what, while completely missing why. It risks optimising the wrong things, in the wrong direction, for the wrong reasons.

Let me be clear: data is essential. It's one of the most powerful tools in a product manager’s toolkit. But being led by data alone is not the same as making good decisions.

In fact, that kind of thinking can push you toward short-term wins that compromise long-term outcomes—whether that's customer trust, brand perception, or team morale.

When Data Misleads: A Practical Example

Say you're working on an online car marketplace. You run an A/B test on a redesigned purchasing journey. The new variant increases conversion by 5% and customer satisfaction by 10%. Great, right?

In a purely data-driven culture, you'd ship it immediately.

But pause. Why did those numbers go up?

Did the design simply add three “Buy Now” buttons on the page? Did it introduce artificial urgency with messages like “7 people bought this in the past hour”? These are manipulative UX patterns. And while they may spike short-term conversions, they often erode long-term trust.

There’s also the opposite scenario. What if your experiment results in lower conversion—but introduces new branding, UX architecture, or navigation that's foundational for future improvements? What if it sets the stage for a more scalable platform or more inclusive design?

The team at Airbnb has written about this—the "winner’s curse," where the variant that statistically "wins" in a test might actually degrade the overall experience over time. Data doesn’t always tell the full story.

The Real Risk: Proxy Metrics and Ethical Blind Spots

This isn’t just a tactical concern—it’s a strategic and ethical one.

In AI-driven products especially, optimisation can easily be abstracted away from intent. Algorithms chase proxy metrics like engagement or time-on-site, without any understanding of why those things matter. And when the product team blindly follows those outputs, we risk designing for addiction, manipulation, or exclusion—often without realising it.

Being data-driven can create a false sense of certainty. It can absolve teams from asking hard questions about what’s right for users, what aligns with the brand, or what unintended consequences might emerge later.

That’s why the goal isn’t to be data-driven. It’s to be data-informed—balancing quantitative evidence with qualitative understanding, critical thinking, and real-world context.

So What Should We Do Instead?

Talk to your users. Regularly. With purpose.

This sounds obvious, but applying discipline to your user research will supercharge its value. A few questions I always ask before launching research:

• What’s the point of this research?

• Why are we doing it now?

• What format or method is most suitable?

• What does success look like?

• What assumptions or hypotheses are we testing?

• Where could bias (ours or the users') creep in—and how can we account for it?
  
  

And when you talk to users, go beyond what they say—observe how they say it. Pay attention to their tone, hesitation, frustration. And if possible, see them using your product in context, not just over a Zoom call. You’ll uncover friction and unmet needs that never show up in your dashboards.

The Bottom Line

Data helps you see. Research helps you understand. Judgment helps you decide.

The best product teams don’t lead with data. They lead with empathy, guided by data, framed by purpose, and grounded in ethics.

So no, I don’t believe in data-driven product management. I believe in human-centered, data-informed decision making.

That’s how you build products that truly matter.