The folks at Pragmatic Marketing have a popular saying in their training and materials — “Your opinion, while interesting, is irrelevant.” For the most part, they’re absolutely right — the point that they’re making is that, while you as a Product Manager may have knowledge of the market and of the users, you yourself are not the market, nor are you your users. Rather, you have an opinion that’s just as valid as any other opinion in your company — no better and no worse. The assertion is that your decisions need to be based on objective data, nor subjective feelings — on research and testing and reports that you can glean from the actual usage of your product or your actual users.
The problem is, while that’s a good rule of thumb, and when such data is available, it’s absolutely necessary when making decisions. But we also know, as Product Managers, that such data isn’t always available, or it’s very costly to obtain, or you have to make a decision on the fly.
Objective Data is for Disruptive Change
While we should always err on the side of obtaining and using objective data in our decision-making efforts as Product Managers, there are times when it’s essential to obtain such data — when you are making a disruptive change. This could be strategically disruptive, it could be tactically disruptive, or it could simply be disruptive in your daily tasks. But, when something comes up that poses a big enough change to the status quo that its effects will have ripple effects throughout the product, the company, or the market, you want to be as sure as possible that the decision is the right one. You want to make sure that when you shift direction into a new market, that market is ready, willing, and able to buy your product. You want to be sure that when you move something to the highest priority for you development team, that it’s objectively more important than whatever it is that you’re displacing in order to fit it in. When you’re changing the user interface for a core component of your product, you want to make sure that you have as much data as you can that demonstrates that it’s a better user experience than what existed before.
And we should not only expect this kind of research and justification from ourselves, but also from others whom we work with. When Sales comes to you with a hot-off-the-press deal, ready to close but for that one feature that the customer so desperately wants, you can ask them to commit to a number in order to change the priority. When your CEO comes to you with the next big thing, you can ask them what validation they have done on the idea in the marketplace. When your Dev manager tells you that there’s some pending technical apocalypse, you can ask him to provide you with details about just what’s going to tip over, when it’s going to happen, and what the mitigation plan is in the meantime. All of this is part of our job as a Product Manager, to not only do our own due diligence and perform our own research, but to hold others’ feet to the fire when they come to use with the “next big thing” that they just thought of over lunch. Sometimes, they’ll be right and you’ll work together to validate the idea — more often than not, it’s just the irrelevant opinion of one single person (or one single customer), and nobody wants to turn the boat around just for that.
Subjective Data is for Evolutionary Change
On the other side of the spectrum, there are certainly day-to-day decisions that we all must make as Product Managers that might not always be based on hard, objective data — but rather on our knowledge, expertise, and (gasp!) our instincts. These decisions should always be made with a mindfulness that we’re not using objective data, but that shouldn’t stop us from moving forward and validating the decision at a later time — preferably as a “package” of such decisions. What color to make an icon, where to position a button in a prototype, placeholder text that will be altered by the time the product ships — all of these things are small, evolutionary changes that move the project or product forward, but which allow for easy correction if your instincts or knowledge turn out to be incorrect.
Certainly, it’s always better to test and validate and make these decisions based on objective data — but it’s not always available, sometimes you have to make a decision and stick with it. And that’s okay. In fact, it’s often necessary — when a developer is blocked from forward progress on something that they have a question about, you can’t afford the delay needed to obtain a representative, statistically-significant sample upon which to make your decision. Rather, you make the best decision you can with the information available to you — and if you need to change it later, you do so. If one of our goals as a Product Manager is to help the development team to delivery working software, then sometimes we have to rely on what we know, rather than what others can tell us. Assuming that you’re validating your solutions with customers or customer proxies, any mistakes will be corrected before the product is generally available anyway.
Err on the Side of Data, But Avoid Analysis Paralysis
The bottom line is that, as Product Managers, we should always strive to make our decisions on as much information as we can quickly, accurately, and competently obtain. Whenever possible, we should ensure that our customers, our stakeholders, and other market representatives have as much say in what we decide to do as possible. But at the same time, we can’t validate every single choice we make with our users. We can’t ask the development team to hold on while we create a SurveyMonkey email and wait for responses that might not come for days. We can’t expect our companies to invest in structured UX testing for placeholder text that’s going to be reviewed multiple times before shipping the product. But we can make sure that we’re mindful of the decisions that we’re making them, how we’re making them, and why we’re making them based on the information available. Knowing how to balance data and instinct is one of the hallmarks of a great Product Manager.
