🔥 Hot Take Alert #7: Motion vs. Progress
Thoughts on changing product culture from Hubspot, Shipt, Slack, Upwork, Chime and more!
Hi there, it’s Adam. 🤗 Welcome to my weekly newsletter. I started this newsletter to provide a no-bullshit, guided approach to solving some of the hardest problems for people and companies. That includes Growth, Product, company building and parenting while working. Subscribe and never miss an issue. I’ve got a podcast on fatherhood and startups - check out Startup Dad here. Questions? Ask them here.
Welcome to another 🔥 Hot Take Alert 🔥 where I opine on something that I feel very strongly about and try to make it a little bit better. I don’t expect you to agree with all of them but please keep an open mind. Or don’t. It’s your subscription.
Past 🔥 Hot Take Alerts 🔥 have included:
🔥 Hot Take Alert #1: PRDs are the worst way to drive product progress
🔥 Hot Take Alert #5: No you shouldn’t do a Spotify Wrapped campaign
Q: As a product manager how do I get my company to focus more on the outcomes and metrics from our work?
I thought you’d never ask!
If you’re a student of the “greats” of Product Management, you’ve read articles from Marty Cagan, purchased all the books, read Escaping the Build Trap, and probably subscribe to 150 substacks that preach about Outcomes over Outputs.
But you know what? That hasn’t fixed the problem that most companies still focus more on shipping things than on maximizing the outcome of that work. In fact, I’d say the vast majority of companies are still centered on the act of building and shipping products vs. the reason the products exist (value for the customer and the company). This is incredibly frustrating and warranted a 🔥 hot take.
One of my favorite quotes about this, attributed to Alfred Armand Montapert, author of The Supreme Philosophy of Man: The Laws of Life is:
“Don’t confuse motion and progress. A rocking horse keeps moving but doesn't make any progress."
You may have also heard this referred to as a focus on:
Outputs vs. Outcomes
Effort vs. Results
Products vs. Impacts
Deliverables vs. Effects
Creation vs. Impact
Immediate Results vs. Long-term Consequences
Short-term Gain vs. Long-term Benefits
I don’t care what you call it, but it’s incredibly pervasive in companies even though the drumbeat against it is deafening. All the product experts and thought leaders say so. Just focus on outcomes and everything will be right with the world.
This is, of course, much easier said than done. For starters, if it was easy to do then all of the experts would be out of a job and you probably wouldn’t subscribe to this newsletter. And second, we are fundamentally wired to celebrate the release of something new. Call it “shiny object syndrome” or whatever term you’d like to ascribe to this attraction to new products, features, etc. It feels good to release stuff. It seems like progress.
In some ways, this is what we’ve been taught for awhile as part of fostering a Growth Mindset. Focus on the effort not the results as a means of encouraging experimentation, learning and personal growth. I still believe that effort matters—especially when encouraging children (and adults) to try new things, build resilience to failure and step outside their comfort zone. But unfortunately not everyone gets a participation trophy in life and great companies aren’t built on effort alone.
Inside of a company, and specifically a lot of startups, the pervasive drive for speed and shipping can become really unproductive over time. Nowhere is this more apparent than with Twitter (dba X.com) right now. You have two, divided camps: those that laud Elon as a hero for just shipping constantly and those that vilify his approach as chaotic and not strategic.
Now, I’m not advocating for moving at a snail’s pace. There is an importance in maintaining a healthy velocity which sits at the intersection of speed and direction. I’ve advocated for establishing that direction in prior newsletters, like WTF is Strategy. I think clarifying where you’ll derive the biggest impact actually enables increased shipping speed and is able to give you both outputs AND outcomes.
But this isn’t about that. It’s about the majority of companies (and yes, it is a majority) that operate as if one more feature is going to be the thing that saves them. And it’s about how to shift the incentives that exist to reward motion and leave people scratching their heads when the company misses its targets.
Back to the original question that launched this take: what do you do if you find yourself (as most do) in a culture where you’re building endless features for a sales team, a CEO, or any other team where the stakeholder relationship is more of a directive than in input?
Here are several ways that you can try to effect change and shift your company to one that rewards progress over motion.
Get your data house in order
If you, as product manager or leader, are not effectively measuring the results of what you release then the first step is to get your act together on that.
And even before that I recommend aligning on a product team (or pod) North Star metric and measurable objectives. This doesn’t have to be complicated. For example, at ResortPass one of our core product teams is accountable for marketplace transactions. Their first gate when evaluating a customer problem is then a question:
Will solving this problem effectively increase marketplace transactions?
At Lyft and most other marketplaces this is a similar metric—successful transactions as measured by whatever the definition of a transaction is (ride, booking, purchase, etc.)
Alternatively, if you’re a B2B company you might have something like new activated teams.
After you’ve got a north star for a product team you need to be clear on what actually moves this metric. Again, keep it simple. If you can’t highlight <10 and ideally ~5 metrics that really matter towards your north star then in the words of the AI bros on LinkedIn: “you’re doing it wrong.”
For example, marketplace transactions at ResortPass are moved by:
Searches
Availability
Views of the Detailed Listing Page
Conversion from DLP -> Purchase
Repeat transactions
Once you’ve identified all this make sure it is instrumented, measured, and reported on. Then you can walk around shouting “outcomes” all day and pointing to them.
And lest you think that this is only important for communicating cross-functionally, it is also possible for this to be a systemic issue within the product org. That’s right, the calls are coming from INSIDE the house.
If you can’t get this right as a product org how can you expect other teams to understand it?
Here’s how Dan Wolchonok (Head of New Products at Reforge, former Head of Data) worked on solving this problem while he was at Hubspot:
I joined HubSpot in 2013 and the product team was pumping out features. We held a monthly "science fair" where people demo'd new features and launches. So much of it was spot on: you can only demo production software (no mockups, prototypes, etc), you have three minutes tops, and it felt like a rock concert. I judged the presentations by how long and loud the applause. The CEO was always present and the whole company could come.
One thing that was missing was a culture of "did the thing that we shipped work" or "has this had an impact". I had a rule where I never wanted to show off something unless I had some data to showcase an improvement (50% faster load times for 100k users...translates to X human years of time saved), qual feedback submissions, or how it affected an output metric.
It was a little controversial when I introduced a powerpoint slide showing graphs of usage / performance, but it became a staple of presentations and became part of the culture.
It took awhile to change the culture. I needed to do it consistently, I needed to receive public feedback that it was great, and leadership had to communicate to people to incorporate elements of that into their demos.
Disclaimer: not everything can be distilled into a chart, but it should be the exception not the rule.
I also talked to Irem Metin, former VP Product at Upwork, Shiftgig and TripAdvisor PM. She saw the team at TripAdvisor get into the trap of shipping fast but then prioritizing shipping the feature over the actual result:
“One of our strengths and a big part of company culture at TripAdvisor was our speed. We were really good at shipping new tests on time to the extent that it became our Achilles’ heel. We could set goals like ship x feature by y date. Over time, we started seeing diminishing business impact despite our success in shipping new tests on time.
To address this we focused our product reviews on two questions: what metric are you expecting to move and why does that metric matter for the customer and the business?”
If you’re clear on your data and the metrics that matter the next factor that may be holding you back is a lack of (or unclear) product strategy.
Get your product strategy in order
As I mention in WTF is Strategy—and Bradford Coffey (fmr Chief Strategy Officer at Hubspot) puts it—strategy is:
“How a company can – consistently, over a long period of time – (1) create value for its customers, and (2) monetize that value.”
Executing on that strategy and demonstrating impact involves the same—customer value and company value. If you’re working on the right stuff and not just “stuff” then when you solve a problem for a customer you should see that reflected in your data. But if you don’t have a defined product strategy then you probably won’t be building valuable products and you certainly won’t be creating company value.
But Adam! We’ve been solving customer problems and it leads to company outcomes and we don’t have a product strategy. My hat is off to you and as my grandfather used to say: “even a broken clock is right twice a day.”
Most likely you won’t be doing the above but it’ll still feel like you’re doing a lot because you’re building “stuff.” Of course, if you’ve gotten your data house in order you’ll soon be able to see that what you’re building isn’t delivering much and it’ll be time for a deep look in the mirror.
If you have your strategy right and you’re measuring effectively you might still be failing to deliver meaningful outcomes. There could still be one large product-org-driven cause of this: you think you’re solving an important customer problem, but you’re really not.
Building the Wrong Solutions
For product orgs that have their data and strategy organized this is the next horizon to get right (and a very hard one at that). It often stems from a misunderstanding of customer problems or poor research practices.
What do I mean?
Customer Problems
Customers have LOTS of problems and lots of problems with your product. But not all of those problems are meaningful and (unfortunately) not all of those customers are the ones you should be talking to.
Shreyas Doshi talks about the first part in this tweet (Xeet? post?) “Destined to Fail.” His solution is the CPSR or Customer Problems Stack Rank.
I like and have leveraged this strategy in the past via the “$100 exercise” in customer focus groups. Give your customers a hypothetical $100 and ask them to spend it on solving various problems with the product. They can’t spend it evenly on everything and they only get the $100. This exercise helps you avoid the bias inherent in the Focusing Illusion.
For the second part of the above—that not all customers are the ones you should be talking to—my pal Casey Winters has a great piece on this related to the trap of building for Power Users. In a startup especially the most vocal customers tend to be your earliest users. But there aren’t that many of them out there and so they can lead you astray if you overly fixate on solving their problems as you grow. This is similar to the concept of anti-personas that Behzod Sirjani (Reforge Partner and former EIR) writes about in Brian Balfour’s blog.
Research Practices
In addition to the Focusing Illusion, which you can easily surface during research, there are other flavors of this problem that we run into with research.
According to Behzod in his article “The Mistake Almost Everyone Makes When Doing User Research” when you focus research on learning vs. decision making you introduce three key problems:
Choosing the wrong research approach.
Gathering the wrong data.
Not involving the necessary stakeholders
If you’re “decision-last” as he calls it (rather than “decision-first”) it’s prioritizing motion over progress. You’re focusing on the thing that looks or feels valuable (“Hey, we talked to some customers! Hooray!”) rather than what you need to do to move the business forward (gathering evidence to help you make decisions). The point Behzod makes is that you should learn to help you make decisions vs. learning just because.
There is also a subset of this issue introduced by leading with the solution rather than the problem. A lot of product managers think that conducting research involves asking a customer if they like their proposed solution.
As Behzod says,
“The two most important words I teach product managers are ‘how’ and ‘why’ because they completely change the way they talk to customers.”
When you’re polishing a prototype or ironing out a flow it’s fine to probe on the solution. But if you’re early in the process it’s a terrible way to conduct research.
Behzod again,
“‘Would you use this?’ is a motion question, because even if you hear yes, there’s not much that you can abstract away from that to help you position the product or know how to make improvements, not to mention you haven’t learned much about the customer.”
Don’t do this.
Instead, ask “How would you use this?” Because, as Behzod says,
“‘How would you use this?’ is a progress question. You’re able to dig into the customer’s workflow, how something like what you’ve built helps or hinders their core activities, and where you might anticipate challenges or friction for that customer in the future.”
Getting your own house in order around data, product strategy, and research is just one part of the equation. If you’ve gotten that to a good place it’s time to work on your stakeholders.
Influencing Powerful Stakeholders
It’s common to feel a lot of pressure from stakeholders to build the things they “need” to be successful. We’ve all been there. You’ll find this with CEOs and other C-level executives, your sales team, customer support, marketing, etc. And they probably need something but they may not need the something they’re asking for.
Empathy
The first step on this journey should be a healthy dose of empathy. Your stakeholders have goals that they’re trying to hit and those goals are important to them. Maybe their bonus or performance review depends on it. If they’re asking you to build something it’s because they think their goals will be more easily achieved if your team builds “fill-in-the-blank-feature.”
The reality is that it very likely won’t, but it’s worth understanding the ask.
A few questions you can ask them:
Tell me more about the goal you’re trying to achieve?
What company objective does that goal ladder up to?
What is the problem that building X feature will solve?
Which customers will be impacted by this feature?
Can we talk to those customers?
Notice that you’re approaching this from a place of inquisitiveness and seeking to understand vs. defensiveness. You’re encouraging conversation rather than shutting it down. This is important because oftentimes a product team will get a bad rap for saying “no” to everything (which, let’s be honest, is probably the right thing most of the time). But it can lead to escalation and a sense that you’re not a good partner. Of course, this isn’t true, but it’s likely to happen with the “no” approach.
What you’re trying to uncover with the questions above is threefold:
Is there a pervasive problem that exists?
Does this problem impact a wide swath of valuable customers?
Are we missing out on meaningful, potential revenue because of it?
If the answer to those questions is “yes” then you likely have identified a high-impact area. That doesn’t mean it’s a current priority, but it’s worth considering to see if it intersects with your product strategy.
If the answer is no then you need to do some rewiring.
At ResortPass, one way that we manage, group, and organize requests from across the organization is with an Airtable database. We have a simple form that teams can fill out that looks like this:
When I joined the company I added the “OKR” section which forces a submitting team to think through why the feedback matters and brings them back to our company goals. It’s not perfect, but it provides a system for capturing and categorizing customer feedback.
Having a system for requests and feedback doesn’t solve the problem of motion vs. progress though. That’s where a regular and consistent roadshow of your product strategy comes into play.
Product Strategy Roadshow
If you’ve done the work above to get your product strategy in order then you’re on the right track. You’ve probably also spent some time talking it through with stakeholders and getting their feedback and input. If so, excellent job! If not, then you’re not done with the “getting it in order” part so hop to it.
The problem you’ll encounter is that stakeholders have the memory of a goldfish. Your CEO, for example, is juggling a million things in their head. If you think you’re done after communicating your strategy once, think again.
Every interaction with a stakeholder is an opportunity to remind them of your product strategy and priorities. This can be in product reviews, weekly or bi-weekly stakeholder meetings, all hands presentations, and even your end of sprint share out.
Why does this matter?
It helps remind people what you’ve all agreed to in terms of important, needle-moving priorities for the product organization. So when an out-of-the-blue idea comes up you can run through the questions I outlined above and ask what part of the product strategy it should map to. If it doesn’t, it’s an easier conversation on punting or de-prioritizing that work in favor of the outcomes-oriented work you’ve outlined in your strategy.
Metrics Meetings
A third way to start the shift from motion to progress with stakeholders is the introduction of cross-functional metrics meetings – like a weekly business review (WBR). This concept, popularized by Amazon, brings a level of rigor and focus to the input metrics that matter for the business.
You may not be ready for a WBR at your company so one place you can start including your metrics review is in your end of sprint summaries. You can also start more slowly with a monthly metrics review. One of your earliest allies here should be your Finance leader. Why? Because, as I mentioned in my newsletter on evaluating product investments they care about the returns generated from the effort exerted.
Regardless of where you start it’s important to bring together a cross-functional group and start getting familiar with the levers that move your business forward. When you’re reviewing this on a regular cadence as a group it becomes much easier to have conversations pointing out where the output didn’t yield anything or the opposite—where output yielded quite a significant outcome.
But what do you do when the returns from some projects aren’t immediately apparent?
To dig into this I spoke with Vishal Kapoor, Senior Director of Product at Shipt. He talked to me about three different types of projects:
There are typically 3 types of projects: (1) those for which impact on either the end users or the business can be quickly measured, (2) those for which it cannot, and (3) the in-betweens.
Projects in the (1) category are usually considered outcome-oriented, and (2) and (3) typically get looked at as output-oriented. This is critical, because the clarity of outcome-oriented projects makes it easier for their owners to rally others around them, making them more evergreen. Alternatively, output-oriented projects constantly run the risk of being categorized as nice-to-have and sunsetted during business pivots.
Today, our process looks like this:
During quarterly planning, I mandate my teams to be outcome-oriented, and goal their own north star metrics against the business goals shared by the company’s leadership, instead of using vanity metrics to track their progress without any obvious connection to the topline business metrics.
For each idea that they pitch in the roadmap, they calculate baselines for their metric and estimate the expected impact of that milestone on that metric. This is similar to estimating development effort for the milestone, which they also do. Both of these help us all compare ROI and tradeoffs for different ideas side-by-side, and we provide metric impact estimates to any other teams (like Finance) as needed for their planning.
During execution, this is how they measure and report outcomes in their weekly, monthly, and quarterly business reviews. And we review both how much we impacted, the metrics as well as if we delivered the milestone on time as per the dev estimate.
And in case you think that this transformation was easy and immediate, Vishal again:
“It took us about 3 quarters to re-align the responsibilities of our Product and Operations Managers, mature our measurement systems, and hit our stride to a truly outcome-oriented company.”
My experience has been similar; it takes at least 3 quarters to get into this rhythm and often longer (18 months) to get “good” at OKRs.
Irem had this to add from her experience at Upwork:
“We were always customer and data driven, but we gradually improved which metrics we focused on. Our goal setting process used to be very bottoms up. Teams knew their customers and picked goals that mattered. We knew that we had the opportunity to improve alignment, so we changed our goal setting process.
First, we developed an even deeper understanding of which inputs drive business outcomes. For example, we analyzed which inputs such as speed, choice, quality have the highest impact on conversion. We made sure the most important ones were owned by a specific team.
Second, we changed how we reported on goals, always leaving room for new learning and discovery of new inputs.
Third, we focused the teams on metrics that are aligned towards the same outcome so we could make progress faster.”
Empathy for your stakeholders, a shared understanding of the product strategy, and cross-functional visibility into the metrics that matter are three ways that you can start influencing stakeholders towards a more progress-oriented culture. And if none of those work you can always just walk around shouting “outcomes!” all day 😀.
Parting thoughts
There are many company cultures that are entrenched in the belief that the product development organization should build the features that the rest of the company wants them to build. As much of a mistake as this is, I would argue that it is still the prevailing belief in most companies.
Some cultures will be so entrenched, and the product teams so meek, that they can’t overcome this inertia. But the playbook above—getting your own house in order across data, strategy, and research then working with stakeholders through empathizing, sharing your strategy and providing visibility into the metrics—should provide enough for you to get started in the right direction.
A few other notes in my conversations with folks around this topic. One conversation with Ely Lerner, former head of consumer product at Chime, was about affecting this change within your engineering teams:
The first thing that comes to mind, for me here, is actually engineering. Pretty much every time I’ve done this shift or been involved in it, the first step is getting eng bought in on the idea of moving from teams that own “features” and/or systems etc to cross-functional teams aligned to outcomes.
In the bucket of selling the wider eng team there’s a decent narrative to be had here that outcome focused teams actually help engineering folks ‘have a seat at the table,’ and also that this structure actually means a lot less thrash and interruption for many eng teams.
And finally, I’ll end with a conversation I had with Tom Willerer (Opendoor, Coursera, Netflix and who will also be making an appearance on an upcoming episode of Startup Dad). Tom doesn’t fixate on outputs or outcomes, rather he thinks the emphasis should be on the quality of the decision and the decision-making process:
“I actually have my own hot take on this which is that I think it's probably better to focus on the decisions and the decision making process than either the output or the outcome. The output is not valuable and the outcome can be out of one person's control. But the decision and the decision making process are entirely in someone's control and can be modified and improved upon.
As an example:
A VC can get lucky once but can they create a repeatable decision making process that allows them to have sustained success? That's what separates the best from the rest.”
His last point is a good one that can be translated to product teams—it’s important to focus on the decision-making process that leads to repeatable, successful outcomes. But I’ll leave that for another newsletter.
Special thanks to Dan Wolchonok, Behzod Sirjani, Irem Metin, Vishal Kapoor, Tom Willerer and Ely Lerner for their contributions to this newsletter.
Read all of my Hot Take Alerts here.