Это видео недоступно.
Сожалеем об этом.
The UX Research reckoning is here | Judd Antin (Airbnb, Meta)
HTML-код
- Опубликовано: 14 авг 2024
- Judd Antin has spent 15 years leading research and design teams at companies like Yahoo, Meta, and Airbnb. His direct reports have gone on to lead user research at Figma, Notion, Slack, Robinhood, Duolingo, AllTrails, and more. In our conversation, we unpack the transformation that the user-research field is experiencing. Specifically:
• Where user research went wrong over the past decade
• The three types of research-macro, middle-range, and micro-and the purpose of each
• How to effectively integrate researchers into the product development process
• The “user-centered performance” phenomenon and why it’s a waste of time
• Common tropes about PMs, from researchers
• The ideal ratio of researchers in a company
• Why Judd says NPS is useless, and what to use instead
-
Brought to you by Teal-Your personal career growth platform: tealhq.com/lenny | Vanta-Automate compliance. Simplify security: vanta.com/lenny | Ahrefs-Improve your website’s SEO for free: ahrefs.com/awt
Find the full transcript at: www.lennysnews...
Where to find Judd Antin:
• LinkedIn: / juddantin
• Website: juddantin.com/
• Blog: / onebigthought
Where to find Lenny:
• Newsletter: www.lennysnews...
• X: / lennysan
• LinkedIn: / lennyrachitsky
In this episode, we cover:
(00:00) Judd’s background
(04:16) Critiques and responses to Judd’s post “The UX Research Reckoning Is Here”
(07:33) The state of user research
(08:53) Macro, middle-range, and micro research
(14:05) What teams get wrong when it comes to research
(15:46) The importance of integrating research from the beginning
(17:30) Traits of great researchers
(19:53) Advice for evaluating user researchers
(21:10) Balancing business and product focus
(23:55) User-centered performance
(26:42) The role of intuition in product development
(30:15) Checking your gut instincts
(32:54) Common tropes about PMs, from researchers
(41:02) A/B testing vs. user research
(43:15) Hindsight bias and narrative fallacy
(44:55) Making recommendations based on research
(47:26) Advice for teams on how to leverage researchers
(51:18) How product managers can be better partners to user researchers
(56:53) The ideal ratio of researchers in a company
(59:43) Empowering user researchers to drive impact
(01:03:39) The limitations of NPS as a metric
(01:06:48) The risks of dogfooding
(01:08:51) Lightning round
Referenced:
• Matt Gallivan on LinkedIn: / mattgallivan
• Janna Bray on LinkedIn: / janna-bray-a4046a25
• Celeste Ridlen on LinkedIn: / celesteridlen
• Rebecca Gray on LinkedIn: / rebeccagray2
• Hannah Pileggi on LinkedIn: / hannah-pileggi-43169314
• Louise Beryl on LinkedIn: / louise-beryl-13225833
• The UX Research Reckoning Is Here: / the-ux-research-reckon...
• The end of the “free money” era: www.theguardia...
• Cognitive biases: en.wikipedia.o...
• IDEO design thinking: designthinking...
• Everything Is Obvious: How Common Sense Fails Us: www.amazon.com...
• Patrick Collison’s tweet: pa...
• Brian Chesky on LinkedIn: / brianchesky
• Brian Chesky on Lenny’s Podcast: www.lennyspodc...
• What is CSAT and how do you measure it?: www.qualtrics....
• Michael Murakami on LinkedIn: / michaelhmurakami
• Bad Leadership: What It Is, How It Happens, Why It Matters: www.amazon.com...
• Demon Copperhead: www.amazon.com...
• All Systems Red: The Murderbot Diaries: www.amazon.com...
• The Last of Us on HBO: www.hbo.com/th...
• Belay glasses: www.amazon.com...
• Epictetus: Control What You Can-Especially Yourself: www.shortform....
• The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change: www.amazon.com...
Production and marketing by penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.
Lenny may be an investor in the companies discussed.
Most companies simply don't know how to use researchers well. Also, I've found the best UXRs who can uncover revenue generating or cost savings insights, have work experience in another job function before switching to researchers. Finally, the best research teams understand how to triangulate across marketing, UX and analytics using multiple methods like Judd says to give direction or precision in a given set of time, headcount or operational cost constraints.
Amazing content! As a UXR it truly resonates with me. Food for thought: as a next episode, would love to see what are the different ramifications / implications of AI in the UXR/UXD field.
As a young entrepreneur with little experience outside of sales or software, I absolutely love the breadth and depth of the guests you get on your podcast .
The amount of knowledge I've learned and implemented from your show is ridiculous.
Keep it up Lenny ❤
Nothing makes me happier to hear. Thank you for the kind note Stephen.
Thank you so much for this insightful talk. After listening to this podcast, I have to admit that I did user research wrongly during my 2-yr-exp as a Product Owner at a startup co.
I did conduct usability testing, user research or AB Testing just for user-centric performance. To have evidence for my ideas of new improvements or features (to convince myself and report to stakeholders), I conducted interviews just to know if users get value from which I assumed or not.
I just only brainstormed for the improvements by myself, not with other coworkers or hearing from users.
I wish I could hear these things sooner. Thank you so much, you guys changed my mind.
Most complete channel to learn product Management from top industry leaders, Loved it
I am certainly not as experienced the both of you but the idea that PMs or Researchers who don't focus on growth or profits are in the wrong industry is something I, personally don't agree. It's like saying Doctors should focus on profits , instead of making patients better the best way possible. It leads to shortcuts and risks to patients. I believe there should be a balance. Growth and Profits as the good and should be considered but it shouldn't be their singular focus. When you focus on delivering useful and effective user solutions, the growth and profits become easier to get.
I completely agree there should be a balance! And I'd say your medical analogy works pretty well. A doctor who worries only about patient care but can't keep their practice afloat isn't going to be in a position to help people for long...
In my experience it's a little bit of magical thinking - if we just take care of users the profits will take care of themselves. In fact researchers and designers need to more explicitly balance the two, carefully finding problems and solutions at the intersection of what users need and what the business needs.
@@juddantin What happens when what users need are opposed to what the business needs?
@@froglegsfried1303 That sounds like (1) a great opportunity for insights to help align the two, (2) a company that won't be around long, or (3) a company you don't want to work for anyway.
There are definitely some examples of evil companies out there, particularly extractive forms of capitalism. But in my experience it's pretty much always possible to align business and user needs, and usually relatively easy to demonstrate how that's better for companies in the long-term.
@@froglegsfried1303That is when you revisit the problem. Did you miss something? Where is the misalignment? Are they the actual customers? Are you solving a problem users don't have or are the users unable to see their problems and how do you open the eyes of the users? Examples: Who knew we needed touch screen phones or living in someone's house or sitting in someone's car? On the opposite, who thought we wouldn't still be using Segways? It is really interesting when solving these issues. You pivot, or you kot that nail on the head or your are left with no customers.
@@froglegsfried1303If business needs and user needs diverge you probably need to revisit your business model. You can resist giving the users what they want/need for only so long, because eventually a competitor with a better business model will swoop in.
Feels like Alexander Osterwalder's business model canvas should be obligatory for onboarding so everyone can speak the same business language. You wouldn't have the problem of researchers who don't understand the investors' deck. Meaning a company should prepare the business model canvas adapted to its own specificity and people in the company should know it by heart. Doesn't matter if one is from product, design, or engineering so everyone is on the same page.
Lenny, you should definitely invite Alexander Osterwalder to the Podcast.
Eric Ries knows him and can get you the intro (all three of them, Eric Ries, Steve Blank, and Alexander Osterwalder are in the lean movement).
Loved this interview.
User centred performance bit was hard-hitting. Unfortunately, guilty as charged.
On a side note: how does one product a software like this? How's the audio so good despite them being on a video call ?
This is a great talk. Although we may all be working in different fields, different products, and different steps of product development cycle; we need to perform research to assure the product is actually usable. The key is to find actionable insights instead of just collecting data because it's your job (to Mr. Antin's point make sure you can use multiple methods and know how to communicate them to the development team).
As I've heard many human factor engineers and user experience researchers say, "engineers make products work, researchers make products usable".
As a newish PM, I learned so much from this episode. Thank you for this fresh perspective.
"validate vs falsify" that was heavy
This discussion covered phenomena ranging from reactance to resignation. Reactance was described early in the discussion and made manifest in the backlash Judd Antin experienced from employing the term "reckoning" which he used to describe the change he believes is needed in the UX research profession. He deduced "reckoning" was overly strong, eliciting a powerful, although unintentionally negative reaction from his audience. Toward the end, Antin discussed the feeling of resignation shared by many UX Researchers at some point in their career. With large enterprise companies that have engaged with research performance for too long, it can be exceedingly difficult to build the relationships required to grow the insights team the company needs to both contribute meaningfully to growth and to do so in a rewarding and sustainable manner.
Thank you for this Chris. I have this video bookmarked to view again, because there is a lot being said in here. But I was hoping for more discussion about the reckoning itself. Maybe you were too.
I am only one member of the audience, but for my money, I want our community of practice move past the notion of that a reckoning might feel embarrassing to talk about, when so many (researchers, designers, and PMs) are still living with the consequences of very real job losses, and losses in credibility as well.
I want to hear more from people who are speaking with executives - directly - to learn the why? And at what point did it start making more sense to fire the whole digital design establishment, rather than pretend that nothing was ever wrong. So I want more 'from the executive's mouth', verbatim, about how the reckoning happened so that we can define a different future from the one where people were faking it to make it.
the first minute is stil ringing in my ears and I watched this days ago
Best researcher have five tools: 1. formative; 2. evaluative; 3. surveys; 4. applied stats; 5. SQL
I agree 100% but research needs to evolve or nothing will change for the better.
Love him for the fiction recommendations! (and everything else ofc)
I'm only one minute in and I'm like oooohh this is going to be good!
Louder for the people in the back: (When PM's do research, it's) "garbage in garbage out"... "A researcher is there to help you turn garbage into something that's not garbage and avoid the bias that can come from you just reaching out to your cousin's family and then doing whatever they thought you should do to the product."
I hope one day PMs stop with their Dunning-Kruger syndrome thinking everybody should do user research.
"Strategic planning" is just planning. It's not strategy. Pass it on.
nice discussion, the designer also has to learn business 🎉
Designers are concerned with business, they aren't read in quite often. While my last PMs were dreaming up shit that was impossible hard to achieve or convoluted, I solved a customer problem that helped increase their revenue. The PM role is just a business analyst in disguise. It has no formal education that one gets at an undergrad level.
Unboxing Pandora's box with truth and only truth!
‘Good researchers don’t ask what users want’ Why’s that?
Because in most cases users are unwilling or unable to articulate the best solutions to their problems. It's not fair to expect. It's not that we don't care what they want, it's just that understanding their problems so that we can use them to craft a solution is almost always the better approach.
@@juddantin Got it. But by understanding how user imagines the solution - even if completely "wrongly" - can get deeper insights into the actual problem and the general expectations, does it not? Or does that in fact pose more risk of skewing towards suboptimal solution + setting users to expect something they won't likely get?
@@Puleczech I agree with you, it can be part of a broader strategy to understand. But the point is that good research and useful insight takes a lot more than just asking "What do you want?" or "Do you like it?" Those are wrong and damaging stereotypes of what research is that (unfortunately) many folks out there have.
@@juddantin Agreed. I'd be horrified if my company hired a researcher with that kind of mindset, or worse, if that was our expectation of what researcher should really do. Thanks for responding.
Framed another way, people, especially professionals (as opposed to unqualified users or the general public) often present themselves as problem solvers with can-do attitudes. They don't want to be viewed in a negative light as a complainer. Rather, they want to be seen as someone that rises to the occasion and gets things done. It's up to a skilled researcher to know how to establish rapport and the right context to observe user behavior (the gold standard) and deduce what the challenges, opportunities, or needs are, as opposed to simply asking their research participant what they want or need. They may feel everything is great, even if you've just watched them struggle to complete a task they've mentioned they do several times daily.
Systems broken you say!!??
Deming deming deming
like
The last of us video game was exponentially better than the show in terms of storytelling