🤩Comment below: what tradeoff questions do you want to see me solve? If you need help for upcoming product interviews, book 1:1 coaching @ superpeer.com/dyau
you have the best product management videos. i have seen so many but none is as insightful as yours. your approach is systematic and practical...! thank you so much for sharing
Great video Diana , would be great to see other examples - perhaps the IG stories one you mentioned or trade offs between comments and reactions in FB - I think that would really help
I can’t thank you enough for all of your tips and for you taking the time to teach and prepare me for the FB interviews! I’m so excited to say that I got a FB offer and I certainly credit it to you. Thank you for your personal sessions with me and creating relevant and real content that can’t be copied. I love your authenticity and your passion to help others excel.
Thank you Dianna. I am preparing my interview for product sense and just found your channel. It's really helpful that you provide a useful template to structure different product sense questions!
I can’t thank you enough for all of your tips and for you taking the time to teach and prepare me for the FB interviews! I’m so excited to say that I got a FB offer and I certainly credit it to you. Thank you for your personal sessions with me and creating relevant and real content that can’t be copied. I love your authenticity and your passion to help others excel.
Dianna, THANK YOU so much for this video. I just finished my execution interview and am so glad I watched this prior to it! Your frameworks and tip are all so to the point, you deserve more subscribers and I’m sure they’ll come before you know it. Appreciate everything you do!
Hey Dianna, Great video - I really liked how you put forward a well defined structure for the Case. I do have a few questions though 1.Regarding Metrics - Can we ask the interviewer what stage the product is in (Growth vs Engagement vs Monetization) 2.Regarding A/B Testing :Should the control groups be segmented as per geography ? For example 3 groups per country 3. Regarding A/B Testing :Should we consider factors such as - User age / Gender / Time when the user had opened the video? Appreciate your content!
Hey Rahul! 1) you can ask that but best practice is to consider all 3 stages for metrics 2) you don’t need to specify geography for ab testing 3) no need for these specifics on ab testing, these are things your data scientist would figure out and usually shouldn’t specify things like age gender
Happy to help with the goal :). Hi Dianna, is there a particular reason why you didn't call out advertisers as one of the user types of this ecosystem?
Your videos and content are really helpful! Keep it coming! One piece of feedback, the soundtrack is mildly distracting as I try to listen to you and take notes.
Thank you Diana for explaining this tradeoff framework with a real example. I am wondering if you would make a video for the strategy questions in PM interview?
Hi Dianna!, thanks for another video, Is your framework suitable for this kind of tradeoff question, for example, discuss the tradeoff between "People you may know feature" and showing ads on the newsfeed?
Yess, this trade off framework works for those as well but take a look at the latest trade off questions in my channel bc they have a slightly updated framework
Question 1: When running the A/B test here, would the null hypothesis be that ad placement within the video has no impact on total ad clicks? Question 2: This video seems to be more about setting up an A/B test rather than purely tradeoff questions?
I miss clarifying questions at some point: how are those ads? We know that sometime youtube forces you to see the full ad and sometimes they allow the user to resume the video in a few seconds. Are those ads skipable or forced? Are the ads equal? Which is the average duration of the content?
Hi Diana, great videos. Thank you. I stopped watching other channels and videos. I am glad I found your channel. Is there anywhere I can find a comprehensive list of the metrics for FB per product and funnel stage(reach, activation, engagement, retention, revenue, and referral )? Many thanks.
Hi, can you please add subtitles? Also wanted to ask, why the number of videos watched is a metric you would look into after the A/B test? Which videos are you talking about here?
Great content from you Dianna. You add a lot of value to the users :) My only thought would be to do the metrics after the hypothesis section. Your thoughts?
Hi Dianna, thanks for such great video! I am wondering why do you select a set of videos instead of doing a/b testing on all the videos? Is there any criteria you used to select the testing video set?
How about introducing in new subs as one metrics , if ads are too intrusive can lead to increase in subs , and can increase ARPU , also what metrics to measure is there a rule in terms of how many metrics should i mention or a structure which i need to follow here
Hey Dianna, wouldn't a control group in this example be videos with no ads? That way we can see a group un-influenced by advertisements, and how those numbers would change with your three experiment groups.
You could add a holdout group like the one you suggested but that would be to understand the two variations from a no ads benchmark , but the question is asking about the incremental difference between the two variations so the no ads isn’t necessary
Hello Dianna, thanks for all your insightful videos and playlists. I see a conflicting point in 2 of your videos. In this video, you mention "identify metrics" as 2nd point whereas in "tradeoff FB reaction vs comment video", you mention metrics as 3rd point (metrics for those hypothesis). Which is the correct sequence? Thanks
As I figure out better ways to tackle these questions, I update videos to have the most updated best practices. So currently I’d recommend hypotheses first before metrics.
I am wondering why do you select a set of videos instead of testing on all the videos? Is there any criteria that you used to select those videos to test on?
my $0.02, you don't need the entire universe of videos, just enough of them to have enough power on your A/B Test. Also, not all videos are made equal (think length mostly, especially important if you want to test middle ads, since a quick 1min video for instance likely won't have enough length to merit a mid ad).
If it's the first time the product is launched (according to the question), acquisition, engagement, retention are always more important, IMO. Once in the growth or mature stage, revenue metics will start weighing more.
that's not something you'll need to answer in typical product mgmt interviews mostly because it'll really depend on what you're testing & require some analysis from a data scientist
Hi Diana, how do you approach a Tradeoff question that already has metrics provided? E.g. do you ship a feature that increased watch time but reduced no of comments and likes on Facebook watch
Hey Vidhi! in that case you'd skip the experiment part and evaluate which metrics are most important to the surface we're talking about. e.g. the surface is FB Watch so which metric is most important for FB watch, is it watch time or engagement? Or which of them is correlated with higher level metrics like Daily active usage on Watch, and FB
It's a great video and I appreciate you sharing this knowledge in such an understandable and easy-to-digest form. However while I understand that today's standards let a lot of video editing slide, in your videos I find it a tiny bit tiring after a while. Most extreme example: ruclips.net/video/5lqIdfBJqZw/видео.html The first syllable is doubled and for a while it resembled a deep fried meme video :D I'm not saying it's the end of the world but if you managed to reduce the amount of cuts, I bet the overall UX of your videos would soar!
🤩Comment below: what tradeoff questions do you want to see me solve? If you need help for upcoming product interviews, book 1:1 coaching @ superpeer.com/dyau
you have the best product management videos. i have seen so many but none is as insightful as yours. your approach is systematic and practical...! thank you so much for sharing
Great video Diana , would be great to see other examples - perhaps the IG stories one you mentioned or trade offs between comments and reactions in FB - I think that would really help
I can’t thank you enough for all of your tips and for you taking the time to teach and prepare me for the FB interviews! I’m so excited to say that I got a FB offer and I certainly credit it to you.
Thank you for your personal sessions with me and creating relevant and real content that can’t be copied. I love your authenticity and your passion to help others excel.
Thank you Dianna. I am preparing my interview for product sense and just found your channel. It's really helpful that you provide a useful template to structure different product sense questions!
So glad it’s helpful! Let me know what other content could be useful! There’s a few product sense videos coming up this week and next so stay tuned!
I can’t thank you enough for all of your tips and for you taking the time to teach and prepare me for the FB interviews! I’m so excited to say that I got a FB offer and I certainly credit it to you.
Thank you for your personal sessions with me and creating relevant and real content that can’t be copied. I love your authenticity and your passion to help others excel.
Wow great!!!
Dianna, THANK YOU so much for this video. I just finished my execution interview and am so glad I watched this prior to it! Your frameworks and tip are all so to the point, you deserve more subscribers and I’m sure they’ll come before you know it. Appreciate everything you do!
I’m so so happy to hear this was helpful for the interviews you just had! And crossing my fingers you make it and if you do, please let us know!
Hey Dianna, Great video - I really liked how you put forward a well defined structure for the Case. I do have a few questions though
1.Regarding Metrics - Can we ask the interviewer what stage the product is in (Growth vs Engagement vs Monetization)
2.Regarding A/B Testing :Should the control groups be segmented as per geography ? For example 3 groups per country
3. Regarding A/B Testing :Should we consider factors such as - User age / Gender / Time when the user had opened the video?
Appreciate your content!
Hey Rahul!
1) you can ask that but best practice is to consider all 3 stages for metrics
2) you don’t need to specify geography for ab testing
3) no need for these specifics on ab testing, these are things your data scientist would figure out and usually shouldn’t specify things like age gender
Happy to help with the goal :). Hi Dianna, is there a particular reason why you didn't call out advertisers as one of the user types of this ecosystem?
Your videos and content are really helpful! Keep it coming! One piece of feedback, the soundtrack is mildly distracting as I try to listen to you and take notes.
Thank you Diana for explaining this tradeoff framework with a real example. I am wondering if you would make a video for the strategy questions in PM interview?
What are some example strategy questions you’ve seen?
An example can be if Facebook should merge all its messaging apps
Thanks for sharing this framework on tradeoffs
Great video, actually
Hi Dianna!, thanks for another video, Is your framework suitable for this kind of tradeoff question, for example, discuss the tradeoff between "People you may know feature" and showing ads on the newsfeed?
Yess, this trade off framework works for those as well but take a look at the latest trade off questions in my channel bc they have a slightly updated framework
Question 1: When running the A/B test here, would the null hypothesis be that ad placement within the video has no impact on total ad clicks?
Question 2: This video seems to be more about setting up an A/B test rather than purely tradeoff questions?
I miss clarifying questions at some point: how are those ads? We know that sometime youtube forces you to see the full ad and sometimes they allow the user to resume the video in a few seconds. Are those ads skipable or forced? Are the ads equal? Which is the average duration of the content?
Hi Diana, great videos. Thank you. I stopped watching other channels and videos. I am glad I found your channel. Is there anywhere I can find a comprehensive list of the metrics for FB per product and funnel stage(reach, activation, engagement, retention, revenue, and referral )? Many thanks.
Hi, can you please add subtitles? Also wanted to ask, why the number of videos watched is a metric you would look into after the A/B test? Which videos are you talking about here?
Great content from you Dianna. You add a lot of value to the users :) My only thought would be to do the metrics after the hypothesis section. Your thoughts?
Yesss, agreed! You’ll see in the newer trade off videos I have the metrics after the hypotheses
Hi Dianna, thanks for such great video! I am wondering why do you select a set of videos instead of doing a/b testing on all the videos? Is there any criteria you used to select the testing video set?
Maybe video length plays a part
How about introducing in new subs as one metrics , if ads are too intrusive can lead to increase in subs , and can increase ARPU , also what metrics to measure is there a rule in terms of how many metrics should i mention or a structure which i need to follow here
Hi Dianna, thanks for such great video!
Hey Dianna, wouldn't a control group in this example be videos with no ads? That way we can see a group un-influenced by advertisements, and how those numbers would change with your three experiment groups.
You could add a holdout group like the one you suggested but that would be to understand the two variations from a no ads benchmark , but the question is asking about the incremental difference between the two variations so the no ads isn’t necessary
Hello Dianna, thanks for all your insightful videos and playlists. I see a conflicting point in 2 of your videos. In this video, you mention "identify metrics" as 2nd point whereas in "tradeoff FB reaction vs comment video", you mention metrics as 3rd point (metrics for those hypothesis). Which is the correct sequence? Thanks
As I figure out better ways to tackle these questions, I update videos to have the most updated best practices. So currently I’d recommend hypotheses first before metrics.
Do they expect you to talk about the p-values and confidence intervals during such tradeoff questions at FB?
I am wondering why do you select a set of videos instead of testing on all the videos? Is there any criteria that you used to select those videos to test on?
my $0.02, you don't need the entire universe of videos, just enough of them to have enough power on your A/B Test. Also, not all videos are made equal (think length mostly, especially important if you want to test middle ads, since a quick 1min video for instance likely won't have enough length to merit a mid ad).
Always proud of you
Thank you for creating these incredible, valuable, clear contents.👍✌ Please keep it up!
Too good!
Why is revenue not a key metric here ?
If it's the first time the product is launched (according to the question), acquisition, engagement, retention are always more important, IMO. Once in the growth or mature stage, revenue metics will start weighing more.
What would be the size of user groups for A/B testing
that's not something you'll need to answer in typical product mgmt interviews mostly because it'll really depend on what you're testing & require some analysis from a data scientist
love these, thanks (:
Hi Diana, how do you approach a Tradeoff question that already has metrics provided? E.g. do you ship a feature that increased watch time but reduced no of comments and likes on Facebook watch
Hey Vidhi! in that case you'd skip the experiment part and evaluate which metrics are most important to the surface we're talking about. e.g. the surface is FB Watch so which metric is most important for FB watch, is it watch time or engagement? Or which of them is correlated with higher level metrics like Daily active usage on Watch, and FB
It's a great video and I appreciate you sharing this knowledge in such an understandable and easy-to-digest form. However while I understand that today's standards let a lot of video editing slide, in your videos I find it a tiny bit tiring after a while. Most extreme example: ruclips.net/video/5lqIdfBJqZw/видео.html The first syllable is doubled and for a while it resembled a deep fried meme video :D I'm not saying it's the end of the world but if you managed to reduce the amount of cuts, I bet the overall UX of your videos would soar!
Thank you for the tip! Good call out and definitely something I’m going to continue to work on
At roughly 12:30 you said "number of ad clicks go down, but as long as total volume of clicks is going up " that confused me. How can that be ?
That confused me too. Diana Could it be explained please?
I think what she meant was % of users with an ad click goes down but since now more users see ads the total volume will go up
The background music is very annoying :(
It is very loud, and actually not needed here, only distracting...
I agree…😂 learned to stop adding music to the background for future videos