My key take aways.. If faced with a new launch situation - 1. Think about what kind of feature was it (UX,UI, Backend,Algo) 2. Discuss and clarify launch criteria - what were the metrics that were decided 3. Discuss the tradeoffs that were decided with the launch Keep in mind the User Attention Span with UX(Social Media) platforms Also, ask how are the other related metrics trending. When it is a downtrend metric question - Breakdown the metric to - Geography, Technology, Time, Trend, Source, Category
This is an excellent response, but it could have been made more concise. Bringing what she described in the beginning to the end made it seem like an awesome chess move. Definitely, she is an intuitive most likely an INTP. If I were a tired interviewer, I would zone out during the interview, because she took too long to get to the point.
One of the best and confident responses to the RCA kind of questions. Loved the way she started with pre-cursors to this launch rather than diving straight into the usual framework of asking clarifying questions. Always good to get the bigger picture first.
Hey Amit! Glad you found this video useful! Watching others approach questions is a great way to expose yourself to different approaches or ways of thinking. Hope you'll find our other videos just as useful!
Great video, I loved watching her go super in depth. I'd love to see her do a mock interview again for a non-Google product though, as I'm not sure how normal this would be for an interviewee with no prior working knowledge of the product they're interviewing for.
There is nothing particularly special about the metrics that she presented that a reasonably experienced PM could not deduce from perusing the product. :)
@@Theartsygalslays I do think though that her familiarity with RUclips helped her come up with product ideas to mitigate the watch time decrease, like reducing the screen space for comments, changing the ranking algorithm, and checking for spam & abuse.
Excellent trade-off and goal analysis! One thing that I didn't hear the analysis hit on as deeply till the very end, and which would be a base hypothesis from my perspective, is around comment QUALITY -- she did surface the concern about spam, but that isn't an only concern with comment integrity. One reason some videos have comments turned off completely is exactly this concern with quality: bullying in comments, trolling, etc. (none technically spam per se). Even valid comments can be dissuading: it's possible that users scroll through the comment content before even deciding to engage with the video, and they might leave / click away if discouraged (though again, we should be able to see some scroll activity through the comments, in that case, and the click path -- e.g., whether they click back to the search results or to one of the recommendations from there). We could possibly look at whether there's additional reduction in other activity: for example, are viewers doing less than before initial playback (first 10 s), skips forward, etc., because they already got what they needed out of comments and the video description?
This is one of the best interviews on metrics questions. The interviewee clearly demonstrates that the deep knowledge she has on each part of the RUclips features.
Great answer. Could have been even better if she had structured all of this on a Google doc while sharing her screen.. that would also force her to slow down and not have the interviewer lose focus
Very good response from her. Got an insight on how actually product managers think on a daily basis when making a go/no-go decision or when diagnosing some issue. However, I felt that this interview could have been more interactive instead of her laying forward all scenarios and then making her own assumptions.
Some of the key takeaways for me atleast are: Consideration of RUclips Layout across both Web and Mobile App. Consideration of language semantics across countries in the comments section. She pulled this execution nicely. However, these days we see Ads just below the comments and then the recommended videos. But overall a great mock for an aspiring PM like me. Thank you Exponent ❤
I thought this might have been improved at the start. She tries to explain pre-launch criteria, saying "it depends on what the % metrics were on comments engagement vs. watch time, and we would just decide on that", I would have maybe done this framework: - Clarify the feature that was launched and what it looks like e.g. the UI - Understand the metrics and how they are calculated - What was the exact metric numbers and averages, any idea? - Clarify if the metric changes were global, on specific versions on mobile, specific geos, demographics etc Then, see why this might have happened: - External - did something external impact the launch? e.g. competitors launching? Is this across the entire RUclips e.g. watches going down overall not just due to the launch? Did we change the make-up of the videos to be shorter? Were there some other platforms that people find RUclips videos e.g. Facebook or Twitter impacted? - Internal - were there other products in the launch that launched in same time that caused watch time? Did the feature only launch on short videos by accident due to a bug? Did people engage with the comments more than the videos so naturally watch time went down? Did we change the UI in some way that cannibalised focus on the videos watched? Was engagement on comments more on short videos than long videos which affected the mix? - Out of control - was there something that caused the watch time to be decreased?
Great interview - I would thought that to begin with, i may clarify the reason why we need to ensure viewership while the comment starts to grow. In another word, what would be a healthy matrix of comment vs. viewing time , and what is the busienss reason behind it ( e.g. ads income)
this is pretty good. however, i feel that she doesnt really engage with her interviewer as much. might just be a personal thing. but it just seems like she is just rambling a lot.
Very strong interviewee. I think she might can stop when she asks questions about geographic regional decline, gradual decline etc instead of making assumptions
The title is misleading. This is not a RCA since the cause is already stated. “You shipped RUclips comments on mobile”. So this is a trade off question and next steps
She's assuming that this dip in comments is happening at the launch of the comments feature, but i could easily see a case where comments was successfully launched w/ stable watch times in the past but in the present, watch time took a dive for some unrelated reason (not because of comments)
She merely listed her assumptions and thought process to narrow down the problem, create some context and triangulate possible root causes - just like in the real world where a high level question needs to be scrutinised. Not shocking for a PM’s role in this.
It's a diagnostic exercise: you keep going and ruling out possible causes, and discussing how you might do that (e.g., looking at logs, looking at other metrics, etc.).
Recommendations being generated but not being shown on screen. RUclips servers down which led to drop in viewership. Aren't these too far flung reasons to be discussing? You'd assume that Google would test new features before launch (recc not visible) and that you and the whole world would know about recent downtimes for Google.
this interview was not good. she just started listing out potential issues rather than talk with the interviewer and actually find the root cause. sure, the analytical skills are there but she didn't really get to the answer as you would be expected to do in such RCA interviews..
Surprising that ranking of comments didn't come up (likes, user replies, creator replies, aggregate..). Could have brushed upon A/B testing for these. Especially when so much of the interview was about recommendations of Next Up videos
In all the mock PM interviews, the candidate keeps asking questions which is great. But the interviewer's answer is always so predictable. "For now let's just take "/"Sure, that sounds good" If the interviewer would provide more meaningful input, it would be more encouraging to engage with them. Now it just sounds like something everyone has to do (to tick a box) that is not really adding value.
Here is my feedback. She used to be PM on RUclips. So she is familiar about the product and her interview should have been about other products like airline products something she is not familiar with.
Many companies, including Google and Facebook, ask questions about their own products, so this is a very typical prep video. Her being at Google ads credibility.
@@Theartsygalslays a ton of companies outside of FAANG ask case studies like this. So no, this is not a very helpful video to anyone who is objectively looking for a realistic answer. Companies do not cater to the product you're interviewing for. So everyone saying Oooo and Wowwww to an answer like this are kidding themselves when it comes time for their case study about a product they have rarely used. I dont understand the responses in most of these comments sections as though PMs are magically experts in every single application in the tech marketplace and should be expected to provide this level of insight.
I feel like youtube actually considered here idea at 14:31. She mentioned to mention 1 comment and then showing recommended videos, which is the current implementation of the RUclips mobile.
Wouldn't the experiment setup be - 1. Control group - Already has comments rolled out to these users and with a decreased watch time 2. Treatment group 1 - 1 recommendation video above the comments section 3. Treatment group 2 - 3 recommendation video above the comments section 4. Treatment group 3 - User education prompts about recommendations below the fold 5. Treatment group 4 - A different tabbed layout that makes it easier for users to navigate between comments and recommended videos We can try different solutions, but the important point here is that the experimental setup would be against the current truth of the system which is where comments has already been shipped and the treatment groups should show an 'increase' in the watch time.
To me it felt organic. But nothing wrong with having an outline up in front of you while interviewing, as well. It’s encouraged to structure your thoughts to avoid rambling and getting off tangents.
She gave an excellent response in spite of a terrible interviewer. Especially that last question about which testing arm to select was just terrible. She's a rockstar!
Don't leave your product management career to chance. Sign up for Exponent's PM interview course today: bit.ly/3iT2D5u
My key take aways..
If faced with a new launch situation -
1. Think about what kind of feature was it (UX,UI, Backend,Algo)
2. Discuss and clarify launch criteria - what were the metrics that were decided
3. Discuss the tradeoffs that were decided with the launch
Keep in mind the User Attention Span with UX(Social Media) platforms
Also, ask how are the other related metrics trending.
When it is a downtrend metric question - Breakdown the metric to - Geography, Technology, Time, Trend, Source, Category
Hey would you please shed some more light on comment "Downtrend metric question".
Her communication skill and clarity of thoughts are spots on. She never trembled. Very calm and confident approach.👏
I wish I get to this level in terms of deep diving and confidence during my interviews, this was a great one to watch. Thanks for posting
definitely true
so relatable.
This is an excellent response, but it could have been made more concise. Bringing what she described in the beginning to the end made it seem like an awesome chess move. Definitely, she is an intuitive most likely an INTP. If I were a tired interviewer, I would zone out during the interview, because she took too long to get to the point.
One of the best and confident responses to the RCA kind of questions. Loved the way she started with pre-cursors to this launch rather than diving straight into the usual framework of asking clarifying questions. Always good to get the bigger picture first.
Hey Amit! Glad you found this video useful! Watching others approach questions is a great way to expose yourself to different approaches or ways of thinking. Hope you'll find our other videos just as useful!
Great video, I loved watching her go super in depth. I'd love to see her do a mock interview again for a non-Google product though, as I'm not sure how normal this would be for an interviewee with no prior working knowledge of the product they're interviewing for.
There is nothing particularly special about the metrics that she presented that a reasonably experienced PM could not deduce from perusing the product. :)
@@Theartsygalslays You are right, just that its a good learning experience for younger PMs
@@Theartsygalslays I do think though that her familiarity with RUclips helped her come up with product ideas to mitigate the watch time decrease, like reducing the screen space for comments, changing the ranking algorithm, and checking for spam & abuse.
I am impressed by how still the interviewer can be. I thought the screen froze for a second lol.
🤣
Excellent trade-off and goal analysis!
One thing that I didn't hear the analysis hit on as deeply till the very end, and which would be a base hypothesis from my perspective, is around comment QUALITY -- she did surface the concern about spam, but that isn't an only concern with comment integrity. One reason some videos have comments turned off completely is exactly this concern with quality: bullying in comments, trolling, etc. (none technically spam per se). Even valid comments can be dissuading: it's possible that users scroll through the comment content before even deciding to engage with the video, and they might leave / click away if discouraged (though again, we should be able to see some scroll activity through the comments, in that case, and the click path -- e.g., whether they click back to the search results or to one of the recommendations from there). We could possibly look at whether there's additional reduction in other activity: for example, are viewers doing less than before initial playback (first 10 s), skips forward, etc., because they already got what they needed out of comments and the video description?
exactly, whole interview was just fixated on a single hypothesis that its a design flaw.
Have a Google interview in 4 hours. Wish me luck!
All the best Ramya!
Let us know here how it went
Went well!
2 down. 2 to go!
Wish me luck! :)
Are you based in the Bay Area?
@@sid4752 No, I’m based out of India.
Update: Done with final round this week. Up to the hiring committee now 🤞
I felt like she definitely rambled and could have been more concise
Shes a PRO !
Is it just me or did anyone feel that the conversation was very one sided ? I felt that there weren't enough check ins with the interviewer.
It's not one sided,she has suggested certain hypothesis which can be validated to deduce the root cause.
This is how a RCA should be done ✅
This is one of the best interviews on metrics questions. The interviewee clearly demonstrates that the deep knowledge she has on each part of the RUclips features.
Great answer. Could have been even better if she had structured all of this on a Google doc while sharing her screen.. that would also force her to slow down and not have the interviewer lose focus
Great opening on how she planned to address this concern....
I really liked how the interviewee started with questions, looked at evidence, then hypothesized a potential cause.
Very good response from her. Got an insight on how actually product managers think on a daily basis when making a go/no-go decision or when diagnosing some issue. However, I felt that this interview could have been more interactive instead of her laying forward all scenarios and then making her own assumptions.
This is the best interview from Exponent.
fab mock interview. she seems to be very learned and also seems to have a super nice work experience in product management. kudos to her.
Some of the key takeaways for me atleast are:
Consideration of RUclips Layout across both Web and Mobile App.
Consideration of language semantics across countries in the comments section.
She pulled this execution nicely. However, these days we see Ads just below the comments and then the recommended videos. But overall a great mock for an aspiring PM like me.
Thank you Exponent ❤
Is this guy even a human? He hasn't blinked for like half an hour!!
😀😂
Really appreciate her analytical skills
gosh the interviewer is so mortifying! the poker face can get anyone nervous. great video btw.
10:18 Kevin watching like, he is in love. Great video, love the flow of the conversation 😃
bruh the guy be looking into our souls
😂😂😂
I thought this might have been improved at the start. She tries to explain pre-launch criteria, saying "it depends on what the % metrics were on comments engagement vs. watch time, and we would just decide on that",
I would have maybe done this framework:
- Clarify the feature that was launched and what it looks like e.g. the UI
- Understand the metrics and how they are calculated
- What was the exact metric numbers and averages, any idea?
- Clarify if the metric changes were global, on specific versions on mobile, specific geos, demographics etc
Then, see why this might have happened:
- External - did something external impact the launch? e.g. competitors launching? Is this across the entire RUclips e.g. watches going down overall not just due to the launch? Did we change the make-up of the videos to be shorter? Were there some other platforms that people find RUclips videos e.g. Facebook or Twitter impacted?
- Internal - were there other products in the launch that launched in same time that caused watch time? Did the feature only launch on short videos by accident due to a bug? Did people engage with the comments more than the videos so naturally watch time went down? Did we change the UI in some way that cannibalised focus on the videos watched? Was engagement on comments more on short videos than long videos which affected the mix?
- Out of control - was there something that caused the watch time to be decreased?
Woah Dhruv! Thanks for engaging with the content and taking the time to share your improvements! 💪
Great interview - I would thought that to begin with, i may clarify the reason why we need to ensure viewership while the comment starts to grow. In another word, what would be a healthy matrix of comment vs. viewing time , and what is the busienss reason behind it ( e.g. ads income)
completely wowed by her clarity
I would have also considered mentioning if ads are the possible cause but overall this was great interview 👍
this is pretty good. however, i feel that she doesnt really engage with her interviewer as much. might just be a personal thing. but it just seems like she is just rambling a lot.
Very strong interviewee. I think she might can stop when she asks questions about geographic regional decline, gradual decline etc instead of making assumptions
I loved the insightful questions and answers she came up with during the discussion.
Thank you for theses videos. They are really helpful. The interviewee took a well thought through approach to resolve the challenge
What PM level get this type of question? PM1 or 2, Senior PM, Group PM?
Random answer!! Should have asked clarifying questions instead of just saying own thoughts!
Not a PM but that was a very good analysis ! Just gives me more breadth
The title is misleading. This is not a RCA since the cause is already stated. “You shipped RUclips comments on mobile”. So this is a trade off question and next steps
She's assuming that this dip in comments is happening at the launch of the comments feature, but i could easily see a case where comments was successfully launched w/ stable watch times in the past but in the present, watch time took a dive for some unrelated reason (not because of comments)
That was not the question though.
Ridiculously good articulation! And yes, internationalisation is a horrible word.
Loved her approach
Maybe I'm missing something but didn't she just change the question in first 6 mins?
She merely listed her assumptions and thought process to narrow down the problem, create some context and triangulate possible root causes - just like in the real world where a high level question needs to be scrutinised. Not shocking for a PM’s role in this.
Just to add , maybe users are getting irrate due to multiple ads being displayed n dy get less engaged.
Correct me if i'm wrong...
So we don't actually have to reach to a root cause for a root cause analysis. Probable reasons are good enough.
It's a diagnostic exercise: you keep going and ruling out possible causes, and discussing how you might do that (e.g., looking at logs, looking at other metrics, etc.).
Loving the youtube comments all of a sudden
Recommendations being generated but not being shown on screen.
RUclips servers down which led to drop in viewership.
Aren't these too far flung reasons to be discussing?
You'd assume that Google would test new features before launch (recc not visible) and that you and the whole world would know about recent downtimes for Google.
this interview was not good. she just started listing out potential issues rather than talk with the interviewer and actually find the root cause. sure, the analytical skills are there but she didn't really get to the answer as you would be expected to do in such RCA interviews..
Surprising that ranking of comments didn't come up (likes, user replies, creator replies, aggregate..).
Could have brushed upon A/B testing for these.
Especially when so much of the interview was about recommendations of Next Up videos
In all the mock PM interviews, the candidate keeps asking questions which is great.
But the interviewer's answer is always so predictable. "For now let's just take "/"Sure, that sounds good"
If the interviewer would provide more meaningful input, it would be more encouraging to engage with them. Now it just sounds like something everyone has to do (to tick a box) that is not really adding value.
One of the best interviews
she gave good background. but i feel that she over indexed her time on that and did not dive quickly in on execution on an issue.
The scoping question was actually a scoping speech
Here is my feedback. She used to be PM on RUclips. So she is familiar about the product and her interview should have been about other products like airline products something she is not familiar with.
Many companies, including Google and Facebook, ask questions about their own products, so this is a very typical prep video. Her being at Google ads credibility.
@@Theartsygalslays a ton of companies outside of FAANG ask case studies like this. So no, this is not a very helpful video to anyone who is objectively looking for a realistic answer. Companies do not cater to the product you're interviewing for. So everyone saying Oooo and Wowwww to an answer like this are kidding themselves when it comes time for their case study about a product they have rarely used. I dont understand the responses in most of these comments sections as though PMs are magically experts in every single application in the tech marketplace and should be expected to provide this level of insight.
Amazing interview. Perfect.
8:15 hitting the nail right there. I wonder after all this time how google is managing this issue.
I feel like youtube actually considered here idea at 14:31. She mentioned to mention 1 comment and then showing recommended videos, which is the current implementation of the RUclips mobile.
What are you saying? She's THE PM at RUclips. Maybe she was already leading the team building it at the time :D
Pre roll and post roll ads is another reason, introduction lot of ads
That would not be mobile specific
Man, RCA is where my interviewing is the weakest in.
Does she have a background in consulting? Very hypothesis-driven solution
Cherrie is really good.
What is a backstop metric?
A counter metric that you should be watching.
@@bedantamadhabgogoi sometimes also called guardrail metrics.
Wouldn't the experiment setup be -
1. Control group - Already has comments rolled out to these users and with a decreased watch time
2. Treatment group 1 - 1 recommendation video above the comments section
3. Treatment group 2 - 3 recommendation video above the comments section
4. Treatment group 3 - User education prompts about recommendations below the fold
5. Treatment group 4 - A different tabbed layout that makes it easier for users to navigate between comments and recommended videos
We can try different solutions, but the important point here is that the experimental setup would be against the current truth of the system which is where comments has already been shipped and the treatment groups should show an 'increase' in the watch time.
She is next level
This dude doesn't smile.
Just like most of interviewers :)
Stop finding negative things about the host and the guest and appreciate what they are sharing for you to get value for free!
@@omaralmunawer yes omar u r right 👍
OMG SHE IS GOOD
Got it!
My key take away is:
I love her
Sweet at 1:45 was sweet xD
She is amaing
Can't stop myself from asking this, is she reading out of a script?
To me it felt organic. But nothing wrong with having an outline up in front of you while interviewing, as well. It’s encouraged to structure your thoughts to avoid rambling and getting off tangents.
I just started watching this one and NGL it seems so because of her pauses and tone.
Sweet
bro is staring inside me
looks like shes reading from something..
Just because someone has confidence doesn't mean it is the right approach!
I never seen anybody giving this much good PM interview
Bla bla bla
I don't think it's very good.
Especially her answer to the A/B test, which shows her little understanding of it.
Can you provide some more constructive criticism on what you thought was wrong with it?
She gave an excellent response in spite of a terrible interviewer. Especially that last question about which testing arm to select was just terrible. She's a rockstar!
Why was the interview terrible?
Excellent Brainstorming!