Product Execution Flashcards
Product execution: Goal-setting and metric definition
https://medium.com/agileinsider/how-i-prepared-for-facebook-pm-interviews-execution-metrics-8067d942a6d1
Goals: Being mindful of how the goals (especially quantitative goals) can be gamed or how they can sometimes be counter indicative of progress.
Metrics: What would you use to measure if the product is healthy? Which one would you prioritize? What happens if one is decreasing and the other is increasing?
Framework:
1) Clarify
2) Product + company mission
3) Goal
4) Key users & stakeholders
5) Key actions
6) Metrics (types: solution success, adoption success, system health)
7) Tradeoff table
8) Summarize
GAME method:
1) Goals - of the product (e.g., engagement)
2) Actions - the user can take in the product (e.g., create, view, comment, share, like)
3) Metrics - for each action above (types: solution success, adoption success, system health)
4) Evaluations - summary, edge cases, warnings
Product execution: Debugging / Root cause analysis
https://medium.com/agileinsider/how-i-prepared-for-facebook-pm-interviews-execution-metrics-8067d942a6d1
Debugging: Say you notice a specific metric dropping week after week. The interviewer will present a problem statement, and you should ask questions to describe how you’d approach this challenge and determine what’s causing this metric to drop.
Framework 1:
1) Clarify - understand the metric, over what time?
2) Segment - scope the change
3) Contextual - is this seasonal, political change? is anything else down?
4) Internal hypothesis - data accuracy, bug, outage?
5) External hypothesis - competition, user habits, geo-political, pop-culture?
(ME-CE: mutually exclusive - collectively exhaustive)
Product execution: Trade-off between A and B
https://medium.com/agileinsider/how-i-prepared-for-facebook-pm-interviews-execution-metrics-8067d942a6d1
Navigating a complex trade-off: “A” or “B” option—how do you know what to show to which communities of users?
Decision framework 1 (RICE): 1) Reach 2) Impact 3) Confidence 4) Effort RICE Score = R*I*C / E
Answer framework:
1) Clarify
2) Product + company mission
3) Goal
4) Key users & stakeholders
5) Key actions
6) Metrics (types: north star, solution success, adoption success, system health)
7) Features
8) Pros and cons
9) Experiment design - A/B test
10) Decision framework - RICE
11) Tradeoff table
12) Summarize
Trade-off: You are the PM for Facebook live — what features would you prioritize?
1) Clarify: FB Live, Launching / launched?
2) Product + company mission: Empowering people to build communities and bring the world closer together.
3) Goal: Empower people to share what’s going on in their world live. Allow people to engage with what’s going on with the world live.
4) Key users & stakeholders: Creator (Media, Users), Consumer (Connections, Strangers)
5) Key actions: Create, View, Share, Comment, React
6) Metrics (types: north star, solution success, adoption success, system health)
Creator:
- # of total Live sessions (type: adoption success)
- # of Live sessions per active user (type: adoption success)
- % of stable sessions (type: system health)
- # of concurrent live sessions (type: system health)
Consumer:
- Engagement - shares, comments, reactions (type: adoption success)
- # of views (type: solution success)
- # of viewers per session (type: solution success)
7) Features:
For creator:
- Acquisition: Big, attractive button
- Activation: Notification, prompts to go Live at interesting moments of activity
- Retention: Sharing the highlights of the previous Live session to encourage again
For consumer:
- Discoverability: Notifications
8) Pros and cons
9) Experiment design
10) Decision framework - RICE
11) Tradeoff table
12) Summarize
Trade-off: You are the PM for Facebook pages — what features would you prioritize?
TODO
Goal-setting and metric definition: You are the PM for Facebook posts — what reactions should we add next?
TODO
Goal-setting and metric definition: How would you set goals and measure success for Facebook live?
Framework:
1) Clarify - FB Live?
2) Product + company mission: Empowering people to build communities and bring the world closer together.
3) Goal: Empower people to share what’s going on in their world live. Allow people to engage with what’s going on with the world live.
4) Key users & stakeholders: Creator (Media, Users), Consumer (Connections, Strangers)
5) Key actions: Create, View, Share, Comment, React
6) Metrics (types: solution success, adoption success, system health)
Creator:
- # of total Live sessions (type: adoption success)
- # of Live sessions per active user (type: adoption success)
- % of stable sessions (type: system health)
- # of concurrent live sessions (type: system health)
Consumer:
- Engagement - shares, comments, reactions (type: adoption success)
- # of views (type: solution success)
- # of viewers per session (type: solution success)
7) Tradeoff table
8) Summarize
Goal-setting and metric definition: How would you set goals and measure success for Facebook notifications?
TODO
Goal-setting and metric definition: How would you set goals and measure success for Instagram stories?
TODO
Debugging / Root cause analysis: Facebook groups usage dropped 10% — what do you do?
1) Clarify - Usage means visits, posts or something else?
2) Segment - When did the dip start? How long did it last? Was it on mobile, web or everywhere? Was it on a certain geo? Was it on a certain genre or type (size, etc.) of group?
3) Contextual - is this seasonal, political change?
4) Internal hypothesis - data accuracy, bug, outage?
5) External hypothesis - competition, user habits, geo-political, pop-culture?
(ME-CE: mutually exclusive - collectively exhaustive)
Debugging / Root cause analysis: Facebook ads revenue dropped 20% — what do you do?
TODO
Debugging / Root cause analysis: Facebook newsfeed engagement dropped by 2% — what do you do?
TODO
Goal-setting and metric definition: You are the PM for Facebook newsfeed — how would you rank posts?
TODO
North Star metrics
Revenue (sales) Customer growth (paying users, marketshare) Adoption (features enabled) Engagement (MAU, WAU, DAU) User experience (NPS)
Estimation question / Fermi problem
Framework:
1) Clarify - e.g., which revenue? (ads or everything), which customer? (consumer or enterprise), which locale? (EMEA or WW), how are the revenues calculated?
2) Come up with different formulae:
- Equation 1 (users X views per user X $ per view)
- Equation 2 (% of total revenues)
3) Calculate each part by asking questions / making assumptions / getting outside research, extrapolating, etc.
4) Sense-check results
5) Conclude
