Metrics Flashcards
(34 cards)
Work in progress
Open PR with activity in the last 72 hrs
Why: A low WIP will lead to improved cycle time
How: Look at this number in relation to Active Contributors in your group.
Active contributors
Individuals who authored or pushed a commit in the last 24 hours
Why: When team members are available to code, it can be a strength
How: Use it to get a better sense of whats impacting availability.
PR Activity
Total time that has gone into a pull request.
Why: CD requires work to be done in small batches in order to ship frequently
How: How amount of activity w/in a PR could signal an issue
Weekly coding days
Avg numbers of days per week that a Dev spends coding.
Why: Represents a teams capacity.
How: When the number is low it warrants a re-priortization on coding. When the number is high it shows that Devs are being over-worked.
Commits per day
Avg number of time code is committed per day
Why: Smaller commits represent more frequent checkpointing.
How: A low number of commits can indicate a bottleneck, or issue with codebase.
Pushes per day
Avg number of pushes per day
Why: Serves as another checkpoint
How: Frequent pushes relates to working in small batches and incremental changes.
Rework
Percentage of rewrites to code.
Why: Rework or churn represents efficiency at the coding level.
How: High Rework can be due to late changes in product requirements, or unfamiliarity with the codebase.
Time to open
Proxy for how long it takes to develop something.
Why: Provides visibility of an engineer’s work to the rest of the team.
How: may indicate that they don’t have clearly established norms around when pull requests are ready for review.
PR size
LOC that were changed, added or removed
Why: Smaller PR’s are typically easier to review.
How: When this number is high, its typically a good coaching opportunity.
Impact
Magnitude of changes to the codebase over a period of time.
Why: This is a synthetic measure we developed to help teams understand the significance of all the changes to the code.
How: Impact represents the collective significance of commits to the codebase.
Review Cycles
Time it takes for a PR to go back and forth between author and reviewer.
Why: It’s important to keep cycle times low
How: It gives you a better sense of where bottlenecks can be occurring in your process.
Unreviewed PR requests
The amount of PR’s that were merged without review
Why: Unreviewed PR’s represent a risk to your codebase.
How: Gain a better understanding of your of your review process.
PR’s reviewed
The count of pull requests that have been reviewed.
Why: Helps you understand how well the burden of code review is distributed amongst the team.
How: Requests Reviewed may be a sign of conflicting priorities, disengagement, or difficulty with the codebase or process.
Review speed
Time between when a pull request is opened and when the reviewer first submits feedback.
Why: you’ll want to make sure that work is consistently moving through the software development process.
Review coverage
The percentage of files changed that receive at least one code review comment.
Why: Review Coverage is a proxy for review thoroughness
How: Will give you an understanding of how the quantity of feedback compares to its actionability.
Review influence
Review comments that are addressed either by a response comment or a change to the code.
Why: Shows you how meaningful review feedback is to pull request authors
How: Use it to ensure that the team is leaving meaningful comments.
PR throughput
A count of how many pull requests are merged over a period of time
Why: A total count of merged pull requests can serve as a proxy for value delivered.
How: This metric signals whether your engineering organization is getting more or less productive
Cycle time
The time between when the first commit is authored to when a pull request is merged.
Why: Cycle Time represents your team’s time-to-market, or how quickly software is delivered to customers.
How: Use it to understand baseline productivity.
PR Success rate
The percentage of opened Pull Requests closed that are successfully merged.
Why: Measurement of overall efficiency
How: Determine what’s effecting the rate (changing requirements, unclear technical direction, or implementation challenges) and course correct
Revert rate
Percentage of total pull requests opened that are reverts
Why: Reflects the changes that caused the author to reverse the change
How: Reverts identify waste or defect in the process.
Change type
The portion of your team’s efforts that are spent on developing new features, refactoring, and rework.
Why: New Code Written can be used as a proxy for raw innovation; Updating Older Code represents refactoring; and Rework is a form of churn.
How: Depending on your org’s structure, this information can mean different things.
Time to merge
The duration between when a pull request is opened and when it is merged.
Why: Time to Merge is an indicator of how much inventory your team is managing at any given point in time.
How: When this metric is high, it’s usually a good time to re-evaluate collaborative processes. It may be that the latter end of the software delivery process has too many bottlenecks.
Reviewers count
The total number of unique reviewers per pull request.
Why: The Reviewers Count shows you how many developers have to context switch in order to make time for code reviews.
How: If its low then expectations haven’t been clearly communicated. If its high then there may be over-review taking place.
File count
The total number of files changed in a pull request.
Why: File Count is another way of understanding the magnitude of a pull request
How: This isn’t a metric you need to optimize.
