Are we grading engineers?

The ask

The dashboard was already built when research joined. There had been no discovery work. The product manager's request was simple: run a focus group and gather feedback.

Feedback on screens would have produced a list of fixes. It would not have answered whether the tool should exist in its current form. So before writing a discussion guide, I asked two questions of my own: what is developer productivity, and what have other organizations learned about measuring it? That reading shaped the sessions. We still demoed the product, but purpose was on the agenda alongside usability.

What we heard

Sentiment was negative across all three groups. The strongest reactions came in the first minutes of the demo, at the summary tiles.

"I must admit this is the second time I've seen [the dashboard], it actually made me angry. Are we trying to grade engineers based on how many times they commit? This data can be used in a very negative way."

"I worry more than I see the benefit."

Developers: activity isn't the work

• Commits misrepresent real work. Large tasks start with days of analysis and no commits. Reviews, design and research produce none. One chart's empty ring read as "I didn't commit any code."
• Streaks invite gaming. "I need to get my streak going. Let me get a couple of defects so I can get my streak up."
• After-hours metrics reward the wrong thing. Weekend commits could be read as a positive. One participant said simply: "I wouldn't show it at all."
• Numbers need context. Meetings and planning explain quiet days. "Numbers are deceptive and need context."

Managers: show me the team, not a ranking

When I asked an engineering manager whether the tool would make him more productive, he answered: "Short answer... no." He knows what he did this week from his tickets and sprint goals. His worry was for others: "As an experienced developer that may not worry me, but as an inexperienced developer I may work nights to squeeze my commits in because it looks bad."

He did see value at the team level. If a team worked three nights in a row to squeeze code in, that tells a manager the sprint plan didn't work. Managers also asked for practical fixes: totals that add up, every engineer in one sortable list instead of pages, and open pull requests sorted by how long they've waited.

The words did the damage

The labels on the page decided how people read the data. "Top contributor" implies a leaderboard. "Activities outside business hours" reads as surveillance. Renaming the most charged labels was the cheapest trust fix available.

Context makes data useful

The weekday heat map was the view participants liked most, but only once it could explain itself. A light Tuesday might mean recurring meetings, not a slow engineer.

Recommendations

1. State the purpose and privacy model in the product: activity for understanding how work flows, not individual performance or policing.
2. Rename disciplinary labels, and remove "top contributor" and after-hours metrics.
3. Drop commit streaks from the landing view.
4. Add context to activity views: meetings, sprint phase, and the time range in view.
5. Fix what managers need: reconciled totals, one sortable list, oldest open pull requests first.
6. Restart with engineers in the room: a design-thinking workshop with focus-group members to choose which metrics matter, then a blue-sky redesign.

Three of these shipped: the commit streak came off the landing view, the disciplinary labels were renamed, and weekend activity was made neutral rather than highlighted. This study was the first of four on measuring engineers; the synthesis is in Measuring developer productivity without breaking trust.