Venkat Venkataramani's '1,000x engineers now' claim does exactly what you describe: it repackages the same toxic metric in an AI wrapper. The Oso permissions data is an interesting companion — enterprises already can't audit what their human engineers touch, and now we're celebrating individual engineers who touch more. At theaifounder.substack.com I track AI builder patterns, and the measurement problem gets worse in early-stage teams, where the 10x narrative gets used to justify not hiring and not building systems. What metrics have you seen work for measuring engineering health in teams using AI coding tools, beyond velocity?
Hi Rohan, thank you for reading my take on this topic and pointing me to The AI Founder Substack, I'll check it out! I've found that a good metric to use is the Change Failure Rate (CFR): the percentage of deployments that break something in production.
Industry benchmark is 4%. Some teams using AI coding tools are now running at 6%, shipping 50% more defects than before. So keeping an eye on that is a good indicator how things are changing.
Venkat Venkataramani's '1,000x engineers now' claim does exactly what you describe: it repackages the same toxic metric in an AI wrapper. The Oso permissions data is an interesting companion — enterprises already can't audit what their human engineers touch, and now we're celebrating individual engineers who touch more. At theaifounder.substack.com I track AI builder patterns, and the measurement problem gets worse in early-stage teams, where the 10x narrative gets used to justify not hiring and not building systems. What metrics have you seen work for measuring engineering health in teams using AI coding tools, beyond velocity?
Hi Rohan, thank you for reading my take on this topic and pointing me to The AI Founder Substack, I'll check it out! I've found that a good metric to use is the Change Failure Rate (CFR): the percentage of deployments that break something in production.
Industry benchmark is 4%. Some teams using AI coding tools are now running at 6%, shipping 50% more defects than before. So keeping an eye on that is a good indicator how things are changing.