Key takeaways
AI productivity risks
AI overreliance in practice
AI slop slows delivery
AI’s impact on team learning
Smarter AI governance
When AI usage in software development goes too far
Every leader responsible for software delivery has heard the pitch: ship faster, cut costs and free your developers from grunt work. And it’s not wrong, interviews with 30 software practitioners found that generative AI drives genuine, unanimous productivity gains in day-to-day development work, while a separate study of 15 senior executives described it as a decision-support and innovation amplifier. But both perspectives point to the same management question: does faster production create durable business value after review, rework, risk and capability costs are counted? When organizations push AI usage past a certain point, the same tool that saves time starts quietly taking something back.
The "illusion of competence" problem
Developers who lean on AI without staying cognitively engaged risk losing the very expertise that makes them valuable. Researchers describe this as an “illusion of competence” – teams believe they understand a solution because it works, only to discover the gaps when something breaks. Teams ship features successfully while individuals stop developing debugging skills, technical confidence and design judgment – a pattern researchers call “leadership atrophy”. Team members challenged on an AI-generated design often have no rationale to defend it because they were not part of the critical thinking behind it.
This is more than an individual skills issue. The executive research identifies workforce readiness, continuous learning and human oversight as conditions for sustainable AI adoption. When these capabilities weaken, the organization becomes more dependent on a smaller group of experts to validate AI output and recover when it fails, creating a long-term resilience risk.
Overreliance has a name, and a growing record of incidents
Practitioners describe a recurring “mistrust of AI,” rooted in unclear data handling and unvalidated output. That mistrust is not paranoia: an AI coding agent once deleted a live production database and made unauthorized changes affecting over 1,200 executives and 1,190 companies. Separately, developers who relied heavily on AI tools persistently underperformed at neural, linguistic and behavioral levels compared with those who did not. For managers, failures like these can become service disruption, regulatory exposure, customer harm and reputational damage.
Developers have coined a term for the result: “AI slop” – low-quality, AI-generated output marked by superficial competence. The costs are real: one team received 30 AI-assisted pull requests per day across just six reviewers, shifting the burden from the person who generated the code to the person who had to verify it. From a business perspective, that is potential ROI erosion: time saved during generation can reappear as review effort, rework, defects, technical debt and maintenance cost. The executive study also identifies legacy systems, limited AI expertise and enterprise integration as costs and barriers that managers must consider before scaling.
Excessive AI use also erodes collaboration. As developers turn to AI first for small technical questions, junior-senior interactions decline, weakening the trust that collaboration depends on. In agile teams, daily stand-ups have become “performative” status reports instead of collaborative problem-solving – “ritual erosion”. At the organizational level, weaker interaction also threatens knowledge transfer, team cohesion and the development of future senior expertise.
What leaders should do about it
The answer is not to reject AI. It is to manage adoption as a business system rather than a software-tool rollout:
Pilot before scaling. Require each use case to demonstrate both value and compliance, then expand only when downstream teams can absorb the additional output.
Measure end-to-end value. Track review hours, defect and rework rates, maintenance backlog, incident recovery, integration and training costs, and the business outcomes delivered—not only developer speed or AI usage.
Keep a named human accountable. No AI-generated change should reach production without an owner who understands it, can explain it and has authority to stop or reverse it.
Match autonomy to risk. Restrict production and data access, define escalation paths and apply stronger oversight to customer-facing, regulated or business-critical systems.
Protect organizational capability. Preserve mentoring, cross-training and periodic AI-free practice so critical knowledge is distributed and recoverable rather than concentrated in a tool or a few senior people.
Govern cross-functionally. Bring business, engineering, operations, legal and security together to revisit value, risk and workforce readiness as tools and regulations change.
The dividing line is not between using AI and avoiding it. It is between adoption that strengthens the operating model and adoption that quietly consumes review capacity, increases exposure and weakens the organization’s ability to respond. For managers, the right objective is not “more AI, faster”; it is better business performance with controlled risk and resilient human capability.
Take a look at how your team actually uses AI day to day - where it's saving time, and where it might be quietly costing you skills, trust, or review capacity.
Use the bulletpoints as a starting point and bring it to your engineering leads to start the conversation before reliance turns into risk.
