AI has made it remarkably easy to produce working software. Describe a feature, let an agent implement it, run the tests, and examine the results.
But there's an important question that can get lost in the process:
Just because the software works as specified, does that mean we specified the right thing?
This week's Ask Good Questions articles explore the difference between technical correctness and designing an application that actually helps someone accomplish their work.
The Demo Is Part of the Design
A realistic demonstration shouldn't simply prove that the software works. It should help us discover whether we've designed the right experience in the first place.
The Demo That Passed Every Test
An AI agent delivers a feature. It builds successfully, the tests pass, and everything appears to work.
But when we try to accomplish a realistic task from beginning to end, we may discover problems that none of those tests were designed to catch.
One of the things I've found particularly interesting about AI-assisted development is that it dramatically reduces the cost of experimenting with different designs.
That gives us an opportunity to do more than build software faster. We can use working prototypes to challenge our assumptions before they become embedded in the application.
The most important test may be the one we never wrote: Can someone actually accomplish what they came here to do?
I'd be interested in hearing whether others have encountered this distinction when working with AI coding agents.
Charles