Can We Rely on Unpredictable Systems to Build Predictable Ones?
A reflection on what changes when probabilistic AI systems become part of software systems we expect to behave reliably.
Software teams are used to building systems where the same input should produce the same output. Probabilistic AI breaks that assumption in useful and uncomfortable ways.
The question is not whether we should use these systems. We already are. The question is where unpredictability belongs — and where we still need guarantees.
Reliability is a design choice
When AI becomes part of a product, reliability stops being a property of the model alone. It becomes a property of the system around the model: validation, fallbacks, human review, observability, and clear boundaries on what the system is allowed to decide.
Predictable software still matters. We just have to be honest about which layer is predictable and which layer is probabilistic.
What I am watching
I am interested in architectures that treat AI as a collaborator inside a reliable system, not as a replacement for system design. That usually means smaller surfaces, clearer contracts, and more explicit human checkpoints than the demo suggested.