The Specs Are Different Now for Probabilistic Software

Building an AI product often starts with a comforting illusion: writing the specification document the way we always have. We list the user stories. We define the acceptance criteria. We assume that if the engineering team builds exactly what we wrote, the system will behave exactly as we expect.

But AI operates in probabilistic environments. A vision-based system might identify a person walking perfectly, but completely miss a pedestrian in a wheelchair or someone wearing a heavy winter coat. Traditional deterministic specifications fail here because they cannot capture the infinite, intuitive variations of the real world. We are no longer building systems where logic paths dictate outcomes; we are building systems where complex data patterns dictate behavior. The specs are different now.

Key takeaways

• Traditional requirements engineering relies on static verification, which fails for probabilistic AI models that derive behavior from complex data patterns.
• Without formal, dynamic specifications, evaluating AI behavior relies on subjective human intuition or flawed assumptions about data volume.
• A 2025 study from Northern Illinois University demonstrates a framework that translates semantic ontologies into visual datasets to evaluate an AI’s actual perception.
• Product teams must shift from defining what the software does to defining how the software perceives and continuously mapping that against expected variations.

The Limits of Deterministic Specs

In conventional software development, a requirement is a hard contract. If the user clicks a specific button, a modal opens. If the backend logic is correct, the test passes. When product teams bring this deterministic mindset to AI-based software, they quickly hit a wall.

As noted in a 2025 study on AI-driven requirements engineering by Barzamini et al., traditional verification techniques fall short for AI systems. The challenge stems from the intuitive nature of domain concepts. Humans instinctively understand that a person on a skateboard, a child in a stroller, and someone carrying a surfboard are all variations of the concept “pedestrian.” We do not need a formal rule to recognize them.

AI, however, requires explicit descriptions of these concepts. If a product manager writes a requirement stating simply that “the system must avoid pedestrians,” they have written an untestable specification. It is impossible to know what variations the model will recognize without a structured reference point. The absence of a formal benchmark makes evaluating the model highly abstract and heavily dependent on individual subjective judgment.

This issue compounds when moving across different domains. The study evaluated perception across automotive and aviation contexts. While “pedestrians” presented one set of challenges (wheelchairs, scooters, costumes), “aircraft” introduced entirely different semantic variations: unmanned vehicles, historic aircraft, general aviation, and military planes. A generic requirement for an AI to “detect aircraft” is meaningless if the system only recognizes commercial airliners and completely misses a drone.

The Fallacy of Treating Data as the Specification

When faced with edge cases and unexpected model behavior, the default response from many product teams is to feed the model more data. The assumption is that sheer volume will eventually cover all possible variations.

The research study points out that the size of a dataset is not a direct indicator of its quality. In fact, relying on excessively large, unstructured datasets often leads to overfitting. The model becomes too tailored to the specific training data and loses its ability to generalize to new, real-world scenarios. More data does not guarantee better results; it simply obscures the gaps in the system’s understanding.

Instead of mindlessly collecting more images or text, teams need a standardized reference point. You cannot build a reliable decision infrastructure if the underlying requirements for what the model should perceive are ambiguous. We have to define the boundaries of the concept before we can measure how well the model understands it.

Translating Domain Knowledge into Benchmarks

To bridge the gap between human intuition and machine perception, the Northern Illinois University researchers propose a structured approach: using formal ontologies to create a minimum set of requirements.

In their framework, called W-AIS, they built semantic webs capturing various specific topics within a domain. For the concept of a pedestrian, this meant defining explicit variants such as “physically-challenged pedestrians,” “children pedestrians,” or “careless pedestrians” (those jaywalking or looking at their phones). For aircraft, the ontology divided the concept into variations like “mobility issues,” “detection technologies,” or “historic aircraft.”

This represents a major departure from how product managers typically write requirements. You are not just writing user stories; you are constructing a formal knowledge base of concept variations. The researchers then translated these natural language specifications into a visual dataset. This translation is the bridge that allows teams to test whether the AI model actually perceives the required variations in the data.

Closing the Perception Gap with Explainable AI

The true test of a specification in an AI system is whether the model’s perception aligns with the human-defined requirement. The W-AIS framework uses explainable AI (XAI) to measure the semantic distance between the model’s generated understanding of an input and the formal human specification.

For example, when shown an image of a pedestrian in a wheelchair, an AI model might focus entirely on the wheels and misclassify the person as “riding a bike.” The XAI component highlights exactly where the model is placing its attention through heat maps and textual explanations. By identifying this mismatch between the highly-attentive features of the model and the detailed specifications of the concept variant, product teams can see exactly where the model’s perception diverges from the formal requirement.

This level of transparency changes how teams approach debugging. Instead of guessing why a model failed, the team can map the failure back to the specific node in their ontology that the model misunderstood.

Specification-Guided Retraining

This insight allows for highly targeted fine-tuning. Instead of blindly expanding the entire dataset, engineers can retrieve specific additional examples only for the under-perceived variants. The study demonstrated that this specification-guided retraining significantly improved the model’s ability to recognize complex variations, aligning its output much closer to human-generated captions.

For instance, the researchers measured the semantic similarity between the AI’s generated captions and the established ontology. By focusing their retraining only on the areas where the semantic distance was greatest, they improved the classification accuracy for both pedestrian and aircraft images by double-digit margins. They achieved this without diluting the model’s performance on the variations it already understood well.

The Cost and Reality of Implementation

Adopting this approach is not without its costs. Building a comprehensive semantic ontology requires significant upfront effort. It forces product and engineering teams to systematically map out their domain before they start training models. Translating those ontologies into visual datasets, whether through benchmark searches or web scraping, as the researchers did, adds another layer of complexity to the pipeline.

However, the cost of not doing this is far higher. Deploying probabilistic software without a structured way to measure its perception guarantees that you will discover its blind spots in production, often at the expense of user trust or safety. The upfront investment in a semantic ontology pays off by turning subjective model evaluation into an objective, measurable process.

The New Role of the Product Manager

For product leaders, this research marks a fundamental shift in how we approach product specification. We must stop pretending that AI features can be defined with the same static acceptance criteria we used for CRUD applications. That shift runs alongside a second one, where the spec becomes the artifact your coding agents build from.

The job of a product manager working on probabilistic software is no longer just writing the specification document. It is defining the boundaries of perception. It is building an ontology of the domain and continuously testing the model against that structured representation of the real world. If you cannot explicitly define the variations your system needs to understand, you have no way of knowing if it actually understands them.

References

• Barzamini, H., Nazaritiji, F., Brockmann, A., Ferdowsi, H., & Rahimi, M. (2025). An AI-driven Requirements Engineering Framework Tailored for Evaluating AI-Based Software. Proceedings of the 4th International Conference on AI Engineering, Software Engineering for AI (CAIN).