GenAI Product Doctrine: Products of Judgement

Your output is someone else's input. That fact governs everything else.

A Product of Judgment feeds Products of Consequence. You don't sign the contract, move the money, or treat the patient. You inform the system that does. Data, signals, assessments, decisioning. The raw material for action that happens elsewhere.

Many products don't realize they're in this category. They think they're "just returning data." They're wrong.

The Line

One question separates Products of Judgment from Products of Action: are you returning something retrieved, or something derived?

If aggregation, matching, inference, synthesis, or evaluation happens between input and output, you're making a call. You're a Product of Judgment.

Package location from a tracking number is retrieval. Fraud score on a transaction is judgment. Current stock price is retrieval. Analyst rating is judgment. Temperature reading is retrieval. Five-day forecast is judgment. Customer record by ID is retrieval. "Is this the same person?" is judgment.

Many products mix both. A person record with a match confidence score. A company profile with a "still operating" flag that's actually an inference. Ground truth alongside derived assessments. When they're mixed, the product is Judgment. The doctrine applies. The facts are easy. The assessments are where you owe the work.

Get this wrong and your product will be blamed for downstream failures you don't control. Get it wrong long enough and you'll be replaced by a cheaper alternative. The doctrine exists because the stakes are real.

The Sneaky Cases

Some products have been making judgments so long they forgot they were judgments.

That "last verified" timestamp feels like a fact. A date something happened. Often it's an inference. You saw activity signals that suggested the entity was still around, so you updated the timestamp. That's not verification. That's a judgment call about what activity signals mean.

That "current employer" field looks like ground truth, but it's a match. You saw a name at a company in some data source and concluded it was the same person. Maybe it was, maybe it wasn't. That's judgment.

That "still in business" flag is binary, simple, feels like fact. But you're inferring it from a combination of signals: web presence, transaction activity, filing history. None of those individually say "still in business." You're synthesizing.

These inferences get cached and start feeling like facts. They propagate through your system, show up in API responses without any indication they're derived. Downstream consumers treat them as ground truth. When they're wrong, no one knows where the error originated.

If you've been in business long enough, you have fields like this. Audit them. The doctrine applies to all of them.

The Shift

Your consumer used to be a human. They looked at your output, applied their own judgment, and decided what to do. If your signal was ambiguous, they figured it out. If confidence was low, they dug deeper. Interpretation happened on their side.

Increasingly, your consumer is an agent. An LLM-based system consuming your signal and making decisions in an automated workflow. The human supervises the agent, not your output.

A human can work with a naked answer. An agent needs more. It needs to know not just what you're claiming, but how much to trust it, why, and what to do at different levels of certainty. That context has to come from you.

Richer signals help humans too. They always have. A human making a lending decision benefits from knowing why a credit score is what it is. A human investigating fraud benefits from structured reasoning about the alert. But humans could muddle through without it. They could apply their own judgment, dig deeper when something felt off, call a colleague.

Agents can't muddle.

The agentic era doesn't invent the need for rich, structured outputs. It makes that need mandatory. This isn't a tax for the new era. It's work you've been able to skip. The agents call the bluff.

The Failure Modes

Products of Judgment fail in predictable ways.

The naked answer. You return a result with no confidence signal. A match, a score, a classification. Just the output, nothing else. The downstream system has no way to know whether to trust it. They either trust everything, which means they act on your worst calls, or they trust nothing, which means they're doing their own verification and your product adds no value. Naked answers force your customers into bad choices.

The black box. You return a confidence score but no reasoning. "82% confident." Why? What's driving that number? What would make it higher or lower? The downstream system can't reason about your uncertainty because you haven't told them what's uncertain. They can set thresholds, but they can't make intelligent decisions about edge cases. When your score is wrong, they have no way to investigate. Black boxes train customers to ignore your confidence signals entirely.

The trust-me. You return reasoning, but it's vague or self-serving. "Based on our proprietary analysis." "According to our models." "High confidence based on multiple factors." This is worse than no reasoning at all. It has the shape of transparency without the substance. Customers learn that your explanations don't help them, so they stop reading. You've poisoned the well.

All three come from the same mistake: treating your output as the end of the line instead of an input to someone else's decision. Optimizing for looking right instead of being useful to act on.

The Doctrine

Know what you are. If any part of your output is derived, you're a Product of Judgment. The presence of facts in your payload doesn't exempt you. The derived parts are what matter to downstream decisions, and that's where you owe the work.

Confidence is a first-class output. Not metadata. Not a nice-to-have. Part of the core response contract. Every assessment comes with a signal about how much to trust it. If you're returning a match, return the match confidence. If you're returning a prediction, return the probability. If you're returning a classification, return the strength of the signal. The downstream system will use this. Give it to them.

Structure your uncertainty. A confidence score isn't enough. The downstream system needs to know why confidence is low. Missing data? Ambiguous input? Conflicting sources? Stale information? A weak match on one dimension but strong on others? Different causes of uncertainty call for different responses. "70% confident because we haven't seen this entity in six months" is different from "70% confident because two sources disagree." Structure your uncertainty so the consumer can reason about it.

Distinguish fact from inference. Your response mixes ground truth with derived assessments. Be clear about which is which. That "last verified" timestamp that's actually an inference from activity signals. That "current employer" that's actually a match against recent data. That risk flag that's actually a model output. Label them. The downstream system needs to know what it can take as given and what it should weigh.

Make your reasoning legible. When something goes wrong downstream, the Product of Consequence investigates. They trace back through the decision chain, and your reasoning has to be visible. Not just for humans debugging, but for systems doing that work. Why this confidence level? What inputs drove the assessment? What would have changed the outcome? The agentic era means more automated investigation of failures. Your outputs need to support that.

Liability stays where it is. You're a signal, not a decision. The Product of Consequence owns the outcome. This was true when humans were in the loop, and it's true when agents are. The contractual structure is well-established: you provide information, not advice; you make no guarantees about fitness for particular purposes; the customer assumes responsibility for how they use your signal. What changes is the richness of signal you need to provide, not who answers for the result.

The Anatomy of Good Confidence

The doctrine says what you owe. This section shows what good looks like.

A well-structured confidence signal has layers.

The score itself is the starting point. A number between 0 and 1, or a category like high/medium/low. Also the least useful part.

Dimension breakdowns matter more. If you're matching a person, confidence might be high on name, medium on address, low on phone. The overall score is a blend, but the dimensions tell the consumer where the weakness is. Maybe they don't care about phone. Now they know they can trust the match for their purposes, even though the headline number looks mediocre.

Reason codes make uncertainty actionable. Structured, enumerated explanations for what's affecting confidence: "STALE_DATA: primary source last updated 18 months ago." "CONFLICTING_SOURCES: two sources disagree on employer." "PARTIAL_INPUT: phone number not provided." Reason codes are machine-readable. Agents can act on them programmatically, and humans can scan them quickly.

Recency metadata answers a question that confidence scores hide: how old is this? When did you last see confirming evidence? A high-confidence match against two-year-old data means something different than a high-confidence match against last week's data.

Counterfactuals help where you can provide them. What would change this score? "Confidence would increase to 95% with phone verification." "Confidence would decrease if the input name were more common." This helps downstream systems decide whether to seek additional information or proceed with what they have.

The point isn't to dump everything you know into the response. Different consumers care about different dimensions. Give them the structure to pick what matters for their decision.

What Legible Reasoning Looks Like

The anatomy describes the structure. This section describes what breaks when you skip it.

Legibility enables investigation. When a Product of Consequence traces a failure back to your signal, they need answers: What inputs did you receive? What sources did you consult? What logic did you apply? What made you confident or uncertain? If any of these questions require a support ticket, your reasoning isn't legible enough.

Every response should be self-describing. An investigator looking at your output, without access to your internal systems, should understand what happened. Not the full implementation detail, but the decision-relevant facts.

For automated investigation, reasoning needs to be structured, not just present. Natural language explanations are fine for humans. Agents need reason codes, factor weights, input fingerprints, source timestamps. They need to programmatically compare your output on two similar inputs and understand why the confidence differed.

Can you explain a six-month-old response? If a customer asks why you returned what you returned on a query from last quarter, can you reconstruct it? If the answer is "we'd have to re-run it and see," your reasoning isn't legible. It's ephemeral. You can't learn from failures you can't reconstruct.

Legibility has costs. Storing explanations, structuring outputs, maintaining audit trails. Those costs are real. But the alternative is being a black box, and black boxes become commodities. If your customer can't tell why your signal is better than a competitor's, they'll buy on price.

The Opportunity

Products of Judgment have always existed. Credit bureaus, data brokers, research providers, analytics platforms. These businesses have been feeding decision-makers for decades.

What's new is the consumer. An agent can actually use structured uncertainty. It can programmatically adjust its behavior based on your confidence levels, factor your reasoning into its own, escalate to a human when your signal degrades below a threshold. The return on richer signals goes up dramatically when your consumer can act on them automatically.

Products that provide naked answers will become commodities. Products that provide decision-ready signals will become essential infrastructure for the agentic era. Confidence, structure, reasoning, clarity.

Your output is someone else's input. Make it good enough to act on.