Risk Scoring for AI Decisions: A Practical Formula

Juan Piaggio · 2026-03-16 · 8 min read · ai · risk · product-development

The hardest question in any human-in-the-loop system is deceptively simple: which AI decisions actually need a human? Review everything and you drown your team. Review nothing and you gamble. The answer is a risk score, and it does not need to be complicated to be good.

A transparent formula beats an opaque model here almost every time. When you can read the math, you can explain a routing decision to your team, tune it when reality disagrees, and trust it because nothing is hidden. Let us build one you could ship.

The three factors that matter

Most AI decisions carry risk along a few intuitive axes. For agent-proposed actions on a ticket, three factors capture most of it:

Consequence — did the action do something hard to undo, like closing an issue?
Age — how long has this decision been sitting, unreviewed and potentially stale?
Confidence — how sure was the model about its own proposal?

A decision that closes an issue, sat for hours, and came from a low-confidence model is exactly the one you want a human to see. A fresh, high-confidence label suggestion is exactly the one you can let flow. The formula's job is to turn that intuition into a single comparable number.

The formula

Here is the exact scoring formula Meshworq uses to prioritize its approval queue. Each factor contributes independently, and they sum to a score between 0 and 1.

Risk Score = issueClosedFactor + ageFactor + confidenceFactor

issueClosedFactor = 0.3  if the agent closed the issue, else 0
ageFactor         = min(ageMinutes / 480, 1) × 0.2      // 480 min = 8 hours
confidenceFactor  = (1 - confidenceScore) × 0.5

Risk Level:
  HIGH    score ≥ 0.67
  MEDIUM  score ≥ 0.34
  LOW     score <  0.34

Read it top to bottom and the design choices reveal themselves.

issueClosedFactor is a flat 0.3 penalty for the one consequential, hard-to-reverse action. It is binary because "did it close the issue" is binary.
ageFactor ramps linearly with age, capped at 0.2 once a decision is eight hours old. Staleness matters, but only so much; an ancient decision is not infinitely riskier than an old one.
confidenceFactor is the heaviest weight, up to 0.5. Low model confidence is the strongest single signal that a human should look, so it dominates. Note it uses 1 - confidenceScore: the less sure the model, the more the risk.

The weights encode a philosophy: confidence is the loudest signal, consequence is a firm nudge, and age is a gentle tiebreaker. You can disagree with those weights, and that is the point. Because they are explicit, disagreement becomes a one-line change, not an argument with a black box.

A worked example

Take an agent proposal that is 600 minutes old with a model confidence of 0.1, and the agent did not close the issue.

issueClosedFactor = 0                            (not closed)
ageFactor         = min(600/480, 1) × 0.2 = 0.2  (capped at 8h)
confidenceFactor  = (1 - 0.1) × 0.5      = 0.45
Risk Score        = 0 + 0.2 + 0.45       = 0.65  → MEDIUM

A score of 0.65 lands in MEDIUM: worth a human's eyes, but not an emergency. Now change one thing. Suppose the agent did close the issue:

issueClosedFactor = 0.3
ageFactor         = 0.2
confidenceFactor  = 0.45
Risk Score        = 0.3 + 0.2 + 0.45 = 0.95  → HIGH

That single consequential action pushes the same decision from MEDIUM to HIGH. This is exactly the behavior you want. A low-confidence, stale proposal is concerning; a low-confidence, stale proposal that closed a customer's issue is a decision that should be near the top of someone's queue right now.

Why transparency beats a smarter model

You could train a model to predict "does this need review." It might even be marginally more accurate. But you would lose the three things that make this formula worth shipping:

Explainability — you can tell a skeptical stakeholder precisely why a decision was flagged HIGH.
Tunability — if your team finds age matters more in your domain, you change one coefficient, not a training pipeline.
Trust — reviewers believe a score they can recompute by hand.

A risk score you can explain on a whiteboard will be used. A risk score you cannot will be ignored.

There is a deeper reason too. This score routes human attention, and the humans need to trust the routing or they will second-guess it and review everything anyway, defeating the purpose. Transparency is not a nice-to-have here. It is what makes the whole system function.

Adapt it to your domain

The specific factors are Meshworq's, tuned for ticket approvals, but the shape transfers to almost any AI decision. Pick your consequence factor (the irreversible action in your world), your staleness factor (how long is too long for you), and your confidence factor (usually the heaviest). Cap the factors so no single axis dominates unfairly, and set thresholds that match how much review capacity you actually have. Then watch where reviewers overrule the score and let that feedback retune your weights.

Start simple, ship it, and adjust with evidence. A crude formula in production teaches you more in a week than a sophisticated one in a spec.

The takeaway

Risk scoring is how you convert "should a human look at this?" from a gut call into a number your system can act on at scale. The formula that works is not the cleverest one; it is the one your team can read, recompute, and tune. Meshworq's three-factor score, consequence plus age plus confidence, is a concrete, battle-tested starting point. Borrow its shape, swap in the factors that matter in your domain, keep the whole thing legible, and let the score do what it does best: point scarce human judgment at the decisions that actually deserve it.

← All Field Notes