Your AI CMO

Practical field guide

Designing an Explainable B2B Intent Scoring Rubric

Build a transparent way to prioritize signals while showing evidence, uncertainty, and the action each score should trigger.

KREV AI Editorial Team6 min read
What you will take away

Start with interpretable criteria, then calibrate with outcomes

Try the interactive lab
Framework visual

The signal review funnel

Scores help route review. They should never hide the original evidence or act as a verified precision claim.
01

Eligibility

Remove irrelevant, duplicate, and prohibited records.

02

Evidence

Read the source and note what the buyer actually said.

03

Priority

Rate fit, timing, specificity, and confidence.

04

Action

Match the response to the signal and channel.

Decision gate: A high score with weak evidence should be sent to review, not directly to outreach.

Decide what the score is for

A scoring system is useful only when it changes a decision: which conversations deserve research, which accounts need a follow-up, or which records should be discarded. Define that decision before assigning points. Keep eligibility rules separate from priority. A duplicate, a prohibited contact, or a conversation unrelated to the product should not become actionable simply because it matches a keyword.

Start with four dimensions that a reviewer can explain: fit, specificity of need, timing, and evidence confidence. Give each a short definition and positive and negative examples. A buyer who states a concrete migration problem may be more useful than someone at a perfect-fit company who shared a general industry article. Unknown information should remain unknown rather than default to a favorable score.

DimensionStrong evidenceWeak evidence
FitWorkflow and organization match the offerOnly a broad industry match
NeedSpecific pain or required capabilityCategory name with no problem
TimingActive project or stated deadlineNo sign of an evaluation
ConfidenceDirect source and clear contextInferred intent from a fragment

Choose weights from outcomes, not intuition alone

An initial rubric can use ordinal labels rather than a 100-point formula. Once reviewers have labeled a sample and outcomes have matured, compare scores with qualified opportunities and false positives. Adjust criteria that fail to discriminate. Keep a holdout sample for evaluation, and recheck results by source and segment; a model that works for one community may perform poorly elsewhere.

Record why a record received its priority. A short evidence note should include the source, date, relevant quote, assumptions, and reviewer. This allows sales to choose an appropriate response and makes errors correctable. If automated classification is used, sample its decisions regularly and let reviewers override them with a reason.

Connect thresholds to responsible actions

A high-priority record may trigger human review, not automatic email. Medium priority might become an account research task. Low confidence might need more evidence or no action. Define suppression and privacy rules alongside the threshold. Monitor precision among reviewed records, reviewer agreement, opportunity quality, and negative feedback from the channel. These measures are more meaningful than an unverified “99% precision” headline.

Calibrate the rubric with labeled examples

Gather a sample of past signals that reached an actual review decision. Include successful and unsuccessful examples from each source rather than choosing only obvious wins. Two reviewers should read the full source context and label fit, stated need, timing, and confidence independently. Where they disagree, refine the definitions and keep the disputed examples. This process exposes whether “strong intent” is a shared concept or merely a persuasive phrase.

Only then consider numerical weights. Start with a transparent baseline, such as equal weights, and compare its routing decisions with outcomes. A high score that mainly reflects company fit may route too many curious but inactive accounts. A timing weight that depends on a single phrase may miss slower enterprise evaluations. Adjust one rule at a time and keep a holdout sample so improvements are not judged on the same examples used to tune the rubric.

Define actions for uncertainty

Priority is not the same as permission. A high score should first tell a person which evidence to review. A medium score might prompt account research or a helpful public response. An unclear source should remain in research, even if other dimensions look promising. Add a suppression rule for prohibited channels, duplicate records, and people who have declined contact.

Revisit the rubric every reporting cycle. Compare reviewer agreement, the fraction of routed records that become qualified, missed opportunities found later, and negative feedback. Slice results by channel because the meaning of a signal differs between a technical forum and a professional network. Report the sample size and labeling method with any performance figure. An explainable system earns trust by making mistakes visible and correctable, not by claiming near-perfect precision.

Questions to settle before a score routes a person

What does a score of 80 actually mean?

Nothing by itself. It means something only when the dimensions, weights, source quality, and routing action are defined. In the interactive lab, 80 is arithmetic from four equal-weight sliders. It is deliberately not a probability of purchase or a tested precision measure. A real team should compare scores with independently labeled outcomes and inspect false positives by channel before setting operational thresholds.

Should one weak dimension veto the total?

Sometimes. Evidence confidence is a good example: an apparently urgent statement quoted out of context should trigger verification, not outreach. Eligibility and channel rules can also be hard gates. Define these overrides explicitly so a high aggregate cannot conceal a prohibited action or an unknown identity. Reviewers should see the original source and the reason for every override.

Copy this scorecard specification

Write decision being routed, eligible sources, dimensions and definitions, example evidence for each level, weights, unknown-value handling, hard exclusions, human review step, and feedback fields. Save the first labeled sample before changing weights. On each calibration cycle, report the sample size, reviewer agreement, false positives, qualified outcomes, and what changed in the rubric. Keep the score as a tool for allocating attention rather than a substitute for judgment.

Interactive lab

Try an explainable intent rubric

Adjust four dimensions for a sample conversation. Equal weights are illustrative; calibrate any real rubric against labeled outcomes before using it to route work.

2 / 5

Does this team plausibly need the solution?

No evidenceStrong evidence
2 / 5

Is there a stated problem or requirement?

No evidenceStrong evidence
2 / 5

Is evaluation active rather than hypothetical?

No evidenceStrong evidence
2 / 5

Is the source direct, recent, and in context?

No evidenceStrong evidence

Illustrative total

40/ 100

Verify the source first

First inspect: account and workflow fit. A human should read the original conversation before deciding what, if anything, to do.

Illustrative example

A high score built from missing fields

A lead-scoring rule gives a public post a high priority because the company matches the target segment. The post says little about the person's role or purchase timing, but unknown fields were silently treated as positive.

How the team responds

The sales-operations owner separates fit from need and confidence. The record moves to human research, and the reviewer can see which dimensions are unknown. After several weeks, the team compares similar records with qualified outcomes and revises the routing threshold.

What this illustrates

Explainability changes behavior. A score should show why a person needs review, not camouflage the absence of evidence.

Put it to work

A practical exercise

Label thirty recent signals using fit, need, timing, and confidence. Have a second reviewer score ten without seeing the first labels.

Make

A rubric with disagreements, examples, routing actions, and an evidence field for each record.

Measure

Compare reviewer agreement, false positives, and qualified outcomes by signal source.

Watch for

Do not publish a precision percentage without a defined sample, ground truth, and evaluation method.

Keep exploring

All guides