How to choose a Person ReID similarity threshold

“What number should we put in the configuration?” is usually the first threshold question—and the wrong place to begin. Picture two results with the same similarity score. One would merely add a candidate to an operator’s review strip. The other would silently place a photograph in a customer-facing gallery. The number is identical; the cost of being wrong is not.

There is no universal Person ReID similarity threshold. A defensible rule belongs to one model revision, preprocessing pipeline, target environment, gallery construction and decision. Your process is to define what the score controls, build representative positive and negative examples, choose the rule on a calibration set, and evaluate it once on untouched data. Keep a review band for ambiguity instead of pretending one number converts visual similarity into identity.

The practical Person ReID guide places this threshold inside the complete crop, embedding, retrieval and review workflow.

Before choosing a number, name the score

The documented PolyReID v1 contract defines L2-normalized appearance embeddings. For normalized vectors (x) and (y), cosine similarity is their inner product:

similarity(x, y) = x · y
||x - y||² = 2 - 2 × similarity(x, y)

The FAISS documentation on metrics and distances describes this relationship and the need to normalize vectors when using inner product as cosine similarity. A threshold written for cosine similarity cannot be pasted unchanged into code that compares squared L2 distance; the inequality direction and numerical scale differ.

Check four invariants before calibration:

1. query and gallery vectors use the same immutable model revision;
2. the preprocessing version is identical;
3. normalization and distance are explicit;
4. a larger score consistently means a closer candidate.

The guide to ReID embeddings and vector databases covers the metadata that should travel with each vector. Mixing revisions can shift scores for reasons unrelated to the people in the images.

What happens on either side of the line?

A score may control several different products:

• whether a candidate appears in a top-k review list;
• whether two crops are proposed as a possible match;
• whether a photo is added to a provisional group;
• whether the system returns “no candidate”;
• whether a result is accepted without review.

These decisions do not share one cost profile. In an assisted photo workflow, showing an extra candidate may create a small review cost, while silently attaching a photo to the wrong customer gallery may be much more consequential. Before you select a metric, write down the allowed action, required human control and correction path.

The Person ReID evaluation metrics guide separates ranking measures from threshold measures. A high Rank-1 result does not automatically imply that an acceptance threshold is safe.

Build calibration pairs without leakage

Create positive pairs from crops of the same evaluation identity and negative pairs from different identities. Both need to resemble deployment:

• positives should span cameras, viewpoints, lighting, motion and allowed outfit variation;
• negatives should include common cases and hard lookalikes, not only obviously different people;
• crop failures should appear in a separate quality slice and, if they can reach inference, in the end-to-end test;
• the distribution of cameras, events and gallery sizes should follow the intended workflow.

Split by identity or another deployment-appropriate grouping so that closely related samples do not leak across calibration and test. Keep the final test labels hidden from threshold selection. If the same burst of near-duplicate frames appears in both sets, the result can overstate generalization.

Do not assume that generating every possible negative pair creates an equally informative dataset. Those pairs are highly dependent and may be dominated by easy negatives. Preserve query-level and event-level structure so reported uncertainty reflects the actual unit of use.

Inspect distributions, then choose a metric

Plot positive and negative score distributions, but do not expect a clean gap. Similar outfits can create high-scoring negatives; blur, occlusion and wardrobe changes can create low-scoring positives. The overlap is the operating trade-off, not a defect that a prettier chart removes.

For each candidate threshold (t), compute:

precision(t) = TP(t) / (TP(t) + FP(t))
recall(t)    = TP(t) / (TP(t) + FN(t))

The official scikit-learn precision-recall curve documentation provides the definitions and returns the sequence of thresholds for binary labels and scores. Precision depends on the prevalence and candidate-generation process in the evaluated data. A value measured on a balanced pair set will not necessarily describe a gallery where non-matches vastly outnumber matches.

ROC or DET curves can help compare score separation, while precision-recall curves often communicate candidate quality more directly when relevant pairs are rare. Neither curve chooses the business rule. A simple cost expression can make the decision explicit:

cost(t) = C_false_match × FP(t)
        + C_missed_match × FN(t)
        + C_review × reviewed(t)

The coefficients are product decisions, not properties of the model. Document who selected them and why.

Give ambiguous cases somewhere to go

One threshold forces every case into yes or no. A two-boundary rule creates a visible middle state:

• below the lower boundary: do not surface or mark as unlikely;
• between boundaries: send to human review;
• above the upper boundary: show as a stronger candidate, with automatic action only if the use, evidence and governance justify it.

Imagine three result cards: one clearly below the lower boundary, one in the review band and one above the upper boundary. The middle card is not a system failure. It is an explicit admission that the available evidence does not justify an automatic yes or no.

The score should still be displayed as similarity, not as an identity probability, unless a separate and validated calibration model maps it to a probability for the target population. Even then, probability calibration can drift.

Pair the score with rank and context. A top candidate at a moderate score in a small, constrained event gallery may be useful for review. The same score in a much larger and more varied gallery can produce many plausible negatives. Store the gallery definition with the threshold version.

Treat input quality and domain as threshold dependencies

A threshold cannot repair a crop containing two people or mostly background. Route unacceptable inputs through a quality gate using the person-crop quality guide. Measure thresholds separately on meaningful quality slices so the review policy can reject poor inputs instead of absorbing their errors.

Camera replacement, venue lighting, a new detector or a different participant population can alter the score distributions. The Person ReID domain-shift playbook explains how to establish sentinel data and reevaluation triggers. Avoid immediately creating a threshold per camera: first check whether the variation indicates a pipeline defect or insufficient evaluation data.

Version the complete operating rule

A threshold record should include:

• model alias and immutable model revision;
• preprocessing revision and normalization;
• distance function and inequality direction;
• query/gallery filtering and index configuration;
• calibration dataset version and exclusions;
• lower and upper boundaries, if used;
• the action attached to each band;
• test precision, recall, false matches per query and review load;
• slice results and approval date;
• conditions that trigger rollback or recalibration.

The documented PolyReID API contract includes version fields for the embedding operation; the client still owns gallery construction, threshold calibration and downstream decisions. A useful pilot compares the new rule against the current human workflow without silently automating it.

Finally, monitor both outcomes and inputs. Score histograms can reveal change, but only reviewed or otherwise reliable labels can reveal whether relevance changed. Recalibrate after model or preprocessing updates, and reevaluate after material domain changes. Treat the threshold as a versioned component—not a constant hidden in source code forever.