Human review for AI photo matching: an event workflow

You are reviewing two crops from the same event. Both show a blue costume under stage light; the top candidate carries an illustrative similarity score of 0.82, but the face is turned away and the bib is folded. Would you press Accept?

The screen should not force an answer. It should reveal enough evidence to confirm, reject, move or defer the suggestion. This is the narrow role AI photo matching can support: a retrieval system prepares likely groups and concentrates attention on uncertainty instead of asking you to label every frame from scratch. You still need the authority and context to reject it.

Ballroom and action photography make the distinction visible. A turned head may defeat face search, a folded number may defeat OCR, and two competitors may share a costume colour and body shape. Person ReID can rank full-body appearance candidates, but a high rank is not proof that the photographs show the same person. The operational goal is a reviewable suggestion pipeline, not autonomous identity assignment.

Define the output before the threshold

It is tempting to answer the opening question with a threshold: accept everything above one number. Start instead with the action that follows each result.

Result state	What the system knows	Permitted next step
Candidate	One or more crops are visually close	Show them to a reviewer
Ambiguous	Signals conflict or several candidates are plausible	Defer, expand context or request a second review
Confirmed group	An authorised reviewer accepted the assignment	Make it eligible for the approved downstream workflow
Rejected	The proposed relationship is wrong	Return or move the photo without teaching an unreviewed rule
Unresolved	Available evidence is insufficient	Leave unassigned

“Unresolved” is a legitimate outcome. It tells you that the available evidence did not support the next action. Forcing every photograph into a group creates confident-looking errors and hides where the workflow needs another signal.

Give the reviewer the right evidence

Return to the two blue costumes. A contact sheet with tiny thumbnails is not enough to decide between them. For each review, show:

• the full source frame as well as the person crop;
• a sharp representative image for the proposed group;
• several neighbouring candidates, not only the top result;
• event, session, camera and capture-time context;
• roster, heat or competitor-number context when lawfully available;
• the model and preprocessing revision;
• the review state and prior corrections;
• an explicit way to enlarge, reject and defer.

Keep the similarity score available to trained operators, but do not style it as a probability. A score of 0.82, for example, does not mean an 82% chance that two images contain the same dancer. Its meaning depends on the model, crop process and evaluation distribution.

The Person ReID guide explains the difference between ranked retrieval and identity determination; the general comparison separates it from face recognition and tracking.

Route work with review bands

After evaluation on a representative event set, you may route work into bands rather than pretending one pass/fail threshold answers every case:

Band	Example routing policy
Strong, consistent suggestion	Queue for a fast confirmation and include some in quality-control sampling
Plausible suggestion	Require normal review with source context
Conflicting suggestions	Require expanded context or a second reviewer
Weak suggestion	Leave unassigned unless another signal supports it

The actual boundaries must come from local validation. Do not copy values from a paper, a vendor demo or a different competition. Camera position, crop quality, lighting, clothing, venue and population all shift the score distribution.

Sampling apparently easy matches is essential. If reviewers only see low-confidence cases, a systematic mistake in the “strong” band can continue unnoticed.

Use a five-action review loop

When the evidence is on screen, the interface should let the reviewer act without leaving the queue. Five actions cover most corrections:

1. Accept the photo into the proposed group.
2. Reject the proposed relationship and return the image to the unassigned pool.
3. Move the photo to an existing, better-supported group.
4. Split a mixed group into separate dancers or couples.
5. Merge duplicate groups only after comparing representative images and context.

Add defer when the operator needs a second opinion or later roster information. Keyboard shortcuts can improve pace, but destructive actions should remain reversible.

Record who performed the action, when, on which source objects and under which review-policy version. Do not store unnecessary image content or personal details in the audit log.

Handle ballroom-specific ambiguity

Name the reasons a case is difficult. Otherwise the same blue-costume problem returns as an unexplained rejection next week:

• face turned away or fully occluded;
• severe motion blur;
• person crop contains part of a partner;
• near-identical costumes;
• costume change within the event;
• competitor number hidden or unreadable;
• reflection, screen or poster mistaken for a person;
• group spans incompatible sessions;
• crop is head-only or too partial for the intended model;
• roster or partnership information changed.

These reasons guide remediation. A crop problem should return to the crop stage. A costume change may require schedule or roster context rather than a lower similarity threshold. Similar outfits may justify a second reviewer.

PolyReID’s ballroom evaluation case study documents these domain limits without turning internal evaluation into a universal accuracy claim.

Resolve reviewer disagreement

Suppose two reviewers examine the same mixed group and choose different dancers. That disagreement is information, not an inconvenience to hide. For a quality-control sample and all high-impact cases, use independent review before showing one operator the other’s answer. Then compare:

• which cases produce disagreement;
• whether the interface omitted useful context;
• whether one instruction is interpreted differently;
• whether the “correct” group is itself uncertain;
• whether a policy change would move too many cases into forced resolution.

Adjudication should update written guidance, not just fix one group. When the ground truth cannot be established, label the case unresolved rather than manufacturing certainty.

Measure work, not a vanity accuracy

Once the queue is running, measure the work it creates:

• photos reviewed per event and per workflow band;
• time spent on normal, ambiguous and correction queues;
• accept, reject, defer, move, merge and split rates;
• wrong-group reports after publication;
• sampled errors in the strong-suggestion band;
• disagreement and adjudication reasons;
• changes after a new camera, venue or crop process.

Do not collapse these into a single unsupported “AI accuracy” percentage. Retrieval quality, reviewer quality and final gallery quality are different layers.

The downstream experience also matters. A well-structured dance competition gallery should expose only the appropriate reviewed state and provide a correction route.

Keep privacy boundaries in the queue

Reviewers should access only the events and fields required for their role. Use project-scoped collections, short sessions and auditable permissions. Establish retention for source images, crops, embeddings, review logs and exports separately.

Human review does not make an otherwise unlawful processing operation lawful, and encryption does not make a person-appearance vector anonymous. The GDPR guide for event-photo ReID provides a structured issue checklist; obtain qualified advice for the actual deployment.

Before the first reviewer logs in

• The model returns candidates, not a final identity statement.
• Review states and permitted downstream actions are documented.
• Source frame, crop, candidate group and event context are visible.
• Accept, reject, defer, move, split and merge are supported.
• Every correction is reversible and attributable.
• Strong suggestions are sampled for quality control.
• Ambiguous cases can receive independent second review.
• Thresholds and review policies are versioned.
• Drift checks follow material camera or workflow changes.
• Access, retention and deletion cover all review artefacts.

Where the API boundary sits

The PolyReID API accepts one authorised person crop and returns a versioned appearance embedding. It does not detect a crowd, assign a name, set a universal threshold or perform the human decision. Those layers belong to the deployer’s system.

Developers can inspect the current contract on the Person ReID API page. Before building the queue, also read the event-specific comparison of face search and Person ReID so reviewers receive evidence suited to the actual photography problem.

A similarity score such as the illustrative 0.82 must remain one piece of evidence in a decision a reviewer can still question. That is the boundary the workflow must preserve.