Person ReID vs face recognition vs tracking: what is the difference?

Imagine three requests from the same event team:

• “Find the other gallery photos that look like this authorised person crop.”
• “Keep a box attached to this dancer throughout this video clip.”
• “Compare this visible face with the participant’s enrolled portrait.”

The first request points to Person Re-Identification (Person ReID). The second points to tracking. The third points to face recognition. They can all involve images of people, neural-network embeddings and similarity scores, but they solve different problems and produce different evidence.

Calling all three “identity AI” hides the distinction that matters most: what observation is being compared, over what span, and what conclusion is the system allowed to make? A Person ReID result is not a face match. A track ID is not a person’s name. And none of these outputs, by itself, supplies consent, a lawful basis or proof of identity.

The short answer

Capability	Main question	Typical input	Typical output	Scope of the result
Person ReID	“Where else does a person with this appearance occur?”	One or more full-body person crops plus a gallery of crops	Ranked appearance-similarity candidates	Across separate images or cameras
Face recognition	“Does this face correspond to an enrolled face?”	A usable face image plus one reference or an enrolled gallery	Verification score or ranked identity candidates	Across face images
Tracking	“Which detection in the next frame continues this trajectory?”	Ordered video frames, detections and often motion or appearance cues	A sequence of boxes with a temporary track ID	Inside a video or camera sequence

Choose Person ReID when the useful evidence is a person’s overall appearance across separate images. Choose face recognition when facial identity comparison is genuinely required and a usable face is available. Choose tracking when the task is to maintain continuity through ordered video frames. If a roster, barcode, bib number, timestamp or manual folder already solves the problem, you may need none of them.

What Person ReID actually compares

Person ReID maps a crop containing a person into an appearance representation, commonly called an embedding. A retrieval system compares that query embedding with embeddings from a defined gallery and ranks the most similar candidates.

The signal can include clothing colour and texture, body outline and other learned visual patterns. That makes Person ReID useful when a face is turned away, too small or hidden. It also creates characteristic failure modes: two people may wear similar outfits, one person may change costume, a crop may include a partner, or lighting and camera style may shift between the query and gallery.

Academic Person ReID is commonly framed as retrieving a person of interest across non-overlapping cameras, as described in the survey Deep Learning for Person Re-identification: A Survey and Outlook. The same retrieval pattern can be bounded to an authorised event-photo collection, but the output should still be described precisely: it is a ranked list based on visual appearance, not a declaration of civil identity.

The current PolyReID API contract covers this narrow component. One crop containing one person becomes an appearance embedding. Detection in a full photograph, facial comparison, name assignment, video tracking, gallery indexing and final review are separate system responsibilities.

What face recognition actually compares

Face recognition processes facial characteristics rather than full-body appearance. Two common operating modes are:

• one-to-one verification, which compares a probe face with one claimed reference;
• one-to-many identification, which searches a probe face against an enrolled gallery and returns a candidate or candidate list.

The distinction is operationally important. Unlocking an account with an enrolled portrait is not the same search problem as scanning a large gallery for the closest face. NIST evaluates these modes separately; its Face Recognition Vendor Test report on identification describes one-to-many evaluation across controlled and less-constrained face collections.

Face recognition depends on a sufficiently useful face. A back-facing action photograph, a very small head, motion blur, strong profile or occlusion can remove the relevant evidence even when clothing remains easy to see. Conversely, a clean portrait may support facial comparison while saying little about which full-body crop belongs to the person in a crowded frame.

Face detection is also not face recognition. Detection answers “where is a face?” Recognition compares the detected face with a reference or gallery. A product that only finds faces does not establish who they belong to.

What tracking actually maintains

Tracking uses the order of video frames. A multi-object tracker detects or receives objects in successive frames, predicts where each object may move and associates new observations with existing tracks. Its output is usually a trajectory: bounding boxes linked by a temporary track ID.

Motion, box overlap and time may be enough for a simple scene. In crowded or occluded footage, trackers can also use appearance. ByteTrack, for example, formulates multi-object tracking around associating detection boxes across frames. Deep SORT combines motion with an appearance metric learned on Person ReID data to reduce identity switches through occlusion.

That overlap explains some of the confusion. Person ReID can be one component inside a tracker without becoming the same task as tracking. The tracker uses temporal continuity and data association to maintain a trajectory. A standalone Person ReID search can compare images with no adjacent frames and no motion model at all.

A track ID such as track_17 means “the observations currently linked into this trajectory.” It does not necessarily mean “the same known person everywhere,” and it should not silently become a persistent identity across venues, days or camera networks.

Why the three systems fail differently

The most useful comparison is not which technology is “best,” but what makes each result unreliable.

Situation	Person ReID	Face recognition	Tracking
Person turns away	Often still has body-appearance evidence	May lose the face signal	Can continue if motion or other cues remain stable
Near-identical outfits	Confusable candidates	May distinguish visible faces	Tracks may switch when people cross or overlap
Costume change	Same person may appear dissimilar	Face may remain comparable	Short continuous track may be unaffected
Mask, hair or partner hides face	Body crop may remain useful	Face comparison degrades or becomes impossible	May bridge a short occlusion, depending on the scene
Person leaves and returns much later	Can search a bounded image gallery	Can compare a newly visible face	Original track will often end; reconnection requires another signal
Poor crop or mixed-person crop	Embedding may represent the wrong visual evidence	Not applicable unless the face crop is also wrong	Detection and association errors can fragment or merge tracks
New camera, venue or lighting	Appearance ranking can shift	Facial image quality and domain affect results	Detection, motion and appearance components can all shift

These failures lead to different corrections. A ReID workflow needs crop inspection, candidate review and group correction. A face system needs face-quality checks and a review policy suited to verification or identification. A tracker needs tools for identity switches, fragmented tracks and incorrect merges.

A worked event-photography example

Suppose a ballroom photographer has 40,000 still images from one competition. A participant selects an authorised full-body photo and asks to find likely appearances elsewhere in that event.

This is naturally a Person ReID retrieval problem because:

1. the collection contains still images rather than an ordered video stream;
2. many action frames show profiles or backs rather than usable faces;
3. the desired output is a candidate set for the participant or operator to review;
4. the search can remain bounded to one event and its retention policy.

Tracking would not reconstruct reliable trajectories because there are no continuous adjacent frames. Face recognition would only cover the subset with a usable face and would introduce a different biometric comparison. For the narrower event choice, see face search versus Person ReID for event photos.

Now change the input to a ten-minute video of the competition floor. Tracking becomes relevant because motion and frame order are available. Person ReID might help reconnect a dancer after a short occlusion, but it is only one signal within the tracker.

Finally, change the request to account verification against a participant’s enrolled portrait. That is a face-verification brief—not Person ReID and not tracking—even if the wider product also contains event photographs.

Do not merge the scores into one unexplained “confidence”

A mature system may use more than one signal, but each should retain its meaning:

event and session boundary
  + roster, heat or capture-time context
  + bib or barcode evidence when present
  + face candidate when a usable face exists
  + appearance candidate from a valid person crop
  + track continuity when ordered video exists
  → labelled evidence for review

A cosine similarity between two appearance embeddings is not calibrated in the same way as a face-verification score or a tracking association cost. Averaging them into “92% confidence” can make the interface look simpler while making the result impossible to interpret.

Store the source, quality checks and decision rule for each signal. When signals disagree, defer or route the item to review. The reviewer should be able to see the query, the full source frame and the reason each candidate was proposed. The guide to human review for AI photo matching describes practical accept, reject, defer, move, merge and split actions.

Evaluate each task with the metric that matches it

There is no universal “people AI accuracy” number.

For Person ReID, retrieval measures such as rank-k/CMC and mean Average Precision describe where relevant gallery items appear. Open-set systems also need to measure rejection of queries with no valid gallery match. Read the Person ReID evaluation metrics guide before choosing a threshold.

For face recognition, verification and identification have different error trade-offs. False match and false non-match behaviour must be measured at an operating threshold and on data representative of the deployment.

For tracking, measures must reflect detection, localisation, missed tracks, fragmented trajectories and identity switches. A tracker can place boxes accurately while still attaching the wrong track ID after two people cross.

In all three cases, test the actual cameras, crops, lighting, population and workflow. Report reviewer workload and residual errors, not only a model metric. A benchmark result from another domain is evidence about that benchmark, not a promise for your collection.

Privacy and governance change with the purpose

Choosing body appearance instead of a face does not create a privacy exemption. Under the GDPR, an identifiable photograph may be personal data. The GDPR definition of biometric data refers to personal data resulting from specific technical processing of physical, physiological or behavioural characteristics that allow or confirm unique identification. Whether Article 9 applies therefore depends on the processing and purpose, not merely on whether the application stores a name.

Face recognition deserves especially careful necessity, proportionality and legal analysis. The EDPB’s final facial-recognition guidelines for law enforcement concern a specific sector, but they illustrate why intended purpose, data subjects, deployment context and safeguards cannot be separated from the technical design.

For any of the three capabilities, document:

• the specific purpose and why a less intrusive method is insufficient;
• who can submit a query and which collection can be searched;
• whether people are enrolled, informed and offered a reasonable alternative;
• how long source images, embeddings, face templates and track data remain;
• who reviews errors and how a person can challenge or correct a result;
• which uses and downstream decisions are prohibited;
• how access, export, deletion and audit events are controlled.

Do not label an appearance vector “anonymous” simply because it contains no name. A persistent representation used to single out or link a person can remain personal data and may, depending on its processing and purpose, fall within biometric-data rules. The GDPR guide for Person ReID and event photos provides a more focused checklist, but it is not a substitute for advice on the actual jurisdiction and deployment.

A five-question decision test

Before choosing a component, answer these questions in order:

1. Do you have still images or ordered video? Tracking requires temporal continuity. A folder of unrelated stills does not provide it.
2. Which visual evidence is actually present? Use full-body appearance only when person crops are meaningful; use facial comparison only when usable faces exist.
3. Do you need retrieval, continuity or identity comparison? Those outputs are not interchangeable.
4. Can a non-biometric signal solve the task? A roster, schedule, barcode, bib number or manual selection can be more accurate, easier to explain and less intrusive.
5. What happens after an error? If nobody can inspect, correct or appeal the result, the workflow is incomplete regardless of model quality.

The practical rule is simple: choose the minimum-capability system that answers the real question, keep every output inside its documented boundary, and treat similarity or association results as evidence to evaluate—not as automatic proof of who a person is.