Person crop quality for ReID: an input checklist

Place two crops side by side. The first is crisp and well exposed, but half of a dance partner fills the box. The second is slightly soft, yet it contains one clearly selected person from head to toe. Which is the better ReID input? Sharpness alone cannot answer.

A useful crop contains one intended person, enough available appearance, limited unrelated background and preprocessing compatible with the gallery. Rejecting an unusable box is often safer than asking a similarity threshold to compensate for it. Pixel size, body margin and blur cutoffs only become meaningful after you define, test and version them for your own source images and operating domain.

The embedding cannot rescue the wrong box

The embedding endpoint starts after person localization. It does not know whether the box came from a detector, a photographer’s selection or another pipeline. If the wrong person occupies the crop, the resulting vector can be internally valid while representing the wrong subject.

A useful crop policy aims for:

• exactly one primary person;
• the body regions that are genuinely available in the source image;
• minimal inclusion of adjacent people and repeated background;
• consistent orientation, color decoding and resize behavior;
• no geometry-distorting client transformation;
• traceable detector, cropper and preprocessing versions.

The practical Person ReID guide explains why detection, embedding, retrieval and review should remain separate stages. Check the public PolyReID API page for accepted formats and current request limits.

Read the pixels before trusting the label

Two or more people. Picture a foreground dancer whose box also contains a partner’s bright jacket. The embedding may combine cues from both people and the background. A detector box is not proof that only one person contributes pixels.

Severe truncation. A head-only, shoe-only or narrow torso crop contains a different evidence pattern from a full or mostly visible person. Imagine comparing a strip of sequined fabric with a head-to-toe gallery: the inputs do not expose equivalent regions. Partial crops may still be useful with a model and protocol designed for them, but they should not be silently mixed into a holistic workflow.

Occlusion. Furniture, another dancer, a raised arm or a competition number can hide useful appearance. Research such as Pose-Guided Feature Alignment for Occluded Person Re-Identification exists precisely because visible and occluded regions require special handling. Its results do not imply that every general embedding is occlusion-robust.

Motion blur and defocus. Texture and boundaries can disappear. A blur score can flag extreme cases, but it is not a universal ReID-quality measurement: patterned flooring may look sharp while the person is blurred.

Excess background. Venue branding, floor color or a recurring wall can become a shortcut. A loose crop may retrieve the same camera view rather than the same appearance.

Clipped edges and detection offset. Cutting through the body or centring on the wrong subject creates spatial misalignment. The Densely Semantically Aligned Person Re-Identification paper documents imperfect detection, viewpoint, clutter and occlusion as alignment challenges.

Make the same crop twice

Given the same source asset, box coordinates and pipeline revision, you should be able to reproduce the same crop. An auditable pipeline can follow these steps:

1. Decode once. Apply the intended image orientation and convert to a documented color representation.
2. Select the subject. Associate the requested detection or human selection with one person. Do not choose “the largest box” without checking that this matches the selection the interface asked for.
3. Apply a versioned margin. If context around a detector box is needed, use a policy evaluated on the target domain. Clamp the result to the image boundary.
4. Validate the crop. Check dimensions, decode errors, subject count, clipping indicators and obvious corruption.
5. Preserve geometry. Avoid stretching a tall crop into a different aspect ratio in client code. Let the documented model preprocessing own resize and padding, or reproduce that preprocessing exactly.
6. Record provenance. Store source asset ID, box coordinates, cropper revision and quality decision without retaining more personal data than the purpose requires.

Repeated JPEG encoding is also unnecessary. Crop from the best authorized source available and avoid chains of thumbnails when the original pipeline can provide a direct crop.

Give bad and ambiguous inputs different exits

Individual checks are useful signals:

• crop width, height and area;
• box area relative to the source;
• whether the box touches an image boundary;
• estimated number and overlap of person detections;
• blur or exposure proxies;
• fraction of padding introduced;
• detector confidence and detector revision.

None is sufficient alone. Detector confidence estimates the detector’s task, not whether the crop will retrieve the right neighbours. Combine deterministic rejection rules for clearly invalid input with a review state for ambiguous input. Keep the raw signals so rules can be reevaluated later.

Create a small labeled quality taxonomy such as usable, review, wrong-subject, multi-person, truncated, occluded and blurred. Sample crops from every camera and venue, not only API errors. Then connect quality labels to the Person ReID evaluation metrics: measure Rank-k, mAP, false candidates and review workload per quality slice.

See an atypical crop in its cluster context

In the explorer, a clean group forms an envelope around its projected points. Selecting one crop exposes the source thumbnail, its closest visible neighbours and two complementary distances. Center distance measures the straight-line gap in the 3D view. Mahalanobis distance also accounts for the group’s variance on each axis: the same movement looks more unusual when the group is narrow in that direction.

This is useful during quality review. A crop at the edge may contain a partner, an unusual camera angle, heavy occlusion or a framing change worth opening. But “inside the envelope” does not prove that a crop is correct, and “outside” does not prove that it is wrong. t-SNE compresses the full embedding, while the displayed Mahalanobis confidence is a projection-level diagnostic rather than an identity probability. Confirm the pixels and measure retrieval quality in the compatible 768-D space.

Keep query and gallery policies compatible

A carefully framed query compared with a gallery of loose, multi-person crops is still an inconsistent system. Apply the same orientation and model preprocessing to both sides, and document any intentional difference in crop generation.

Do not pre-filter the gallery using the final similarity score and then evaluate on that same filtered set. Keep the full test protocol fixed. When selecting the similarity threshold, include the crop-quality states that can reach production. Otherwise the measured threshold only describes ideal inputs.

For approximate vector search, preserve the model and preprocessing versions with each vector as described in ReID embeddings and vector databases. Regenerate incompatible gallery vectors rather than mixing old and new crop policies invisibly.

Walk one crowded frame through the gate

Suppose a wide frame contains two couples crossing and the interface asks for the competitor in the foreground.

1. Retain the explicit user or roster selection rather than automatically choosing the biggest detection.
2. Inspect overlap with the partner and nearby detections.
3. Crop the selected box with the tested margin, clamped at the frame.
4. Mark significant partner overlap as review; do not hide it behind a high detector confidence.
5. Generate the embedding only if the policy permits.
6. Return neighbours as candidates and retain human confirmation for the ambiguous group.

If this situation is common, it belongs in calibration and final testing. It is not an isolated edge case.

Monitor crop drift

Track quality-signal distributions by camera and cropper revision. A new detector, export preset, portrait orientation or thumbnail provider can change inputs without changing the ReID model. The domain-shift playbook describes how to pair these signals with labeled sentinel queries.

Investigate changes rather than immediately loosening the threshold. A lower score may reflect a faulty orientation step or a new multi-person crop pattern. Threshold changes can conceal the symptom while increasing false matches elsewhere.

Finally, minimise the crop and its retention to the stated purpose. Background removal is not automatically better for model quality, but avoiding unnecessary people and context can reduce both retrieval shortcuts and privacy exposure. Crop quality is therefore an engineering, evaluation and governance decision—not a cosmetic preprocessing tweak.