feat(reid): add Deep OC-SORT adaptive appearance fusion to BoT-SORT by AlexBodner · Pull Request #571 · roboflow/trackers

AlexBodner · 2026-08-19T14:45:38Z

Stacked on #511. Adds Deep OC-SORT's adaptive appearance weighting to BoT-SORT through reid_config=ReIDConfig(fusion="adaptive"), and the ReID benchmarks: every dataset, both fusion methods, generic and fine-tuned encoders, on the test splits. ReIDConfig defaults to fusion="botsort", preserving the existing matching behavior unless adaptive fusion is selected.

soccernet_snmot147_proximity_gates.mp4

Results

Test HOTA for BoT-SORT with its tuned motion parameters, without ReID and with the best ReID configuration:

Dataset	BoT-SORT	+ ReID	Encoder	Fusion
MOT17	63.8	64.6	`fastreid_mot17_sbs50`	`adaptive`
SportsMOT	74.1	77.8	`osnet_x1_0_sportsmot`	`botsort`
DanceTrack	57.8	60.8	`osnet_x1_0_dancetrack`	`botsort`
SoccerNet	85.0	88.4	`osnet_x1_0_soccernet`	`adaptive`

Both fusion methods were tuned with trackers tune, 20 trials each, on each dataset's tuning split (val, MOT17 val-half, SoccerNet train; the fine-tuned SoccerNet encoder only on the 12 train sequences it was not trained on), then scored once on test: Codabench for MOT17, SportsMOT and DanceTrack, locally for SoccerNet. The MOT17 row is the one exception: its tuning split preferred botsort by 0.1 HOTA, and the row shows the best test result instead, which results.md states.

Neither fusion wins everywhere: adaptive is best on MOT17 and SoccerNet, botsort on SportsMOT and DanceTrack. The osnet_x1_0_<dataset> encoders are OSNet x1.0 (2.2M parameters) fine-tuned on each train split; fastreid_mot17_sbs50 (25.4M) was trained on MOT17 by the BoT-SORT authors.

Adaptive fusion

Rule: fuse_adaptive_reid_association in trackers/core/reid/fusion.py scores G + (a_w + w_b) * A, where G is BoT-SORT's score-fused geometric similarity and w_b is the capped gap between a track's best and second-best appearance match. This is the difference form the Deep OC-SORT authors ship as compute_aw_new_metric. Only the weighting is ported, not Dynamic Appearance, the velocity term or OCR.
Parameters: reid_config.fusion ("botsort" or "adaptive"), reid_config.appearance_weight (0.75), reid_config.adaptive_weight_cap (0.5) and reid_config.appearance_floor (0.0, off; below it a pair is scored on geometry alone). NaN, infinite and negative weights are rejected.
BoT-SORT integration: Negative cosine similarities count as zero. A pair outside the reid_config.proximity_threshold gate gets no appearance boost and keeps its geometry score.
Thresholds: the fused score runs up to 1 + weight + cap, so minimum_iou_threshold_first_assoc and minimum_iou_threshold_unconfirmed_assoc need retuning when switching, and reid_config.appearance_threshold is ignored by adaptive fusion.

Docs

ReID guide: choosing an encoder (names, parameter counts), a fusion method and the proximity threshold, and the Results section: fine-tuned and generic tabs of test results with their tuned configurations.
results.md: a BoT-SORT + ReID row in every table. Default rows use fastreid_mot17_sbs50 at library defaults; Tuned rows show the best test result per dataset. Rows are ordered C-BIoU, BoT-SORT, BoT-SORT + ReID, McByte here, in the README and on the landing page.
Removed: osnet_x1_0_msmt17_combineall, which lowered HOTA on all four datasets at defaults, and its SoccerNet figures.
Results tables: a small script merges each dataset's cells into one centred cell.

Before release

The fine-tuned encoder names resolve. The docs already load them by name (osnet_x1_0_mot17, osnet_x1_0_sportsmot, osnet_x1_0_dancetrack, osnet_x1_0_soccernet), so the weights must be uploaded, each name registered with its URL and checksum in the reid catalog, reid released, and the trackers[reid] minimum raised to that release. Both ReIDModel.from_pretrained("osnet_x1_0_sportsmot") and --reid.model osnet_x1_0_sportsmot must load.

Tests

tests/core/test_botsort_reid.py: adaptive fusion units, including regressions ensuring gating and flooring do not inflate the adaptive bonus, a single candidate getting no bonus, the proximity gate reading plain IoU rather than the fused association score, a crossing test that fails if reid_config.appearance_threshold leaks into adaptive fusion, and end-to-end runs of both fusions. tests/core/test_reid_config.py covers validation, defaults and rejection of NaN, infinite and negative weights. Full non-integration suite with #511 merged in: 1972 passed, 5 skipped, 15 deselected.

🤖 Generated with Claude Code

tstanczyk95

@AlexBodner very nice work and a cool feature to be added!

As we discussed earlier online, could you please provide the complete results so that for each of the four datasets: MOT17, SportsMOT, SoccerNet-tracking, DanceTrack, we have two sets of the results:

BoT-SORT run with a re-ID not tuned on the dataset, e.g the one pre-trained on MSMT17
BoT-SORT run with a re-ID tuned on the considered dataset, e.g. for the performance evaluation on SportsMOT, re-ID tuned on SportsMOT.

No need to worry about hyperparameter tuning for now, let's just focus on re-ID model at the moment, with a constant set of reasonable hyperparameter values.

So that we have complete information and see how much the re-ID is helping and how worth it is to tune it on the dataset of interest :)

Let's also add reid_appearance_floor in the PR description so that we have it for future references if needed ;)

BoT-SORT fuses appearance by taking the minimum of the geometry and appearance costs, gated by two fixed thresholds. Deep OC-SORT instead adds a weighted appearance term whose weight grows when the best match stands clear of the runner-up, which helps most in crowded scenes where a fixed weight is either too timid or too aggressive. Adds fuse_adaptive_reid_association alongside the existing fusion and a reid_fusion selector on BoTSORTTracker, with reid_appearance_weight and reid_adaptive_weight_cap for its two knobs. Default stays "botsort", so behaviour is unchanged unless opted into. Implements equations (4)-(6) of the paper, matching compute_aw_new_metric in the authors' integrated_ocsort_embedding tracker (the variant behind their published results) rather than the ratio-based variant that other ports use. Named for what it does rather than where it came from: only the adaptive weighting is ported, onto a BoT-SORT base, without Dynamic Appearance, the velocity term or OCR/OOS. Two documented departures from the reference: negative cosine similarities are clamped to zero, and appearance is gated by reid_proximity_threshold. Deep OC-SORT drops implausible pairs after matching, but this integration thresholds the fused similarity directly, so the gate has to happen during fusion. The discriminativeness gap is measured across all candidates before that gate is applied, so a candidate excluded on geometry cannot read as similarity 0 and earn the bonus for being spatially alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Documents reid_fusion and its parameters, and compares both fusion rules against a geometry-only baseline on SoccerNet, DanceTrack and MOT17 on the published tuned BoT-SORT geometry. The geometry-only column is the one that decides whether to enable appearance at all, so it belongs in the table. Adds a section on reid_proximity_threshold. Its 0.5 default restricts appearance to pairs already overlapping at IoU >= 0.5, and 0.99 still requires IoU >= 0.01, which a target returning from outside the frame does not meet: its prediction has been extrapolated out of frame and overlaps the returning detection by exactly zero. Only 1.0 disables the gate. On SoccerNet test that is worth +1.23 HOTA and a third fewer ID switches on the published tuned geometry, and it is what lets a player who walked out of frame come back with the same id. The reference Deep OC-SORT recovers that case with a geometry-only pass against the last observed box; BoT-SORT has no such stage, so appearance at an open gate is the only route. The same change hurts DanceTrack, whose targets stay in frame, so this is documented as domain guidance rather than a new default. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The adaptive fusion is additive, so any positive cosine similarity lifts a pair above the association threshold once geometry contributes nothing. With the proximity gate disabled that lets a lost track capture an unrelated detection on a weak appearance match rather than letting it start a new track. `reid_appearance_floor` zeroes the appearance term below a given cosine similarity, leaving those pairs scored on geometry alone. Defaults to 0.0, which is the Deep OC-SORT reference behaviour, so nothing changes unless the floor is set. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…e result tables - results.md: a BoT-SORT + ReID row in each Tuned tab with the best configuration found per dataset, its full configuration in the YAML block, and the encoder named above it. A general "When to Use" entry. C-BIoU's SoccerNet claim narrowed to motion-only trackers. - reid.md: one results table per fusion rule with encoder, parameters and ID switches; the generic-encoder DanceTrack pair and the earlier fastreid / MSMT17 runs kept under "Other encoders". The SportsMOT gate caveat replaced by the measured sweep: botsort at 0.99 and 1.0 loses to its own default, so adaptive's lead there is not a gate artifact. DanceTrack's adaptive row is marked pending a run with the fine-tuned encoder rather than borrowing the generic number. - botsort.md: point at the guide for ReID numbers. - README: BoT-SORT accepts an appearance branch through reid_model. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…e SoccerNet generic-encoder table - results.md: BoT-SORT + ReID at library defaults (catalog-default osnet_x1_0_msmt17_combineall, botsort fusion, theta 0.25, gate 0.5) in the DanceTrack and SoccerNet Default tabs: 56.0 / 56.1 / 91.8 and 82.9 / 77.7 / 96.5. MOT17 and SportsMOT follow once Codabench scores them. Intro states the encoder rule for the Default and Tuned rows. - reid.md: the SoccerNet generic-encoder table claimed theta=0.2 gives 84.6 (+0.1). Re-measured on this branch, theta 0.1, 0.2 and 0.25 all give 82.9 / 77.7 / 96.5 against 84.5 for geometry alone; the 84.6 row has no reproduction and is replaced by the measured sweep. The two sentences built on it are rewritten: no threshold makes the generic encoder useful on SoccerNet, the fine-tune is where the gain comes from. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…usion, not by theta Measured on this branch with osnet_x1_0_msmt17_combineall, default geometry, SoccerNet test: botsort fusion loses ~1.6 HOTA at every theta from 0.1 up; adaptive fusion at its defaults is neutral (84.45); adaptive at reid_proximity_threshold=0.99 gains +1.1 (85.68 HOTA, 80.08 IDF1, 2451 ID switches against 2939 for geometry alone). The open gate does not travel: adaptive at 1.0 with the floor loses 1.3, botsort at 1.0 collapses to 65.9 with 24,835 switches, as on SportsMOT. Rewrites the two sentences that said the encoder is the only lever, and adds the two adaptive rows to the SoccerNet generic-encoder table. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…d SportsMOT Scored on the Codabench test splits (submissions 917154 and 917155) with BoT-SORT at library defaults and the catalog-default osnet_x1_0_msmt17_combineall encoder: MOT17 63.4/78.7/79.1 and SportsMOT 73.5/73.1/96.9 HOTA/IDF1/MOTA. McByte still leads HOTA in both Default tables, so the default-parameter claim on the landing page is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The Results section opens on the fine-tuned tab, where ReID helps on every dataset, and shows the generic encoders second. Footnotes are renumbered to follow the new order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

It lowered HOTA on all four benchmarks and is no longer the default, so the guide stops showing it: the SoccerNet distance example and its frame-gap box go, with both figures, and so does its row in the encoder table. The advice that example carried, that no threshold helps when the two distance groups overlap, stays as a sentence after the MOT17 example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

With one published encoder left, the encoder table only repeated the two bullets above it. The bullets now carry what the table said: what fastreid_mot17_sbs50 is and how to load it, and that the fine-tuned results use OSNet x1.0 whose weights are not published yet. The generic bullet no longer says appearance helps little on unfamiliar footage, which the SportsMOT rows contradict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The README, landing page and results.md intros now name the trackers in the same order as the tables below them: C-BIoU before BoT-SORT. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

fastreid_mot17_sbs50 has 25.4M parameters and the fine-tuned OSNet x1.0 has 2.2M, counted on the inference networks without the training classifier. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Markdown tables cannot span rows, so a small script merges each dataset's name into one cell spanning its rows, vertically centred and without the blank cells below it, and marks where each group starts for the heavier rule. It only acts on tables whose first column has blank cells, and the Dataset column is centred in the markdown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The fine-tuned OSNet x1.0 encoders are referred to by the names they will ship under, osnet_x1_0_mot17, osnet_x1_0_sportsmot, osnet_x1_0_dancetrack and osnet_x1_0_soccernet: in the guide's encoder section, with how to load one in Python and on the command line, in the fine-tuned configuration blocks, and in the results.md Tuned notes and configurations. These names do not resolve yet: the weights still need uploading and the names need registering with their URLs before this is released. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot

Copilot review overview

🟡 Changes recommended

Documented checkpoints remain unavailable, and benchmark documentation plus one regression test need correction.

Review effort: Balanced
Findings: 2 · 2

Open (4)

Documented checkpoints are unavailable and dependency is too old · New
Test does not validate flooring regression · New
Update stale BENCH-XREF mappings for new benchmark rows · New
Correct MOT17 baseline from 63.9 to 63.8 · New

What changed in this PR

Adds opt-in Deep OC-SORT adaptive ReID fusion to BoT-SORT and documents benchmark results.

Changes:

Adds adaptive appearance fusion, configuration, validation, and tests.
Documents encoder selection, tuning guidance, and benchmark results.
Adds grouped benchmark-table rendering.

File	Description
`src/trackers/core/reid/fusion.py`	Implements adaptive fusion.
`src/trackers/core/botsort/tracker.py`	Exposes and applies fusion options.
`tests/core/test_botsort_reid.py`	Tests fusion and tracker integration.
`docs/guides/reid.md`	Expands ReID guidance and results.
`docs/evaluations/results.md`	Adds ReID benchmark rows.
`docs/trackers/botsort.md`	Links expanded ReID results.
`docs/index.md`	Adds ReID to the comparison table.
`README.md`	Adds ReID results and guidance.
`docs/javascripts/grouped_tables.js`	Groups repeated table cells.
`docs/overrides/stylesheets/style.css`	Styles grouped tables.
`mkdocs.yml`	Loads the grouping script.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Both candidates sat below the 0.95 floor, so flooring before measuring the top-two margin produced the same zero gap and the test passed either way. One candidate now clears the floor (0.9) and one does not (0.8): the raw gap is 0.1, the floored-first gap would be 0.9 capped to 0.5, so the fused score is 1.32 against 1.5. Checked by moving the floor before the margin: the new test fails, the old one passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The generic tab used the ReID study's own geometry run (63.861, shown as 63.9) while the fine-tuned tab and results.md use the published tuned BoT-SORT result, 63.8. The guide's BoT-SORT rows copy results.md, so both tabs now show 63.8. No bold changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The BENCH-XREF comments and docs/AGENTS.md did not know about the BoT-SORT + ReID rows: the Default rows copied into README and the landing page, the Tuned tables whose BoT-SORT rows the ReID guide copies and whose ReID rows come from it, and the ReID footnote. They also still said the landing page had no C-BIoU row, which predates this branch. Comments only; nothing renders differently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

tstanczyk95

Mostly fine from my side. Some docs changes and suggestions. Otherwise LGTM.

…ReID note Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…age' into feat/core/deep-ocsort-adaptive-fusion # Conflicts: # README.md # docs/evaluations/results.md # src/trackers/core/botsort/tracker.py

AlexBodner requested a review from SkalskiP as a code owner August 19, 2026 14:45

AlexBodner force-pushed the feat/core/deep-ocsort-adaptive-fusion branch 8 times, most recently from 17a80a0 to d7387c8 Compare August 19, 2026 18:53

Borda added the enhancement New feature or request label Aug 23, 2026

tstanczyk95 self-requested a review September 9, 2026 18:28