openptv2 — Software Map: Tracking Algorithms, Implementations, Branches¶
1. Tracking plugins (src/openptv2/plugins/)¶
| Plugin | Engine | Mechanism | Strengths | Weaknesses |
|---|---|---|---|---|
unified (default) |
Auto-selects | Wraps trackcorr or priority_segment_3d based on density/ghost_rate; adds ILP, temporal smoothing, stitching, MHT, quality scoring | Adaptive, auditable, ghost-suppressing | Slower (0.8–1.8s); post-passes can hurt yield if over-tuned |
cython_epipolar_tracking (trackcorr) |
Epipolar (Cython) | 2D epipolar search → triangulate → 3D nearest-neighbor link | Fast (0.35s), high precision (99.5%), bidirectional+postprocess now default | Fragments in dense scenes (epipolar ambiguity rejects valid links) |
cython_3d_tracking (priority_segment_3d) |
3D priority cascade (Cython) | 4-level priority segment linking in 3D space; cold-start gating; gap bridging | Highest yield (1.0) and purity (1.0); full-length tracks; robust to 2D ambiguity | No epipolar verification; can link spatially-close ghosts |
four_be_tracking |
4-frame best estimate | 4BE cost with n+2 lookahead tie-breaker | Good for ambiguous candidates | Slower, niche |
hybrid_deltat_3d |
Coarse-to-fine Hungarian | Links every N-th frame (coarse) then refines intermediate frames | Slow flows at high FPS where Δt=1 is noise-dominated | Complex, slower |
two_phase_tracking |
2-phase (coarse+fine) | Coarse 3D prediction + linear assignment + 2D leaf re-ranking | Configurable cost weights; ambiguous 3D disambiguation | Slower, complex |
splitter_tracking |
4-view splitter hardware | Hardware-specific splitter rig | Niche hardware | Not general |
2. Anti-ghost modules (src/openptv2/)¶
| Module | Purpose | Status |
|---|---|---|
ghost_filter.py |
Early rcm-based ghost probability pre-filter | ✅ |
ghost_components.py |
Conflict graph decomposition (union-find) | ✅ |
ghost_ilp.py |
ILP uniqueness (MILP per component, Hungarian for bipartite) | ✅ |
layered_tracking.py |
Clean-and-Claim (tight core → 2D-target pruning → residual) | ✅ |
track_assisted.py |
Polynomial predictor, subset triangulation, shared targets | ✅ (upgraded) |
temporal_smoothing.py |
IRLS trajectory smoothing, residual + velocity-stats ghost filter, trajectory-level ILP | ✅ |
trajectory_stitch.py |
Merge fragments + split kinks (multi-frame repair) | ✅ |
mht_linker.py |
Sliding-window MHT (Hungarian over windows) | ✅ (auto-disabled pending tuning) |
epipolar_rerank.py |
Multi-D soft scoring (epipolar consistency) | ✅ |
calibration_aware.py |
rcm normalization, calibration-aware thresholds, multi-cycle consistency | ✅ |
frame_quality.py |
Skip/interpolate low-correspondence frames | ✅ |
3. Quality & reporting (src/openptv2/)¶
| Module | Purpose |
|---|---|
tracking_report.py |
Self-consistency metrics, per-trajectory quality scores (length, reciprocity, smoothness, gap), length_match_score, prio persistence |
tracking_decision.py |
Auditable TrackingDecision JSON record |
tracking_probe.py |
Multi-window probe + coordinate descent |
tracking_estimator.py |
Auto-parameter estimation (v_max, density, noise, ghost_rate, conditioning ratio) |
4. Branches¶
| Branch | Focus | Status |
|---|---|---|
feat/unified-tracker |
Unified tracker (Waves 0–5), backbone improvements, clean-and-claim | Merged into ilp branch |
feat/ilp-temporal-smoothing-impl |
ILP + temporal smoothing + stitching + MHT + benchmarks + quality criterion + skills | Current (active) |
feat/tracker-improvements |
Ghost elimination hardening, gap-bridge compaction, kink pruning | Cherry-picked into ilp branch |
main |
Stable release | Base |
5. Key parameters¶
| Parameter | Default | Effect |
|---|---|---|
dvxmin/dvxmax/... |
±10 mm | Velocity search box. Too small → fragments. Too large → ghosts. |
dacc |
5.5–10 | Acceleration tolerance for gap bridging. |
max_gap |
2 frames | Gap bridging length. Raise for occlusions. |
eps0 |
0.1–1.0 mm | Epipolar tolerance for correspondence. |
ilp_uniqueness |
auto | Spatial ghost suppression (ILP). Conservative by default. |
temporal_smoothing |
auto | Trajectory-level ghost filter (residual + velocity-stats). |
peeling |
auto | Clean-and-Claim (tight core → prune → residual). |
stitch |
auto | Trajectory stitching (merge/split). |
mht |
off (auto-disabled) | Sliding-window MHT (over-merges; needs tuning). |
probe |
auto | Multi-window parameter probe. |
corrective_passes |
2 | track_assisted extension passes. |
postprocess |
True | Gap relinking + cold start + reciprocity. |
6. Optimal combination strategy (accuracy + precision + length + minimal ghosts)¶
Recommended pipeline for a new dataset¶
1. Calibrate (openptv-multiplane-calibrate skill)
└─ Verify: rcm, reprojection RMS, camera coverage
2. Detect targets (targ_rec)
└─ One blob per particle, stable counts
3. Correspondences (criteria, eps0, X_lay/Z_lay)
└─ Maximize 4-camera share, minimize rcm
4. Choose engine (unified auto-selects):
├─ Sparse/clean (density < 100, ghost_rate < 0.2): trackcorr (epipolar)
│ └─ Fast (0.35s), high precision, good for clean flows
└─ Dense/ghosty (density ≥ 100 or ghost_rate > 0.2): priority_segment_3d (3D cascade)
└─ Higher yield, full-length tracks, robust to 2D ambiguity
5. Post-processing (auto-enabled):
├─ Gap bridging (max_gap=2, dacc)
├─ Cold-start seeding (first-frame recovery)
├─ Reciprocity enforcement (bidirectional)
└─ Corrective pass (track_assisted, 2 passes, polynomial predictor)
6. Ghost suppression (auto-enabled for dense):
├─ ILP uniqueness (trajectory-level, conservative)
├─ Temporal smoothing (residual + velocity-stats filter)
└─ Stitching (merge fragments + split kinks, smoothness-checked)
7. Quality scoring (compute_trajectory_quality_scores)
├─ length_score (saturates at GT length)
├─ reciprocity fraction
├─ smoothness (kink-free)
└─ gap penalty
→ write_prio for downstream weighting
8. Audit (TrackingDecision JSON)
└─ Resolved config, probe metrics, conditioning ratio, git version
Best tracker by scenario (from benchmarks)¶
| Scenario | Best tracker | Yield | Precision | MedLen | Purity | Time |
|---|---|---|---|---|---|---|
| Clean, sparse | trackcorr | 1.000 | 1.000 | 10.0 | 1.000 | 0.35s |
| Slow/noisy | priority_segment_3d | 0.517 | 0.517 | 10.0 | 1.000 | 0.42s |
| Dense/ghosty | priority_segment_3d | 1.000 | 0.997 | 10.0* | 1.000 | 0.43s |
| Speed-critical | trackcorr | 0.611 | 0.995 | 1.0 | 0.997 | 0.37s |
*priority_segment_3d produces full-length tracks in clean/slow scenes; fragments in dense (med_len 1.0) but still recovers all particles as full tracks in some cases.
The optimal combination (what unified should do)¶
- Engine: priority_segment_3d for dense/ghosty (better yield + length), trackcorr for clean/sparse (faster).
- Gap bridging: max_gap=2, dacc tuned to flow. This is the single biggest lever for track length.
- Cold-start seeding: recovers first-frame links using established velocity field.
- Reciprocity: enforces forward-backward agreement, killing ghost links.
- Corrective pass: track_assisted with polynomial predictor extends tracks.
- ILP (conservative): only when ghost_rate > 0.2. Removes target-conflicting ghosts.
- Temporal smoothing: residual-based ghost drop. Only when slow_flow or high jump_rate.
- Stitching (smoothness-checked): merges fragments across gaps, splits at kinks. Primary repair for fragmentation.
- Quality scoring: weight downstream analysis by track quality.
- MHT (disabled by default): available for experimentation; over-merges at current tuning.
What NOT to do¶
- Don't run ILP per-frame (breaks continuity). Trajectory-level only.
- Don't use wide dv (±12) — creates ghost links. Stick to ±3-6× measured displacement.
- Don't enable MHT without tuning (over-merges).
- Don't skip gap bridging (max_gap=2 is the biggest length lever).
Remaining work for optimal performance¶
- Multi-frame MHT tuning: better cost function (epipolar + velocity + residual) to merge without over-merging.
- Cython-level B6: push epipolar scores into track3d kernel (faster, same algorithm).
- B3 frame-quality: skip/interpolate low-correspondence frames.
- Calibration-aware thresholds: normalize rcm by distance from centre.
- Multi-cycle epipolar: run search with different camera orderings.
- openptv-cloud integration: H2 CI gate, J3 notebooks.