Add TensorRT deployment and ResNet-50 detector support - #1
Open
haruto2002 wants to merge 2 commits into
Open
Conversation
added 2 commits
September 1, 2026 19:18
Reproducible toolchain for deploying the Sparse4D detector with TensorRT, run entirely inside the vss-rt-cv container: checkpoint-to-ONNX export with automatic spec repair, a trtexec engine-build wrapper (FP16 + MSDA plugin, variable camera count), engine inference with externally driven instance-bank recurrence, a hybrid runner pairing the TRT detector with the PyTorch identity layer, and a frame-aligned ground-truth check. Includes an FP16 recurrent-cache NaN-collapse defense at the feedback boundary, from-scratch setup and results docs, and a 4.6x speedup at 16 cameras (30.8 FPS, real time) for a 0.4-1.8 HOTA cost.
Ship an inference architecture config for the public NGC sparse4d_warehouse_v2.2_r50 checkpoint (configs/sparse4d_rn50_v2.2), add quality-weighted detection scoring to sqid infer (--quality-score: sigmoid(cls) * sigmoid(centerness), the TAO decoder convention), document the bfloat16 precision trade-off in REPRODUCE.md, and record the post-challenge findings (detector robustness, precision, emission scoring, ablation ladder, association headroom, runtime) in docs/EXPERIMENTS.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two additions from post-challenge work, one commit each:
trt/) — a reproducible toolchain that takes aSparse4D checkpoint to a real-time FP16 engine.
sparse4d_warehouse_v2.2_r50checkpoint, quality-weighted detectionscoring, and documentation of the findings behind both.
Neither changes existing behavior: the paper's published numbers remain
exactly reproducible with the documented settings.
TensorRT deployment
.pth → ONNX → engine → inference → GT check, run entirelyinside the
vss-rt-cvcontainer (host needs only docker + GPU driver).(FP16 + MSDA plugin, dynamic camera count), engine inference with
externally driven instance-bank recurrence, and a hybrid runner pairing
the TRT detector with the PyTorch identity layer.
(nan_to_num + clamp at the feedback boundary; reproducible with
--no-clamp, also reported to NVIDIA).149.3 ms in PyTorch — for a 0.4–1.8 HOTA cost. Verified on all three
validation scenes × 9,000 frames against the official 3D-HOTA protocol
(
trt/RESULTS.md).ResNet-50 support & quality scoring
src/sparseqid/configs/sparse4d_rn50_v2.2: architecture config + anchorsfor the public rn50 checkpoint (use via
--config-dir).sqid infer --quality-score: scores detections assigmoid(cls) × sigmoid(centerness)(the TAO decoder convention) insteadof the class score alone. Opt-in; default behavior is unchanged.
docs/EXPERIMENTS.mdrecords the post-challenge ablation study: the rn50detector is far more robust under domain shift (val GT-weighted HOTA
25.64 → 35.01), float32 inference is worth several HOTA points over
--ampbfloat16 (+4.6 on the hidden test set), and quality scoring adds~+1. The full ladder reaches 38.69 on val and 55.9 on the hidden test set
(submitted system: 38.01).
docs/REPRODUCE.mddocuments theprecision/reproducibility trade-off of
--amp.Testing
uv run pytest: 14 passed, 1 skipped (unchanged).trt/RESULTS.md).--quality-scoreoutput verified byte-identical to the externallyvalidated driver used in the experiments.