Deduplicate the calibration reference by peptidoform (v4.2.0)

[RobbinBouwmeester]

Problem

A user ran MS²Rescore on a PSM list where the same peptidoform appears in many spectra, each with its own observed retention time (matrix_fix_deeplc/psm_list.tsv: 201,593 PSMs, one run, 177,905 peptidoforms). The reference that auto-calibration selects from it holds 6,331 PSMs but only 2,623 peptidoforms, and the repeats of a peptidoform disagree on the observed RT by up to 236 min (median 0.89 min). The calibration therefore got several conflicting targets per prediction, and each peptidoform was weighed by how often it happened to be identified (up to 261 times).

Change

calibrate, finetune, predict_and_calibrate and finetune_and_predict now keep one PSM per peptidoform, the first observation, before fitting. New deeplc._reference_selection.deduplicate_psms(psm_list, ignore_charge=True).

• Charge states count as repeats: retention time does not depend on precursor charge.
• Opt out with deduplicate_reference=False, or --keep-duplicate-reference-psms on the CLI.
• When repeats disagree by more than a quarter of the observed RT range, DeepLC logs a warning: that means the reference mixes runs or contains low-confidence PSMs, which deduplication hides rather than fixes.
• Prediction is untouched — one prediction per input PSM, same order.
• No changes to MS²Rescore; it picks this up through the new default.

Effect on the reported case

Both fits evaluated on the 2,623 first observations (the only self-consistent target):

	MAE	median	90th pct	within 5 min
deduplicated (new default)	4.51 min	2.09 min	8.57 min	81.4 %
all reference PSMs (old)	9.69 min	5.98 min	21.85 min	44.9 %

Verification

• pytest tests: 126 passed, including 9 new tests in tests/test_deduplication.py (first observation kept, order preserved, idempotent, charge handling, modified peptidoforms distinct, warning on conflicting RTs, missing RTs, calibration behaviour with the flag on and off, parameter forwarded end to end).
• ruff check / ruff format --check clean on the changed files.
• Version bumped to 4.2.0 (default behaviour changes), CHANGELOG and usage docs updated.

🤖 Generated with Claude Code

[RalfG]

Good catch! Seems logical, but one question I had: Can this be set as default behavior without the need for keyword arguments and CLI options? That would keep the API relatively clean and removes the need for pipelining the option through multiple modules/layers of code.

If the dedup option would only need to be disabled in edge cases, we can rely on those users just using the reference selection API directly.