Skip to content

Add live Scenario progress - #2374

Draft
Roman Lutz (romanlutz) wants to merge 4 commits into
mainfrom
copilot/romanlutz-scenario-pr-03-live-upstream
Draft

Add live Scenario progress#2374
Roman Lutz (romanlutz) wants to merge 4 commits into
mainfrom
copilot/romanlutz-scenario-pr-03-live-upstream

Conversation

@romanlutz

@romanlutz Roman Lutz (romanlutz) commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds the live run dashboard with incremental polling, active/completed/pending attack groups, outcome accounting, elapsed time, ETA, cancellation, and result navigation. Progress cursors are isolated per run so route changes cannot reuse another run's opaque cursor.

What this layer adds

  • Incremental polling over lightweight run-bound progress cursors.
  • Overall progress plus completed, active, and pending atomic attack groups.
  • Group/objective outcome accounting, elapsed time, and ETA.
  • Cancellation and navigation to persisted results.
  • Cursor reset when the routed Scenario result changes.

Preview

Live progress with active and queued work

Stack

This is 3 of 7 in native GitHub Stack #2390 and depends on #2373.

Layer Scope Pull request
1 Progress foundation #2372
2 Catalog and launch #2373
3 Live progress (this PR) #2374
4 Run history #2375
5 FIFO scheduling #2376
6 Configuration sizing #2377
7 Result details #2378

See the full implementation, screenshots, walkthrough, validation, and operational notes.

Validation

  • Backend: 15,212 passed, 120 skipped.
  • Frontend: 68/68 suites, 1,286/1,286 tests; lint, TypeScript, and production build passed.
  • Mock E2E, retries disabled: 91 passed, 0 failed, 0 skipped.
  • Seeded E2E, retries disabled: 62 passed, 0 failed, 0 skipped.
  • All configured pre-commit hooks passed; distinguished-engineer review approved the final stack.

@romanlutz
Roman Lutz (romanlutz) force-pushed the copilot/romanlutz-scenario-pr-03-live-upstream branch from 58c5c56 to 4337415 Compare August 21, 2026 04:17
@romanlutz
Roman Lutz (romanlutz) force-pushed the copilot/romanlutz-scenario-pr-03-live-upstream branch from 4337415 to 80a40b7 Compare August 21, 2026 07:52
@romanlutz
Roman Lutz (romanlutz) force-pushed the copilot/romanlutz-scenario-pr-03-live-upstream branch from 80a40b7 to 7fddb1d Compare August 25, 2026 00:56
@romanlutz
Roman Lutz (romanlutz) force-pushed the copilot/romanlutz-scenario-pr-03-live-upstream branch from 7fddb1d to e43a331 Compare August 25, 2026 03:45
@romanlutz
Roman Lutz (romanlutz) force-pushed the copilot/romanlutz-scenario-pr-03-live-upstream branch from e43a331 to 9d55233 Compare August 25, 2026 06:24
@rlundeen2
Richard Lundeen (rlundeen2) force-pushed the copilot/romanlutz-scenario-pr-03-live-upstream branch 3 times, most recently from 2fd41d4 to b6ea257 Compare September 1, 2026 21:00
Base automatically changed from copilot/romanlutz-scenario-pr-02-catalog-upstream to main September 1, 2026 21:34
Copilot AI added 2 commits September 1, 2026 14:34
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5d02c2d5-b499-4f78-a04d-03bffa750817
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@rlundeen2
Richard Lundeen (rlundeen2) force-pushed the copilot/romanlutz-scenario-pr-03-live-upstream branch from b6ea257 to cda766e Compare September 1, 2026 21:34
@behnam-o Behnam (behnam-o) self-assigned this Sep 2, 2026

@behnam-o Behnam (behnam-o) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR should be scoped to the frontend live-progress experience; no changes under /pyrit are needed here. The progress and cancellation backend APIs come from earlier stack layers, ScenarioRunPlanGroupKind is only consumed by #2378, and the dataset-size override contract is first consumed by #2377. Moving those backend changes to their corresponding follow-up PRs would keep this layer focused and independently reviewable.

Comment thread pyrit/models/scenario_progress.py Outdated
SCENARIO_RUN_PLAN_VERSION = 1


class ScenarioRunPlanGroupKind(str, Enum):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we move this enum and group_kind field to the later PR that actually introduces multiple group kinds? In this PR the enum has only ATTACK, and the field is only persisted—not consumed—so it adds a discriminator without distinguishing anything. If this needs to land here for forward compatibility with the stacked changes, it would help to include the upcoming values or otherwise make that dependency explicit.

#: caller-supplied ``include_baseline=True`` raises ``ValueError``.
BASELINE_ATTACK_POLICY: ClassVar[BaselineAttackPolicy] = BaselineAttackPolicy.Enabled

#: Whether the default estimator must mirror matrix-builder seed compatibility.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we move this dataset-sizing contract to #2377, where it is first consumed? The live-progress work in this PR does not use DATASET_SIZE_LIMIT_OVERRIDE_SCOPE or get_dataset_size_limit_override_scope; #2377 wires it into the scenario registry and adds the scenario-specific overrides. Keeping it there would make this PR frontend-only and keep the backend sizing change with its actual behavior.

Behnam Ousat added 2 commits September 2, 2026 10:31
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 5ece2d4e-f111-4a4a-80ba-39a59edd1298
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 5ece2d4e-f111-4a4a-80ba-39a59edd1298
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants