Skip to content

test(e2e): add API conformance framework - #2873

Draft
elezar wants to merge 7 commits into
codex/2183-cli-conformance-minimalfrom
codex/2183-api-conformance-framework
Draft

test(e2e): add API conformance framework#2873
elezar wants to merge 7 commits into
codex/2183-cli-conformance-minimalfrom
codex/2183-api-conformance-framework

Conversation

@elezar

@elezar elezar commented Aug 21, 2026

Copy link
Copy Markdown
Member

Summary

Add the direct-gRPC API conformance framework after the portable CLI baseline is established. The framework runs named, driver-independent behavioral scenarios against an already provisioned gateway and produces structured results.

This is stack 2/2 and is based on the combined CLI conformance baseline in PR #2871. The original prototype PR #2182 remains unchanged for comparison.

Related Issue

Changes

  • add an internal openshell-conformance runner and reusable scenario engine
  • add API scenarios for lifecycle, not-found behavior, repeated deletion, validation, concurrency, labels, streamed command execution, and process hardening
  • run the API profile through existing Docker, Podman, Kubernetes, and VM gateway provisioners
  • export a common gateway endpoint and mTLS connection contract from provisioners
  • migrate the existing core-dump assertion into the cross-driver API baseline
  • document scenario execution, cleanup, output formats, and driver boundaries

Testing

  • locked compilation of affected API, CLI, and VM targets
  • 27 focused unit tests across the E2E library and runner
  • focused Clippy with warnings denied
  • Rust formatting checks
  • shell syntax checks for modified provisioners and wrappers
  • openshell-conformance list --output json
  • git diff --check
  • live Docker, Podman, Kubernetes, and VM API conformance runs
  • mise run pre-commit (mise is unavailable in the current environment)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • API and CLI conformance remain separate profiles
  • No production code or production dependencies changed
  • Documentation updated

elezar added 7 commits August 21, 2026 13:26
Signed-off-by: Evan Lezar <elezar@nvidia.com>

# Conflicts:
#	e2e/rust/Cargo.toml
#	e2e/rust/e2e-kubernetes.sh
#	e2e/rust/e2e-podman.sh
#	e2e/rust/e2e-vm.sh
#	tasks/test.toml
Signed-off-by: Evan Lezar <elezar@nvidia.com>

# Conflicts:
#	e2e/rust/CONFORMANCE.md
#	e2e/rust/e2e-vm.sh
#	e2e/rust/tests/sandbox_lifecycle.rs
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
Signed-off-by: Evan Lezar <elezar@nvidia.com>
@elezar elezar added the test:e2e Requires end-to-end coverage label Aug 21, 2026
@copy-pr-bot

copy-pr-bot Bot commented Aug 21, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown

Label test:e2e applied, but pull-request/2873 does not exist yet. A maintainer needs to comment /ok to test cf9d7ef76856eddb57f4ccff801fdaceabbb3f1d to mirror this PR. Once the mirror exists, re-apply the label or re-run Branch E2E Checks from the Actions tab.

Base automatically changed from codex/2183-cli-conformance-overlap to codex/2183-cli-conformance-minimal August 21, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant