feat(providers): expose actionable OAuth refresh failures

[mrunalp]

User Story

As an application or UI that consumes OpenShell provider refresh status, I want a stable indication of whether an OAuth refresh failure is retryable, requires operator configuration, or requires the user to reconnect their account, so that I can present the correct recovery action without parsing provider-specific error text.

Problem Statement

The gateway refresh worker exchanges a stored OAuth refresh token for a short-lived access token. The refresh token remains in the active credential driver while the gateway periodically replaces the access token.

When the token endpoint returns a non-success response, OpenShell currently loses the structured OAuth error and exposes only a generic failure such as:

token endpoint returned HTTP 400 Bad Request

HTTP status alone does not identify the correct recovery action. OAuth token endpoints can return HTTP 400 for materially different conditions:

• invalid_grant: the refresh grant is expired, revoked, invalid, or subject to an interactive reauthentication requirement. User action is normally required.
• invalid_client: the configured client ID or secret is wrong. An operator must repair configuration; asking the user to log in again is insufficient.
• invalid_scope, unauthorized_client, or unsupported_grant_type: the provider profile or OAuth client configuration is invalid.
• admin_policy_enforced: an administrator has blocked the requested access.
• An unknown or malformed response: its disposition is uncertain.

Google may also return invalid_grant with an error_subtype such as invalid_rapt, indicating that an administrator's reauthentication policy requires a new interactive authorization session.

As a result, refresh status cannot tell consumers whether the gateway should retry, an operator should repair OAuth configuration, or the user must complete interactive authorization again. Terminal failures can also be retried every minute even though retrying cannot repair the grant.

Impact / Why This Matters

Consumers such as OpenClaw cannot reliably decide when to show a “Reconnect Gmail” or equivalent sign-in action. Their current options are to parse unstable human-readable messages, treat every HTTP 400 as requiring SSO, or avoid presenting a recovery action at all.

Those workarounds are insufficient because they can unnecessarily interrupt users for configuration failures, hide genuine reauthorization requirements until the access token expires, and create repeated refresh traffic and operational noise for terminal failures. SDKs and automated operators likewise need a machine-readable recovery contract rather than provider prose.

With a structured recovery action, consumers can deterministically:

• Show a reconnect or sign-in action only for reauthorize.
• Notify an operator rather than the user for fix_configuration.
• Retry only genuinely transient failures.
• Escalate unfamiliar provider responses as investigate.

Proposed Design

Parse a bounded OAuth error response

For non-success responses, decode recognized standard OAuth fields such as error and error_subtype. Apply a small response-size limit and strict length limits to individual fields.

Never retain or log the raw response, arbitrary error_description text, request form, access token, refresh token, client secret, or query parameters. Persist only gateway-owned codes and recognized safe subtypes.

Classify the recovery action

Expose a stable, provider-neutral recovery action:

Refresh classification	Typical signals	Recovery action	Gateway behavior	Operator or user action
reauthorization_required	invalid_grant, including invalid_rapt	reauthorize	Preserve the current access credential until it expires, stop rapid retries, then fail closed	User reconnects the account and installs the replacement grant
configuration_required	invalid_client, unauthorized_client, invalid_scope, unsupported_grant_type, admin_policy_enforced	fix_configuration	Stop rapid retries and surface a stable reason	Operator repairs the OAuth client, provider profile, scopes, or administrator policy
retryable	Network failures, timeouts, HTTP 429, HTTP 5xx, server_error, temporarily_unavailable	retry	Retry with capped exponential backoff and jitter	No immediate user action
unknown	Malformed or unrecognized response	investigate	Retry conservatively with bounded backoff and expose the uncertainty	Investigate if persistent

Provider-specific mappings may refine this table, but the externally visible classifications must remain provider-neutral.

Persist actionable status

Refresh status should expose structured, non-secret fields such as:

• failure_code: a stable gateway code such as oauth_invalid_grant
• recovery_action: reauthorize, fix_configuration, retry, or investigate
• provider_error: a bounded, recognized OAuth error value
• provider_error_subtype: a bounded subtype when recognized
• last_error_at_ms: the time of the last failure

Keep last_error as a safe human-readable summary, but do not require consumers to parse it. For example:

Status: Reauthorization required
Reason: OAuth refresh grant is no longer usable (invalid_grant)
Action: Reconnect the account and configure the replacement refresh grant

Define the reauthorization boundary

OpenShell cannot mint a replacement refresh token by itself. Interactive authorization happens outside the gateway. Once the user obtains a new grant, provider refresh configure installs it and begins a new authorization epoch.

Until then:

• Do not delete the unusable refresh material automatically; retain it for diagnosis and explicit cleanup.
• Do not expose refresh material to the sandbox or UI.
• Continue using the current access token only until its recorded expiration.
• Reject expired credential resolution instead of forwarding stale credentials.
• A manual rotate may retry the exchange but must not clear reauthorization_required unless it succeeds.

Refresh-token storage remains unchanged. Secret material stays behind opaque handles in the configured credential driver, including Vault, Kubernetes Secrets, or the default database credential store.

Acceptance Criteria

• Provider refresh status exposes a stable recovery action without requiring consumers to parse error strings.
• A user OAuth refresh grant rejected with invalid_grant reports reauthorize with a stable failure code and does not continue automatic rapid retries.
• invalid_grant with a recognized subtype such as invalid_rapt retains the safe subtype and requests interactive reauthorization.
• Non-interactive grants and OAuth client, scope, grant-type, or administrator-policy failures report fix_configuration rather than asking the user to sign in.
• Network failures, HTTP 429, HTTP 5xx, server_error, and temporarily_unavailable remain retryable with bounded backoff.
• Oversized, malformed, HTML, and unrecognized provider responses are safely bounded and report investigate.
• Existing access credentials remain usable only until their recorded expiration and then fail closed.
• A successful refresh or explicit refresh reconfiguration clears prior structured failure state and begins a new authorization epoch.
• OAuth error handling never persists or emits credential material, request material, or arbitrary raw provider responses.
• The behavior works with the existing credential-driver abstraction and does not change where refresh material is stored.
• End-to-end coverage with a real OAuth issuer verifies successful refresh, grant revocation, and the resulting reauthorization status.

Alternatives Considered

• Continue exposing only HTTP status and human-readable errors. This leaves every consumer to implement fragile string parsing and cannot distinguish failures that share an HTTP status.
• Treat every HTTP 400 as requiring user login. This incorrectly prompts users for invalid client configuration, scope errors, unsupported grant types, and administrator-policy failures.
• Retry every failure indefinitely. This creates noise and load without recovering expired or revoked grants, while still failing to tell the user what action is required.
• Expose provider error bodies directly. Provider-controlled text is not a stable API contract and may contain sensitive or unsuitable diagnostic content.

The proposed structured recovery action gives consumers a small, stable contract while retaining a safe human-readable diagnostic for operators.

Agent Investigation

• The gateway refresh path owns token exchange, refresh scheduling, credential expiry, and persistence, so it can classify failures once and expose one consistent result to all clients.
• A bounded OAuth error parser can distinguish user refresh grants from non-interactive grants and avoid incorrectly requesting user login for client-credentials or service-account failures.
• Unit coverage should include classification, persistence, retry behavior, bounded error handling, safe diagnostics, user versus non-interactive grants, and rotated refresh-token storage failures.
• Local validation used a dedicated rootless Podman lane with a real Keycloak instance only as the provider OAuth issuer. OpenShell authentication remained local mTLS, and refresh material used the default database credential store.
• The validation refreshed a valid grant, revoked it through Keycloak logout, observed the next refresh return invalid_grant, and verified the persisted reauthorization_required, reauthorize, and oauth_invalid_grant status with no automatic next retry.
• The local validation lane is mise run e2e:provider-refresh-keycloak.
• Related refresh-material storage work: feat(providers): store refresh credentials in credential drivers #2801

Checklist

• I've reviewed existing issues and the architecture docs
• This is a design proposal, not a "please build this" request