Skip to content

Recognize Literal parameter annotations - #908

Open
fzlzjerry wants to merge 8 commits into
nteract:mainfrom
fzlzjerry:fix/906-literal-parameters
Open

Recognize Literal parameter annotations#908
fzlzjerry wants to merge 8 commits into
nteract:mainfrom
fzlzjerry:fix/906-literal-parameters

Conversation

@fzlzjerry

@fzlzjerry fzlzjerry commented Aug 10, 2026

Copy link
Copy Markdown

What does this PR do?

Papermill's Python parameter inspector currently validates annotation text with a narrow character class. String-valued Literal annotations therefore fail to match, causing an otherwise valid parameter to be omitted and later reported as unknown.

This change uses Python's AST for syntactically valid parameter cells so assignment boundaries, annotations, and default values follow Python syntax rather than a character class. It preserves source spelling for ordinary annotations, decodes whole string annotations from their AST constant value, and keeps trailing type/help comments.

Before a second parse attempt, standalone IPython magic, shell, and help commands are replaced with valid position-preserving statements. Assigned !/% commands are replaced by fallible AST placeholders with matching UTF-8 byte widths, while their original command source remains the reported default. Cells that still cannot be parsed continue through the existing best-effort regex fallback.

The AST path also:

  • traverses control-flow blocks while stopping at function, class, and lambda scope boundaries
  • removes comments with Python tokenization so # characters inside quoted annotation values remain intact
  • distinguishes whitespace-delimited help comments on assigned commands from embedded shell fragments such as /#frag
  • removes explicit line-continuation backslashes without stripping backslashes inside multi-line string tokens
  • extracts the complete right-hand side after the top-level assignment operator, preserving syntax-required parentheses around assignment expressions

Regression coverage includes:

  • unquoted and fully quoted Literal[...] annotations, including escaped quotes that use the outer string's quote style
  • partially quoted forward references on either side of a union
  • Literal["key=value", ...] and nested annotation keyword arguments without treating their = characters as assignment boundaries
  • Literal["C#", ...] across multiple lines without treating the string's # as a comment
  • standalone and assigned IPython commands alongside valid Literal declarations
  • a command as the only statement in a compound suite
  • non-ASCII assigned commands with AST byte offsets preserved
  • URL hash fragments and whitespace-delimited command help comments
  • explicit annotation continuations and raw triple-quoted defaults ending a physical line with a backslash
  • assignments in conditional blocks without collecting function-local assignments
  • parenthesized assignment-expression defaults
  • semicolon-separated statements without swallowing the preceding annotation-only declaration

Fixes #906

Tests

  • full test suite: 571 passed
  • focused Python translator tests: 299 passed
  • Black check on changed files
  • Ruff check on changed files
  • git diff --check

@fzlzjerry
fzlzjerry marked this pull request as ready for review August 11, 2026 05:51
Copilot AI lite review requested due to automatic review settings August 11, 2026 05:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃挕 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1e4b4ebef7

鈩癸笍 About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 馃憤.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃挕 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6827089721

鈩癸笍 About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 馃憤.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃挕 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e22bc9fa67

鈩癸笍 About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 馃憤.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃挕 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 61527cd23e

鈩癸笍 About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 馃憤.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃挕 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eae61bde73

鈩癸笍 About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 馃憤.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃挕 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dca3cfa193

鈩癸笍 About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 馃憤.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃挕 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eefb199220

鈩癸笍 About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 馃憤.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread papermill/translators.py
Comment thread papermill/translators.py Outdated
Comment thread papermill/translators.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Literal annotation results in unrecognized parameter

2 participants