Version 3.6.3

3.6.3 (27 September 2024) - hotfix:

Fixed

• Fix interactive console failing on macro imports.
  • Now __init__.py imports mcpyrate.activate as soon as possible.
  • Neither in-tree tests nor the CI automation detected this. It was only when running macropython -i (or IPython with the mcpyrate.repl.iconsole extension) in a separate terminal window, against an installed mcpyrate, that this error came up.
  • For those arriving from Google, the error message was: ImportError: cannot import name 'macros' from 'mcpyrate.quotes'

No other changes to 3.6.2, original announcement below.

---

3.6.2 (27 September 2024) - New tree snakes edition:

IMPORTANT:

• Minimum Python language version is now 3.8.
  • We support 3.8, 3.9, 3.10, 3.11, 3.12, and PyPy3 (language versions 3.8, 3.9, and 3.10).
  • Python 3.6 and 3.7 support dropped, as these language versions have officially reached end-of-life. If you need mcpyrate for Python 3.6 or 3.7, use version 3.6.1.

New:

• Python 3.12 support.

  • Support the type statement (Python 3.12 type alias) when fixing ctx attributes in the global postprocess pass.
  • The unparser now supports the type statement (Python 3.12 type alias).
    • Please note that I pretty much don't use the static typing features of Python myself. This is implemented following the spec, but testing has been rather minimal, so bug reports are welcome!
    • For the curious, the relevant parts of the official Python documentation are:
      • https://docs.python.org/3/library/ast.html#type-parameters
      • https://docs.python.org/3/library/ast.html#ast.TypeAlias
      • https://docs.python.org/3/library/ast.html#ast.TypeVar
      • https://docs.python.org/3/library/ast.html#ast.ParamSpec
      • https://docs.python.org/3/library/ast.html#ast.TypeVarTuple
      • https://docs.python.org/3/library/typing.html#typing.TypeVar
      • https://docs.python.org/3/library/typing.html#typing.ParamSpec
      • https://docs.python.org/3/library/typing.html#typing.TypeVarTuple

• Python 3.11 support.

  • Consider also end_lineno and end_col_offset when fixing AST locations in the global postprocess pass.
    • This is literally the only thing we currently do with end_lineno and end_col_offset.
    • Python 3.11's AST validator (now part of the CPython compiler as of 3.11+) checks e.g. that end_lineno >= lineno.
  • The unparser now supports the try/except* construct (Python 3.11 exception groups).

• New module mcpyrate.astcompat, moved from unpythonic.syntax.astcompat. This module handles version differences in the ast module in various versions of Python.

Fixed:

• Fix #41. The unparser now supports match/case (Python 3.10 pattern matching).
• Fix bug in unparser: a class definition with no bases or keywords should not emit parentheses after the class name.
• Fix bug in rename: rename also in global and nonlocal declarations, and (Python 3.10+) in match/case captures.
• Update links in relevant parts of mcpyrate documentation to point to Python's official AST documentation instead of GTS (Green Tree Snakes).
  • Nowadays Python has proper AST documentation.
  • Thus the separate GTS resource is no longer needed, and is effectively dead as of September 2024.

Version 3.6.2

3.6.2 (27 September 2024) - New tree snakes edition:

IMPORTANT:

• Minimum Python language version is now 3.8.
  • We support 3.8, 3.9, 3.10, 3.11, 3.12, and PyPy3 (language versions 3.8, 3.9, and 3.10).
  • Python 3.6 and 3.7 support dropped, as these language versions have officially reached end-of-life. If you need mcpyrate for Python 3.6 or 3.7, use version 3.6.1.

New:

• Python 3.12 support.

  • Support the type statement (Python 3.12 type alias) when fixing ctx attributes in the global postprocess pass.
  • The unparser now supports the type statement (Python 3.12 type alias).
    • Please note that I pretty much don't use the static typing features of Python myself. This is implemented following the spec, but testing has been rather minimal, so bug reports are welcome!
    • For the curious, the relevant parts of the official Python documentation are:
      • https://docs.python.org/3/library/ast.html#type-parameters
      • https://docs.python.org/3/library/ast.html#ast.TypeAlias
      • https://docs.python.org/3/library/ast.html#ast.TypeVar
      • https://docs.python.org/3/library/ast.html#ast.ParamSpec
      • https://docs.python.org/3/library/ast.html#ast.TypeVarTuple
      • https://docs.python.org/3/library/typing.html#typing.TypeVar
      • https://docs.python.org/3/library/typing.html#typing.ParamSpec
      • https://docs.python.org/3/library/typing.html#typing.TypeVarTuple

• Python 3.11 support.

  • Consider also end_lineno and end_col_offset when fixing AST locations in the global postprocess pass.
    • This is literally the only thing we currently do with end_lineno and end_col_offset.
    • Python 3.11's AST validator (now part of the CPython compiler as of 3.11+) checks e.g. that end_lineno >= lineno.
  • The unparser now supports the try/except* construct (Python 3.11 exception groups).

• New module mcpyrate.astcompat, moved from unpythonic.syntax.astcompat. This module handles version differences in the ast module in various versions of Python.

Fixed:

• Fix #41. The unparser now supports match/case (Python 3.10 pattern matching).
• Fix bug in unparser: a class definition with no bases or keywords should not emit parentheses after the class name.
• Fix bug in rename: rename also in global and nonlocal declarations, and (Python 3.10+) in match/case captures.
• Update links in relevant parts of mcpyrate documentation to point to Python's official AST documentation instead of GTS (Green Tree Snakes).
  • Nowadays Python has proper AST documentation.
  • Thus the separate GTS resource is no longer needed, and is effectively dead as of September 2024.

Version 3.6.1

Fixed:

• Fix #33, with thanks to @set-soft.

Version 3.6.0

3.6.0 (28 January 2022) New Year's edition:

Added:

• Python 3.10 support.
• Add block macro mcpyrate.metatools.expand_first. This can be used to force, within the with expand_first[macro0, ...]: block, the given macros to expand before others. Macros can be specified either by name (will be looked up in the current expander's bindings) or by hygienic capture. See examples in unit tests.
• Add function mcpyrate.utils.get_lineno to conveniently extract a lineno from an AST-node-ish thing, no matter if that thing is an actual AST node, a list of AST nodes (i.e. statement suite), or an AST marker containing either of those, possibly recursively.
• Facilitate programmatic inspection of the whole public API of mcpyrate. See the recipes in troubleshooting.
  • This is an interim solution while we decide whether to start supporting Sphinx at some point, so that we could auto-generate proper API docs from the docstrings (which are carefully maintained, and already contain all the necessary content).

Fixed:

• Fix #29, with thanks to @set-soft and @brathis for reporting. mcpyrate should now support Python 3.10.
• Dialect subsystem fixes.
  • Fix #30, thus extending the fix of #28 (in the previous release) into the dialect subsystem, too.
    • __future__ imports are accounted for both the dialect template and in user code that invokes the template.
    • This is implemented in the utility function mcpyrate.splicing.splice_dialect, so if your dialect definition uses that function in its AST transformer, now your dialect should not choke when the template and/or the user code have __future__ imports.
  • Fix #31; the dialect machinery now has the infrastructure to pass in the source location info of the dialect-import statement.
    • This allows dialects to mark any lines coming from the dialect template as effectively coming from the line that contains the dialect-import. If you import one dialect per line, this makes it easy to see which lines of the expanded code were injected by which dialect, for debugging purposes. (Recall that you can use the StepExpansion dialect from mcpyrate.debug to see the line numbers before and after dialect expansion.)
    • During dialect expansion, DialectExpander automatically makes this info available in self.lineno and self.col_offset of your dialect definition (i.e. in the instance of your subclass of Dialect, which has the transformer methods). In your AST transformer, you can pass these to mcpyrate.splicing.splice_dialect.
    • See updated example dialects in unpythonic.dialects.
  • Fix handling of rare case where the dialect template consists of a single statement that is not wrapped in a list.
• Docstring of mcpyrate.utils.NestingLevelTracker now has usage examples.

Version 3.5.3

Fixed:

• #28: Using __future__ imports when multiphase compilation is enabled causes SyntaxError.

Version 3.5.2

3.5.2 (22 June 2021) - Midsummer's eve edition:

Changed:

• Small improvements to unparser:

  • No space after unary +, - or ~.
  • Future-proofing: yell if an unsupported constant value type is encountered.

• Add a new troubleshooting item on another Heisenbug that can occur when buggy macros are used inside a with step_expansion.

Version 3.5.1

3.5.1 (26 May 2021) - Detailed logbook edition:

Changed:

• Documentation improved. Particularly, AST markers are now documented (in the main user manual).

Version 3.5.0

3.5.0 (9 May 2021)

New:

• Add temporary_module, a context manager that uses create_module, and automatically removes the temporary module from sys.modules when the context exits.

• Add a global postprocessor hook facility. Hooks are called, in the order registered, by global_postprocess when the macro expansion of a module is otherwise done. This e.g. allows a macro library to use its own ASTMarker subclasses for internal communication between macros, and delete (only) its own markers when done. See add_postprocessor and remove_postprocessor in mcpyrate.core.

Fixed:

• Run-time part of n[]: upon a parse error, make it clearer in the error message that what was being compiled was an invocation of n[], not the whole source file. (Because these expressions are often one-liners, usually lineno will be 1, which otherwise looks confusing.)

• Fix error message in run-time typecheck of a (ast-unquote). Now it mentions correctly what was expected.

• Now ASTMarker may contain a statement suite (list of AST nodes) as its body.

  • The debug mode of mcpyrate.unparse now renders such bodies correctly.
  • mcpyrate.markers.delete_markers now deletes such markers correctly, splicing in the list of AST nodes where the marker was.

Version 3.4.1

3.4.1 (4 May 2021):

Changed:

• Update docs: as of unpythonic 0.15, it runs on mcpyrate, and provides fully functional example dialects based on a whole-module AST transformation.
• The colorizer now injects some styles to Style that are missing from colorama 0.4.4, particularly ITALIC.

Fixed:

• Now we pass a filename to ast.parse everywhere. This allows e.g. SyntaxError during macro-import scanning (in the macro-import dependency graph analyzer), and possible internal errors in the interactive consoles, to report the filename correctly.

Version 3.4.0

3.4.0 (2 May 2021) - Quasiquotes ahoy edition:

New:

• The unparser now recognizes hygienic captures and destructures them in debug mode. This makes the result much more readable when you unparse an AST that uses a lot of hygienic unquotes.

  • To see it in action, use mcpyrate.debug.step_expansion macro on unpythonic.syntax.tests.test_lazify. See particularly the HasThon test; both the autocurry and the lazifier produce many hygienic captures.

    Without this helpful destructuring, the macro-expanded code is completely unreadable, but with this, it only exhibits mild symptoms of parenthesitis. For example, this snippet:

    filename=callsite_filename()

    becomes, after autocurry and lazification,

    filename=$h[Lazy]((lambda: $h[maybe_force_args]($h[force]($h[currycall]),
                                                    $h[Lazy]((lambda: $h[force]($h[callsite_filename]))))))

    Here each $h[...] is a hygienic capture. That's seven captures for this very simple input! Compare this notation to the actual AST representation of, e.g., $h[Lazy]:

    __import__('mcpyrate.quotes', globals(), None, (), 0).quotes.lookup_value(('Lazy',
        b'\x80\x04\x95 \x00\x00\x00\x00\x00\x00\x00\x8c\x13unpythonic.lazyutil\x94\x8c\x04Lazy\x94\x93\x94.'))

Fixed:

• The importer now reports the source location if destructuring a macro invocation candidate fails.

  • Some internal functions, including mcpyrate.expander.destructure_candidate, now take a mandatory filename kwarg for this purpose.

• Fix detection of globally bound macro invocations (hygienic macro captures) in the helper method mcpyrate.expander.ismacrocall.

• Fix syntax analysis for detecting expr macro invocations in mcpyrate.expander.destructure_candidate. Version 3.3.0 (and only that version) errored out on the AST for f()[...] even if f was not bound as a macro.