Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#432 Use importlib.metadata.version to get the version #502

Merged
merged 22 commits into from
May 22, 2024

Conversation

agriyakhetarpal
Copy link
Contributor

Description

This PR updates the get_version() method in src/towncrier/_project.py to use importlib.metadata.version() [Fixes #432]

Checklist

  • Make sure changes are covered by existing or new tests.
  • For at least one Python version, make sure local test run is green.
  • Create a file in src/towncrier/newsfragments/. Describe your
    change and include important information. Your change will be included in the public release notes.
  • Make sure all GitHub Actions checks are green (they are automatically checking all of the above).
  • Ensure docs/tutorial.rst is still up-to-date.
  • If you add new CLI arguments (or change the meaning of existing ones), make sure docs/cli.rst reflects those changes.
  • If you add new configuration options (or change the meaning of existing ones), make sure docs/configuration.rst reflects those changes.

@agriyakhetarpal
Copy link
Contributor Author

Some tests in VersionFetchingTests are currently failing locally because importlib.metadata.version() requires a package to be either installed or editable-installed, and the tests do not do so currently when creating a temporary project. How may I proceed? I have caught PackageNotFoundError in a try-except block for now so that the functionality is retained

Alternatives are to try to use importlib.metadata.distribution to get the version, or to just parse pyproject.toml — the latter is not in consideration since it is neither the current standard nor Pythonic.

@SmileyChris
Copy link
Contributor

See also: #491

@hynek
Copy link
Member

hynek commented May 7, 2023

I'm not sure how I feel about this whole"install towncrier along with your package" -- it kinda seems like a misfeature / attractive nuisance for the reasons we've just seen. :-/

I would argue for deprecation and adding ways to extract the version from the outside (pyproject.toml / extract using regexp from a file / run a command [e.g. hatch]).

@agriyakhetarpal
Copy link
Contributor Author

It would be much easier to use a toml parser library to extract package versions from pyproject.toml. With hatch as the build backend, it is just as easy to get it just as it is to update it – from hatch version, which also might be extracting it from [tool.hatch.version] or [project], I am not sure.

@hynek
Copy link
Member

hynek commented May 22, 2023

It’s not that simple: hatch supports dynamic versions using plugins like hatch-vcs that use git metadata. Running hatch would be probably the most correct but also the most invasive approach.

I for one would claim that the user can run towncrier build --version $ (hatch version) themselves just fine?

Don’t y’all feel too that these features are children of a bygone Python packaging era when things were simpler and more homogenous?

@adiroiban
Copy link
Member

I think that towncrier build --version $(hatch version) already works.

At the same time, I think that we should make simple things simple.

So if the version is already defined as a simple text value inside the .toml file, I think that it makes sense to accept a PR in which the version is automatically extracted.

We can document that this only works for limited use cases.


I prefer to have all the common commands stored in file.
So maybe towncrier build --version $(hatch version) can be implemented as

If towncrier version starts with shell, then we will call an external command.
I think that we already are calling git commands

[tool.towncrier]
version = "shell:hatch version" 

or if someone already has the package installed, they can use something like

[tool.towncrier]
version = "importlib.metadata" 

These options will not work for any case... but I don't think that we should reject a PR adding support for a specific use case.

I expect that the majority of project will try to keep things simple, and have the version as a string inside the .toml file.


Just my feedback

@hynek
Copy link
Member

hynek commented Jun 4, 2023

Hmm either way I'm sorry, I think the approach in this PR is the more correct one even though I disagree with the feature as a whole.

It will need something like this and a conditional dependency 'importlib_metadata; python_version<'3.8'" to work on 3.7, though.

Please consider this un-bikeshedded otherwise.

@adiroiban
Copy link
Member

So for this PR to be merged it needs:

  • newsfragment / release notes to advertise this feature
  • (nice to have) Update the existing documentation to mention that if you have your project installed, towncrier can extract the version
  • automated tests to check the new feature - for this, we might want automatically create a super simple package as part of the test suite, install that package and check that version is detected.

I think there are already some tests generating setup.cfg/pyproject.toml on the fly and executing towncrier.

With the automated tests, there is one big problem, if you start a python interpreter/process and while the process is running you install packages... the metadata of the new pacage might not be available.

For my code sometimes I am calling pip like this... and I get a warning that this usage is not supported:

from pip import main
pip_arguments = ['install', 'some-test-package']
result = main(args=pip_arguments)

It works... if in the same process I call uninstall pip complains that the package doesn't exist.

So just something to take into consideration when testing.

@adiroiban
Copy link
Member

I triggered the current test suite to see if there are any regression that might be introduced by this change.

To me, this approach is harmeless ... if you don't have the package install it should work as before.

Also, I don't know if we should worry about python 3.7 ... It will be EOL soon.

@hynek
Copy link
Member

hynek commented Jun 4, 2023

How about we (read: you 😇) cut a release and drop 3.7 afterwards?

(Pls someone merge my typo PR first tho 😅)

@agriyakhetarpal
Copy link
Contributor Author

Dropping Python 3.7 before this makes a lot of sense, though the backport of importlib.metadata works fine as well. I was waiting on a consensus before working further and it shall hopefully be fine using a toml parser rather than using regex, just for the sake of convenience

@adiroiban
Copy link
Member

Well... I think that the overall feedback is that people are expecting from towncrier to auto-magically detect the version of their project.

Using importlib.metadata_version should be the most standard way of doing this.

So the direction of this PR is good.

But in order to merge this PR, we need all tests to be green.

@agriyakhetarpal
Copy link
Contributor Author

agriyakhetarpal commented Jun 5, 2023

Oh yes, I just saw that. I meant that I could wait until the next release until 3.7 support is dropped as @hynek suggests, and then work on it

@adiroiban adiroiban mentioned this pull request Jul 9, 2023
7 tasks
src/towncrier/_project.py Show resolved Hide resolved

version = getattr(module, "__version__", None)
try:
version = metadata_version(f"{module}")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The str of a module is not something you can pass to importlib.metadata.version because you get a repr-like string:

>>> module = importlib.import_module("towncrier")
>>> print(f"{module}")
<module 'towncrier' from '/Users/.../site-packages/towncrier/__init__.py'>

which is probably just a small mistake here, but it reminds me that in fact version() does not accept an Import Package name at all, but rather a Distribution Package. They're commonly the same but not necessarily so, to use some popular examples:

Import Package Distribution Package
attr1 attrs
yaml PyYAML

As an example, let's say I'm the PyYAML developer, and I'm using towncrier with a __version__ variable via package = "yaml". If I want to drop that variable and specify my version in pyproject.toml instead, then we can't pass yaml to importlib.metadata.version. We have to load a mapping from one to the other:

>>> import importlib.metadata
>>> importlib.metadata.packages_distributions()['yaml']
['PyYAML']
>>> importlib.metadata.version('PyYAML')
'6.0.1'

which in the context of get_version would be something like:

dist_packages = distributions_packages()
try:
    dist_package = dist_packages[package]
except KeyError:
    # do fallback to package.__version__ here
else:
    if len(dist_package) > 1:
        raise Exception("I found more than one package")
    version = metadata_version(dist_package[0])

So that's how we'd make towncrier use importlib.metadata in a backwards-compatible way (user doesn't have to change their tool.towncrier config).

It's a little unfortunate, though, that I have to do this:

[project]
name = "PyYAML"
version = "1.2.3"

[tool.towncrier]
package = "yaml"

The version associated with the distribution package is right there, but then I have to tell towncrier one of the import packages I have so that it can look up the name of my distribution package.

PEP 621 says that the project.name is required to be statically defined, which means towncrier could try to read project.name by default if package is not defined and --version is not passed.

Notably Poetry doesn't yet support PEP 621 but the package would still be there for Poetry users.

Footnotes

  1. attrs also has its newer attrs namespace of course :-)

@agriyakhetarpal
Copy link
Contributor Author

agriyakhetarpal commented Feb 19, 2024

I realise that this PR has been lost in the hallows of my GitHub account... and it has taken almost an year for me to get back to this, following the notification for the received review, thanks! 🙂

I will shall look to rebase this branch soon and finish this up, because it looks like other PRs are now depending on this (such as #491).

@SmileyChris
Copy link
Contributor

SmileyChris commented May 7, 2024

So something like my last commit in this PR, @RazerM ?

@agriyakhetarpal
Copy link
Contributor Author

I think the message in the exception raised needs to be updated as well for the test to pass

src/towncrier/_project.py Outdated Show resolved Hide resolved
src/towncrier/_project.py Show resolved Hide resolved
@SmileyChris SmileyChris marked this pull request as ready for review May 22, 2024 00:04
@SmileyChris SmileyChris requested a review from a team as a code owner May 22, 2024 00:04
Copy link
Member

@adiroiban adiroiban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good. Thanks.

Only minor comments.


There might be things that can be improved. but I think that we can do that in a separate PR.

Thanks again.

pyproject.toml Outdated
@@ -31,6 +31,7 @@ requires-python = ">=3.8"
dependencies = [
"click",
"importlib-resources>=5; python_version<'3.10'",
"importlib-metadata==4.6.*; python_version<'3.10'",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this needs a comment to explain why we are pinning on 4.6 version.

Why not use the latest version ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A better pin would be >=4.6 I guess (this is the version that had parity with Python 3.10).

src/towncrier/_project.py Show resolved Hide resolved
src/towncrier/_project.py Show resolved Hide resolved
src/towncrier/_project.py Outdated Show resolved Hide resolved
src/towncrier/test/test_project.py Show resolved Hide resolved
src/towncrier/_project.py Outdated Show resolved Hide resolved
@SmileyChris SmileyChris enabled auto-merge (squash) May 22, 2024 21:03
@SmileyChris SmileyChris merged commit 6d96010 into twisted:trunk May 22, 2024
16 checks passed
@agriyakhetarpal agriyakhetarpal deleted the 432-poetry-version branch May 22, 2024 21:06
@adiroiban adiroiban mentioned this pull request Jul 28, 2024
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Get the version from pyproject.toml [tool.poetry] section
5 participants