Unwanted submodule inside main module makes documentation look weird (and also breaks documentation generators)

[Smertig]

Bug Description

When I build a simple string_sum project from examples, I get weird submodule inside main module that has the same name as a main module (i.e. string_sum.string_sum). I think this caused by module reexporting, but it breaks both docstrings and also many doc generators like pdoc3 (see examples below).

P.S. I don't really know whether it's bug or "by design" feature.

Steps to Reproduce

I use step-by-step guide from introduction:

1. Create and build dummy project

mkdir string_sum && cd string_sum
python -m venv .env
.env\Scripts\Activate.bat
pip install maturin
maturin init --bindings pyo3
maturin develop

2. Import it and try to use string_sum:

$ python
>>> import string_sum
>>> help(string_sum)
Help on package string_sum:

NAME
    string_sum - A Python module implemented in Rust.

PACKAGE CONTENTS
    string_sum     <-------------- what is it? let's check

FUNCTIONS
    sum_as_string(...)
        Formats the sum of two numbers as string.

DATA
    __all__ = ['sum_as_string']

FILE
    .\string_sum\.env\lib\site-packages\string_sum\__init__.py

>>> help(string_sum.string_sum)
Help on module string_sum.string_sum in string_sum:
                           ^^^--- duplicated name

NAME
    string_sum.string_sum - A Python module implemented in Rust.
    ^^^--- duplicated name

FUNCTIONS
    sum_as_string(...)
        Formats the sum of two numbers as string.

DATA
    __all__ = ['sum_as_string']

FILE
    .\string_sum\.env\lib\site-packages\string_sum\string_sum.pyd

>>> help(string_sum.sum_as_string)
Help on built-in function sum_as_string in module string_sum.string_sum:    <--- duplicated name

sum_as_string(...)
    Formats the sum of two numbers as string.

Let's check what docs can be generated with pdoc3:

$ pip install pdoc3
$ pdoc3 string_sum
Module string_sum
=================
A Python module implemented in Rust.

Sub-modules
-----------
* string_sum.string_sum    <--- unwanted submodule

Functions
---------


`sum_as_string(...)`
:   Formats the sum of two numbers as string.

Backtrace

No response

Your operating system and version

Windows 10

Your Python version (python --version)

Python 3.9.0

Your Rust version (rustc --version)

rustc 1.63.0 (4b91a6ea7 2022-08-08)

Your PyO3 version

0.16.6

How did you install python? Did you use a virtualenv?

Manually installed python from python.org. Yes, venv was used in the example above.

Additional Info

I searched for similar issues but didn't find anything.

[davidhewitt]

I've transferred this issue to maturin, which controls the build process. I think there are similar discussions on this topic already but I cannot recall their conclusion.

[messense]

It's by design to unify pure Rust project layout and mixed Rust/Python project layout, see #558

I don't think it breaks any documentation generators, they're correctly documenting it, it's just as a user you prefer not see this implementation detail, right?

[messense]

You can try add del {module_name} after L655 to

maturin/src/module_writer.rs

Lines 648 to 658 in 8f371ff

	writer.add_bytes(
	&module.join("__init__.py"),
	format!(
	r#"from .{module_name} import *
	__doc__ = {module_name}.__doc__
	if hasattr({module_name}, "__all__"):
	__all__ = {module_name}.__all__"#,
	module_name = module_name
	)
	.as_bytes(),

If it works, PR is welcome, but it is a breaking change so it will be shipped in the next minor release, actually since it won't affect existing projects so it's fine to be shipped in a patch release.

[Smertig]

I don't think it breaks any documentation generators, they're correctly documenting it, it's just as a user you prefer not see this implementation detail, right?

You are right. This is purely visual «breakage».

You can try add del {module_name} after L655 to

I tried this; however,

PACKAGE CONTENTS
    string_sum

still there for help(string_sum), but I can't access it with string_sum.string_sum ¯\_(ツ)_/¯

Also, help(string_sum.sum_as_string) still shows me

Help on built-in function sum_as_string in module string_sum.string_sum:
                                                  ^^^^^^^^^^^^^^^^^^^^^
sum_as_string(...)
    Formats the sum of two numbers as string.

Don't know how to deal with it.

[messense]

I'm not sure either, @konstin what do you think?

[konstin]

I think this is a ux bug in python's help(), as we can remove entries from dir() but they still show up in help(). If i use the original pdoc, it does look correct: