Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

regression: pixi tries to GET pypi.org/simple in offline environment #2434

Open
2 tasks done
moritzwilksch opened this issue Nov 7, 2024 · 3 comments
Open
2 tasks done
Labels
🐍 pypi Issue related to PyPI dependencies UX Related to the User Experience of pixi 👩‍💻 cli Issue related to CLI

Comments

@moritzwilksch
Copy link

moritzwilksch commented Nov 7, 2024

Checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

# pixi.toml
[project]
authors = ["Moritz Wilksch <moritz.wilksch@email.de>"]
channels = [
  "https://artifactory.internal-stuff.com/artifactory/conda-forge-mirror",
]
description = "Add a short description here"
name = "pixi-mre"
platforms = ["linux-64"]
version = "0.1.0"

conda-pypi-map = {}

[tasks]

[dependencies]
python = ">=3.13.0,<3.14"
setuptools = ">=75.3.0,<76"

[pypi-options]
no-build-isolation = ['mre']

[pypi-dependencies]
mre = { path = ".", editable = true }
# pyproject.toml
[project]
name = 'mre'
version = '0.1.0'

Issue description

pixi tries to GET pypi.org/simple in an offline environment even though the pypi-conda-map is set to {}. This setup used to work in pixi=0.34.0 but is broken with 0.35.0. I also tried overriding the find-links option, but that didn't help or is there a recommended override value?

➜  pixi-mre pixi i -vvvv
DEBUG pixi_config: Loading config from /etc/pixi/config.toml
 INFO pixi_config: Loaded config from: /etc/pixi/config.toml
DEBUG pixi_config: Loading config from /home/USERNAME/.config/pixi/config.toml
 INFO pixi_config: Loaded config from: /home/USERNAME/.config/pixi/config.toml
DEBUG pixi_config: Loading config from /home/USERNAME/.config/pixi/config.toml
 INFO pixi_config: Loaded config from: /home/USERNAME/.config/pixi/config.toml
DEBUG pixi_config: Loading config from /home/USERNAME/.pixi/config.toml
DEBUG pixi_config: Failed to load global config: /home/USERNAME/.pixi/config.toml (error: failed to read config from '/home/USERNAME/.pixi/config.toml')
DEBUG pixi_config: Loading config from /home/USERNAME/Documents/pixi-mre/.pixi/config.toml
DEBUG pixi_config: Failed to load local config: /home/USERNAME/Documents/pixi-mre/.pixi/config.toml (error: failed to read config from '/home/USERNAME/Documents/pixi-mre/.
pixi/config.toml')
DEBUG pixi::project: Symlink already exists at '/home/USERNAME/Documents/pixi-mre/.pixi/envs', skipping creating symlink.
 INFO pixi::environment: verifying prefix location is unchanged, with prefix file: /opt/pixi/envs/pixi-mre-14179212012911116289/envs/default/conda-meta/pixi_env_prefix
 INFO pixi::lock_file::update: the lock-file is up-to-date
DEBUG pixi::environment: Wrote environment file to: "/opt/pixi/envs/pixi-mre-14179212012911116289/envs/default/conda-meta/pixi"
DEBUG pixi::rlimit: Attempted to set RLIMIT_NOFILE to 1024 but was already set to 65535
 INFO pixi::environment: Creating prefix file at: /opt/pixi/envs/pixi-mre-14179212012911116289/envs/default/conda-meta/pixi_env_prefix
 INFO pixi::environment: No update needed for the prefix file.
 INFO pixi::environment: Checking if history file exists: /opt/pixi/envs/pixi-mre-14179212012911116289/envs/default/conda-meta/history
 INFO pixi::lock_file::resolve::uv_resolution_context: uv keyring provider is disabled
DEBUG uv_client::base_client: Using request timeout of 30s
TRACE get_cacheable:read_and_parse_cache{file=/tmp/USERNAME/cache/rattler/cache/uv-cache/flat-index-v1/html/b2a7eb67d4c26b82.msgpack}: uv_client::cached_client: No cache
entry exists for /tmp/USERNAME/cache/rattler/cache/uv-cache/flat-index-v1/html/b2a7eb67d4c26b82.msgpack
DEBUG get_cacheable: uv_client::cached_client: No cache entry for: https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_client::cached_client: Sending fresh GET request for https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Handling request for https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Request for https://pypi.org/simple is unauthenticated, checking cache
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::cache: No credentials in cache for URL https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Attempting unauthenticated request for https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::pool: checkout waiting for idle connection: ("https", pypi.org)
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: starting new connection: https://pypi.org/
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: proxy(http://MYPROXY:3128) intercepts 'https://pypi.org/'
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: Http::connect; scheme=Some("http"), host=Some("IP
2"), port=Some(Port(3128))
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: connecting to MYPROXY:3128
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: connected to MYPROXY:3128
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: tunneling HTTPS over proxy
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::pool: checkout dropped for ("https", pypi.org)
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_client::base_client: Transient request failure for https://pypi.org/simple, retrying: error sending
request for url (https://pypi.org/simple)
  Caused by: client error (Connect)
  Caused by: unsuccessful tunnel
 WARN get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest_retry::middleware: Retry attempt #0. Sleeping 373.638361ms before the next attempt
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Handling request for https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Request for https://pypi.org/simple is unauthenticated, checking cache
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::cache: No credentials in cache for URL https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Attempting unauthenticated request for https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::pool: checkout waiting for idle connection: ("https", pypi.org)
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: starting new connection: https://pypi.org/
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: proxy(http://MYPROXY:3128) intercepts 'https://pypi.org/'
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: Http::connect; scheme=Some("http"), host=Some("IP
2"), port=Some(Port(3128))
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: connecting to MYPROXY:3128
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: connected to MYPROXY:3128
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: tunneling HTTPS over proxy
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::pool: checkout dropped for ("https", pypi.org)
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_client::base_client: Transient request failure for https://pypi.org/simple, retrying: error sending
request for url (https://pypi.org/simple)
  Caused by: client error (Connect)
  Caused by: unsuccessful tunnel
 WARN get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest_retry::middleware: Retry attempt #1. Sleeping 245.370932ms before the next attempt
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Handling request for https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Request for https://pypi.org/simple is unauthenticated, checking cache
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::cache: No credentials in cache for URL https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Attempting unauthenticated request for https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::pool: checkout waiting for idle connection: ("https", pypi.org)
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: starting new connection: https://pypi.org/
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: proxy(http://MYPROXY:3128) intercepts 'https://pypi.org/'
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: Http::connect; scheme=Some("http"), host=Some("IP
2"), port=Some(Port(3128))
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: connecting to MYPROXY:3128
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: connected to MYPROXY:3128
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: tunneling HTTPS over proxy
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::pool: checkout dropped for ("https", pypi.org)
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_client::base_client: Transient request failure for https://pypi.org/simple, retrying: error sending
request for url (https://pypi.org/simple)
  Caused by: client error (Connect)
  Caused by: unsuccessful tunnel
 WARN get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest_retry::middleware: Retry attempt #2. Sleeping 1.64212909s before the next attempt
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Handling request for https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Request for https://pypi.org/simple is unauthenticated, checking cache
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::cache: No credentials in cache for URL https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_auth::middleware: Attempting unauthenticated request for https://pypi.org/simple
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::pool: checkout waiting for idle connection: ("https", pypi.org)
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: starting new connection: https://pypi.org/
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: proxy(http://MYPROXY:3128) intercepts 'https://pypi.org/'
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: Http::connect; scheme=Some("http"), host=Some("IP
2"), port=Some(Port(3128))
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: connecting to MYPROXY:3128
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::connect::http: connected to MYPROXY:3128
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: reqwest::connect: tunneling HTTPS over proxy
TRACE get_cacheable:fresh_request{url="https://pypi.org/simple"}: hyper_util::client::legacy::pool: checkout dropped for ("https", pypi.org)
DEBUG get_cacheable:fresh_request{url="https://pypi.org/simple"}: uv_client::base_client: Transient request failure for https://pypi.org/simple, retrying: error sending
request for url (https://pypi.org/simple)
  Caused by: client error (Connect)
  Caused by: unsuccessful tunnel
  × default: error installing/updating PyPI dependencies
  ├─▶ Failed to read `--find-links` URL: https://pypi.org/simple
  ├─▶ Failed to fetch: `https://pypi.org/simple`
  ├─▶ Request failed after 3 retries
  ├─▶ error sending request for url (https://pypi.org/simple)
  ├─▶ client error (Connect)
  ╰─▶ unsuccessful tunnel

pixi=0.34.0 works:

➜  pixi-mre pixi exec -s pixi=0.34.0 -- pixi install
✔ The default environment has been installed in '/opt/pixi/envs'.

Expected behavior

Similar to 0.34.0 I can prevent the attempted internet access by setting the conda-pypi-map to empty.

@moritzwilksch
Copy link
Author

maybe related to #2417?

@wolfv
Copy link
Member

wolfv commented Nov 7, 2024

Hmm, I am unsure if I understand entirely. Is this a problem when you already have a lockfile or is this for a completely fresh project? Does mre have any additional dependencies or not?

The pypi-conda mapping doesn't really tie in with accessing pypi.org. Those are separate things.
It's "somewhere" in uv where the pypi index is accessed.

We could potentially add an option to the PyPI options in the pixi / pyproject toml to influence this variable:

.connectivity(Connectivity::Online)
.build(),

Setting that to offline should make things work for your usecasee.

@moritzwilksch
Copy link
Author

Yes, this is a completely fresh project without an existing lock file and no additional dependencies. Basically, the editable install of my local package now requires internet access and didn't before. Exposing an "offline" option would be great!
You're right, it might be a change in uv itself 🤔

The conda-pypi-map has always been a necessary configuration to work in offline envs because otherwise pixi attempts fetching the map from a prefix.dev server. But you're right in that it shouldn't be related to the pypi issue.

@tdejager tdejager added 🐍 pypi Issue related to PyPI dependencies 👩‍💻 cli Issue related to CLI UX Related to the User Experience of pixi labels Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐍 pypi Issue related to PyPI dependencies UX Related to the User Experience of pixi 👩‍💻 cli Issue related to CLI
Projects
None yet
Development

No branches or pull requests

3 participants