Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Windows trampolines for compatibility with Python 3.7 and earlier #8649

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion crates/uv-install-wheel/src/wheel.rs
Original file line number Diff line number Diff line change
Expand Up @@ -227,14 +227,14 @@ pub(crate) fn windows_script_launcher(

let mut launcher: Vec<u8> = Vec::with_capacity(launcher_bin.len() + payload.len());
launcher.extend_from_slice(launcher_bin);
launcher.extend_from_slice(&payload);
launcher.extend_from_slice(python_path.as_bytes());
launcher.extend_from_slice(
&u32::try_from(python_path.as_bytes().len())
.expect("File Path to be smaller than 4GB")
.to_le_bytes(),
);
launcher.extend_from_slice(&LAUNCHER_MAGIC_NUMBER);
launcher.extend_from_slice(&payload);

Ok(launcher)
}
Expand Down
11 changes: 6 additions & 5 deletions crates/uv-trampoline/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,19 +88,20 @@ Basically, this looks up `python.exe` (for console programs) and invokes

The intended use is:

- take your Python script, name it `__main__.py`, and pack it into a `.zip` file. Then concatenate
that `.zip` file onto the end of one of our prebuilt `.exe`s.
- After the zip file content, write the path to the Python executable that the script uses to run
- First, place our prebuilt `.exe` content at the top of the file.
- After the exe file content, write the path to the Python executable that the script uses to run
the Python script as UTF-8 encoded string, followed by the path's length as a 32-bit little-endian
integer.
- At the very end, write the magic number `UVUV` in bytes.
- Write the magic number `UVUV` in bytes.
- Finally, rename your Python script as `__main__.py`, compress it into a `.zip` file, and append
this `.zip` file to the end of one of our prebuilt `.exe` files.

| `launcher.exe` |
| :-------------------------: |
| `<zipped python script>` |
| `<path to python.exe>` |
| `<len(path to python.exe)>` |
| `<b'U', b'V', b'U', b'V'>` |
| `<zipped python script>` |

Then when you run `python` on the `.exe`, it will see the `.zip` trailer at the end of the `.exe`,
and automagically look inside to find and execute `__main__.py`. Easy-peasy.
Expand Down
48 changes: 44 additions & 4 deletions crates/uv-trampoline/src/bounce.rs
Original file line number Diff line number Diff line change
Expand Up @@ -87,11 +87,12 @@ fn push_quoted_path(path: &Path, command: &mut Vec<u8>) {
}

/// Reads the executable binary from the back to find the path to the Python executable that is written
/// after the ZIP file content.
/// before the ZIP file content.
///
/// The executable is expected to have the following format:
/// * The file must end with the magic number 'UVUV'.
/// * The last 4 bytes (little endian) are the length of the path to the Python executable.
/// * The last part of the file is ZIP file content.
/// * The remain part must end with the magic number 'UVUV'.
/// * The last 4 bytes (little endian) before 'UVUV' are the length of the path to the Python executable.
/// * The path encoded as UTF-8 comes right before the length
///
/// # Panics
Expand All @@ -112,6 +113,45 @@ fn find_python_exe(executable_name: &Path) -> PathBuf {
});
let file_size = metadata.len();

// Read the entire end of central directory (EOCD) of the ZIP file, which is 22 bytes long.
let mut eocd_buf: Vec<u8> = vec![0; 22];

file_handle
.seek(SeekFrom::Start(file_size - 22))
.unwrap_or_else(|_| {
print_last_error_and_exit("Failed to set the file pointer to the start of zip EOCD");
});

let read_bytes = file_handle.read(&mut eocd_buf).unwrap_or_else(|_| {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

print_last_error_and_exit("Failed to read the zip EOCD");
});

if read_bytes != 22 {
eprintln!(
"Failed to read the EOCD. Expected 22 bytes but read {}",
read_bytes
);
exit_with_status(1);
}

// Size of the central directory (in bytes)
let cd_size = u32::from_le_bytes(eocd_buf[12..16].try_into().unwrap_or_else(|_| {
eprintln!("Slice length is not equal to 4 bytes");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be unreachable!()

exit_with_status(1);
}));

// Offset of central directory (unit is bytes).
// In other words, the number of bytes in the ZIP file at which the central directory starts.
let cd_offset = u32::from_le_bytes(eocd_buf[16..20].try_into().unwrap_or_else(|_| {
eprintln!("Slice length is not equal to 4 bytes");
exit_with_status(1);
}));

// Calculate the position in the entire executable where the content of the ZIP file begins.
let end_of_cd = file_size - 22;
let start_of_cd = end_of_cd - cd_size as u64;
let start_of_zip = start_of_cd - cd_offset as u64;

// Start with a size of 1024 bytes which should be enough for most paths but avoids reading the
// entire file.
let mut buffer: Vec<u8> = Vec::new();
Expand All @@ -122,7 +162,7 @@ fn find_python_exe(executable_name: &Path) -> PathBuf {
buffer.resize(bytes_to_read as usize, 0);

file_handle
.seek(SeekFrom::Start(file_size - u64::from(bytes_to_read)))
.seek(SeekFrom::Start(start_of_zip - u64::from(bytes_to_read)))
.unwrap_or_else(|_| {
print_last_error_and_exit("Failed to set the file pointer to the end of the file");
});
Expand Down
2 changes: 1 addition & 1 deletion crates/uv-trampoline/tests/harness.rs
Original file line number Diff line number Diff line change
Expand Up @@ -175,14 +175,14 @@ fn windows_script_launcher(

let mut launcher: Vec<u8> = Vec::with_capacity(launcher_bin.len() + payload.len());
launcher.extend_from_slice(launcher_bin);
launcher.extend_from_slice(&payload);
launcher.extend_from_slice(python_path.as_bytes());
launcher.extend_from_slice(
&u32::try_from(python_path.as_bytes().len())
.expect("File Path to be smaller than 4GB")
.to_le_bytes(),
);
launcher.extend_from_slice(&LAUNCHER_MAGIC_NUMBER);
launcher.extend_from_slice(&payload);

Ok(launcher)
}
Expand Down
Binary file modified crates/uv-trampoline/trampolines/uv-trampoline-aarch64-console.exe
Binary file not shown.
Binary file modified crates/uv-trampoline/trampolines/uv-trampoline-aarch64-gui.exe
Binary file not shown.
Binary file modified crates/uv-trampoline/trampolines/uv-trampoline-i686-console.exe
Binary file not shown.
Binary file modified crates/uv-trampoline/trampolines/uv-trampoline-i686-gui.exe
Binary file not shown.
Binary file not shown.
Binary file not shown.
57 changes: 57 additions & 0 deletions crates/uv/tests/it/tool_install.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3112,3 +3112,60 @@ fn tool_install_at_latest_upgrade() {
"###);
});
}

#[cfg(windows)]
#[test]
fn tool_install_windows_3_7() {
let context = TestContext::new("3.7")
.with_filtered_counts()
.with_filtered_exe_suffix();
let tool_dir = context.temp_dir.child("tools");
let bin_dir = context.temp_dir.child("bin");

// Install `black`
uv_snapshot!(context.filters(), context.tool_install()
.arg("black")
.env(EnvVars::UV_TOOL_DIR, tool_dir.as_os_str())
.env(EnvVars::XDG_BIN_HOME, bin_dir.as_os_str())
.env(EnvVars::PATH, bin_dir.as_os_str()), @r###"
success: true
exit_code: 0
----- stdout -----

----- stderr -----
Resolved [N] packages in [TIME]
Prepared [N] packages in [TIME]
Installed [N] packages in [TIME]
+ black==23.3.0
+ click==8.1.7
+ importlib-metadata==6.7.0
+ mypy-extensions==1.0.0
+ packaging==24.0
+ pathspec==0.11.2
+ platformdirs==4.0.0
+ tomli==2.0.1
+ typed-ast==1.5.5
+ typing-extensions==4.7.1
+ zipp==3.15.0
Installed 2 executables: black, blackd
"###);

tool_dir.child("black").assert(predicate::path::is_dir());
tool_dir
.child("black")
.child("uv-receipt.toml")
.assert(predicate::path::exists());

let executable = bin_dir.child(format!("black{}", std::env::consts::EXE_SUFFIX));
assert!(executable.exists());

uv_snapshot!(context.filters(), Command::new("black").arg("--version").env(EnvVars::PATH, bin_dir.as_os_str()), @r###"
success: true
exit_code: 0
----- stdout -----
black, 23.3.0 (compiled: yes)
Python (CPython) 3.7.[X]

----- stderr -----
"###);
}
Loading