Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Installation for Triton #1410

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Installation for Triton #1410

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions bin/yaml/libraries.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,12 @@
libraries:
triton:
triton-shared:
type: tarballs
dir: triton-shared-{name}
compression: gz
url: https://github.com/microsoft/triton-shared/archive/refs/heads/main.tar.gz
targets:
- 'main'
android-java:
android-api-stubs:
check_file: android.jar
Expand Down
16 changes: 16 additions & 0 deletions bin/yaml/tools.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,22 @@ tools:
check_exe: bin/osaca --version
targets:
- 0.5.2
wheel:
type: pip
dir: wheel-{name}
python: /usr/bin/python3
package: wheel=={name}
check_exe: wheel version
targets:
- 0.43.0
pybind11:
type: pip
dir: pybind11-{name}
python: /usr/bin/python3
package: pybind11=={name}
check_exe: bin/pybind11-config --version
targets:
- 2.13.5
rustfmt:
type: tarballs
dir: rustfmt-{name}
Expand Down
22 changes: 22 additions & 0 deletions bin/yaml/triton.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
compilers:
triton:
type: tarballs
dir: triton-{name}
depends:
- tools/cmake 3.29.2
- tools/ninja 1.11.1
- tools/wheel 0.43.0
- tools/pybind11 2.13.5
- compilers/c++/clang 18.1.0
- libraries/triton/triton-shared main
compression: gz
check_exe: python3 venv/lib/python3.12/site-packages/triton/tools/compile.py -h
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably the culprit here. We don't have a "real" python3 in our environment where the compilers run and/or are installed. As you know python has a rich history of complex package management systems and we don't currently well support compilres that are written in python.

The computer that creates the venv will not be the same one that executes the compilers, so a lot of system dependencies could change.

Unfortunately we don't have a great solution for this currently.

If you have Discord you could dro pinto our discord server for a chat about this, we're open to ideas. But python's installation system is a little hostile to our current setup as it stands.

url: https://github.com/triton-lang/triton/archive/refs/tags/v{name}.tar.gz
targets:
- 2.1.0
after_stage_script:
- cd triton-{name}
- python3 -m venv venv
- source venv/bin/activate
- cd python
- TRITON_BUILD_WITH_CLANG_LLD=true TRITON_PLUGIN_DIRS=/opt/compiler-explorer/triton-shared-main pip install -e .
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will build on our "admin" installer node, which isn't a great place for this to go. In other languages we try and build a "hermetic", relocatable tarball that we use as the source of truth.

From (bitter) experience in my day job, python and its various virtual environments aren't great at this kind of thing. pip and poetry don't handle native dependencies at all, conda is not relocatable.

Also hardcoding /opt/compiler-explorer here is not a great idea as we support other installation locations (though pragmatically this will work for the CE main install)

Loading