Releases: dstackai/dstack
0.18.20
Python 3.13 support
Following a recent Python 3.13 release on October 7, 2024, dstack
now supports python: 3.13
in run configurations. python: 3.8
is still supported but deprecated.
Note: the dstack
package itself does not yet work on Python 3.13 due to some limitations in dependencies. We're looking into supporting it as well.
Custom backend tags
You can now define custom tags that dstack
will assign to all cloud resources it creates including instances and volumes. The tags are defined in the backend configuration:
type: aws
tags:
company_department: finance
company_project: dstack
company_user: victor
creds:
type: default
Custom tags are supported for AWS, Azure, and GCP.
Improved support of AWS private subnets
Previously, when configuring an AWS backend to use private subnets (public_ips: false
), dstack
would require a NAT Gateway. Now dstack
supports more networking setups that provide outbound internet traffic including NAT Gateway, Transit Gateway, and VPC Peering Connection.
New required permissions
dstack
now sets labels on GCP volumes which requires acompute.disks.setLabels
permission.
Deprecations
python: 3.8
in run configurations is deprecated.
What's Changed
- Add created_at to projects and users by @r4victor in #1857
- Improvements for model details page in the UI by @olgenn in #1860
- [Bug]: Users logged out after rotating their tokens without seeing tokens by @olgenn in #1861
- [dind] Move dind processes to a separate cgroup by @un-def in #1859
- [Docs] Add Docker protip and Docker Compose example by @un-def in #1858
- Implement custom backend tags by @r4victor in #1872
- [shim] Remove anonymous volumes along associated container by @un-def in #1873
- Allow running services without a gateway by @jvstme in #1869
- [UX]: Resize chat input field based on content #1562 by @olgenn in #1875
- Collect AMD GPU metrics by @r4victor in #1877
- [Blog] Monitoring GPU usage and other container metrics by @peterschmidt85 in #1874
- [Docs] Rename
HUGGING_FACE_HUB_TOKEN
toHF_TOKEN
by @peterschmidt85 in #1871 - Support Python 3.13 and deprecate 3.8 in run configurations by @jvstme in #1878
- Support AWS private subnets with Transit Gateway by @r4victor in #1881
- Fix collecting metrics from CPU instances by @r4victor in #1882
Full Changelog: 0.18.19...0.18.20
0.18.19
This release contains CLI hotfixes for 0.18.18
, including a fix for client backward compatibility and a fix for reported memory usage in dstack stats
. It's recommended to update the CLI from 0.18.18
to 0.18.19
. The server update from 0.18.18
is not necessary.
What's Changed
- Fix Trl example by @Bihan in #1851
- Fix dstack client 0.18.18 compatibility with older servers by @un-def in #1850
- [Docs]: Fix frontend build instructions by @jvstme in #1849
- Show working set memory in dstack stats by @r4victor in #1856
Full Changelog: 0.18.18...0.18.19
0.18.18
Hardware metrics
The CLI introduces a new command, dstack stats
, which displays real-time hardware metrics for runs, including CPU,
memory, and GPU usage per replica and job.
$ dstack stats hot-frog-1
NAME CPU MEMORY GPU
hot-frog-1 2% 15307MB/49152MB #0 22764MB/24576MB 0% Util
Use the -w
option to view stats updating every few seconds in the loop.
You can also retrieve the metrics using the REST API.
Docker inside dstack
Run configurations now have a new optional privileged
property (equivalent to the
--privileged
Docker CLI flag).
When it is set to true
, the run container gets extended privileges, making it possible to use Docker inside
dstack
.
To use Docker and Docker Compose within dstack
, set the image
property
to dstackai/dind
. Additionally, you must invoke start-dockerd
as the first command to start the Docker daemon.
type: task
name: misc-task-dind
image: dstackai/dind
privileged: true
commands:
- start-dockerd
- docker compose up
Dev environment example
type: dev-environment
name: vscode-dind
image: dstackai/dind
privileged: true
ide: vscode
init:
- start-dockerd
See more examples in examples/misc/docker-compose
.
Note
The privileged
property is only supported by VM backends (all backends except runpod
, vastai
, and kubernetes
).
What's changed
- [Feature] Track hardware metrics by @r4victor in #1827
- [Feature] Add
privileged
run property by @un-def in #1835 - [Feature] Run container as
--privileged
ifDSTACK_DOCKER_PRIVILEGED
by @mtaran in #1821 - [Feature] Added the
Code
button on the model page in UI by @olgenn in #1825 - [Feature] Publish
dstackai/dind
image by @un-def in #1837 - [Bugfix] Fix running NVIDIA NIM images by @jvstme in #1843
- [Bugfix] Mitigate concurrent
dstack attach
issues by @un-def in #1816 - [Bugfix] Fixed runs ordering and pagination in UI by @un-def in #1812
- [Bugfix] Removed dummy Test fleet in UI by @priyanshuverma-dev in #1819
- [UX] Display the SSH and vscode prompt to after executing
init
commands by @fool1280 in #1818 - [Blog] AMD MI300X inference benchmark by @peterschmidt85 and @Bihan in #1808
- [Docs] Add AWS EFA support info by @un-def in #1803
- [Docs] Fixed typos and added minor enhancements by @FarukhS52 in #1810 and #1811
- [Docs] Fixed typos and added minor enhancements by @amantyagiprojects in #1800
- [Docs] Use correct discord server invite in readme by @sravan1946 in #1824
- [Docs] Fixed a minor issue in
README.md
by @Hacker24a9 in #1823 - [Docs] Fixed typos and added minor enhancements @VaibhavWakde52 in #1813
- [UX] Show run error on the run page in the UI by @olgenn in #1829
- [UX] Normalize AMD Instinct accelerator names by @un-def in #1817
- [Bugfix] Fixed the issue with attaching volume to existing instance by @un-def in #1836
- [Bugfix] Fix GCP config
public_ips
not allowed via the API by @r4victor in #1840 - [Internal] Don't cache
GpuVendor
inglobal var
by @un-def in #1845 - [Internal] Fix shim tests binding to 0 port by @r4victor in #1846
New contributors
- @FarukhS52 made their first contribution in #1810
- @amantyagiprojects made their first contribution in #1800
- @priyanshuverma-dev made their first contribution in #1819
- @sravan1946 made their first contribution in #1824
- @Hacker24a9 made their first contribution in #1823
- @VaibhavWakde52 made their first contribution in #1813
- @fool1280 made their first contribution in #1818
- @mtaran made their first contribution in #1821
Full changelog: 0.18.17...0.18.18
0.18.17
On-prem AMD GPU support
dstack
now supports SSH fleets with AMD GPUs. Hosts should be pre-installed with Docker and AMDGPU-DKMS kernel driver (e.g. via native package manager or AMDGPU installer).
Elastic Fabric Adapter support
dstack
now automatically enables AWS EFA if it is supported by the instance type, no extra configuration needed. The following EFA-enabled instance types are supported: p5.48xlarge
, p4d.24xlarge
, g4dn.12xlarge
, g4dn.16xlarge
, g4dn.8xlarge
, g4dn.metal
, g5.12xlarge
, g5.16xlarge
, g5.24xlarge
, g5.48xlarge
, g5.8xlarge
, g6.12xlarge
, g6.16xlarge
, g6.24xlarge
, g6.48xlarge
, g6.8xlarge
, gr6.8xlarge
.
Improved apply plan
Previously, dstack apply
showed a plan only for run configurations. Now it shows a plan for all configuration types including fleets, volumes, and gateways. Here's a fleet plan showing configuration parameters and the offers that will be tried for provisioning:
✗ dstack apply -f .dstack/confs/fleet.yaml
Project main
User admin
Configuration .dstack/confs/fleet.yaml
Type fleet
Fleet type cloud
Nodes 2
Placement cluster
Backends aws
Resources 2..xCPU, 8GB.., 100GB.. (disk)
Spot policy on-demand
# BACKEND REGION INSTANCE RESOURCES SPOT PRICE
1 aws eu-west-1 m5.large 2xCPU, 8GB, 100.0GB (disk) no $0.107
2 aws eu-central-1 m5.large 2xCPU, 8GB, 100.0GB (disk) no $0.115
3 aws eu-west-1 c5.xlarge 4xCPU, 8GB, 100.0GB (disk) no $0.192
...
Shown 3 of 82 offers, $40.9447 max
Fleet my-cluster-fleet does not exist yet.
Create the fleet? [y/n]:
Volumes UI
Server administrators and regular users can now see volumes in the UI.
What's Changed
- Dstack version on UI by @olgenn in #1742
- Fix restarting gateway connections by @jvstme in #1746
- Fix Handle KeyboardInterrupt in CLI when getting run plan #1626 by @IshuSinghSE in #1756
- Add AMD support on on-prem fleets by @un-def in #1754
- Implement fleet apply plan by @r4victor in #1765
- chore: update provisioning.py by @eltociear in #1768
- Fix use all available runpod regions by default by @IshuSinghSE in #1757
- Implement apply plan for gateways and volumes by @r4victor in #1774
- Fix connection to ssh instance on non-standard ssh port by @un-def in #1766
- Fix docker SSH commands by @un-def in #1771
- Add Llama3.2 Vision Model Example by @Bihan in #1770
- Disable backend autoconfig via default creds by @r4victor in #1778
- Set backends requests timeouts by @r4victor in #1793
- Add UI for volumes #1683 by @olgenn in #1785
- UI for volumes 1683 by @olgenn in #1795
- [Docs] Add AMD GPU info to ssh fleets section by @un-def in #1779
- [shim] Use DockerRootDir to detect free disk space by @un-def in #1802
- Add AWS EFA support by @un-def in #1801
New Contributors
- @IshuSinghSE made their first contribution in #1756
- @eltociear made their first contribution in #1768
Full Changelog: 0.18.16...0.18.17
0.18.16
New versioning policy
Starting with this release, dstack
adopts a new versioning policy to provide better server and client backward compatibility and improve the upgrading experience. dstack
continues to follow semver versioning scheme ({major}.{minor}.{patch}
) with the following principles:
- The server backward compatibility is maintained across all minor and patch releases. The specific features can be removed but the removal is preceded with deprecation warnings for several minor releases. This means you can use older client versions with newer server versions.
- The client backward compatibility is maintained across patch releases. A new minor release indicates that the release breaks client backward compatibility. This means you don't need to update the server when you update the client to a new patch release. Still, upgrading a client to a new minor version requires upgrading the server too.
Perviously, dstack
never guaranteed client backward compatibility, so you had to always update the server when updating the client. The new versioning policy makes the client and server upgrading more flexible.
Note: The new policy only takes affect after both the clients and the server are upgraded to 0.18.16
. The 0.18.15
server still won't work with newer clients.
dstack attach
The CLI gets a new dstack attach
command that allows attaching to a run. It establishes the SSH tunnel, forwards ports, and streams run logs in real time:
✗ dstack attach silent-panther-1
Attached to run silent-panther-1 (replica=0 job=0)
Forwarded ports (local -> remote):
- localhost:7860 -> 7860
To connect to the run via SSH, use `ssh silent-panther-1`.
Press Ctrl+C to detach...
This command is a replacement for dstack logs --attach
with major improvements and bugfixes.
CloudWatch-related bugfixes
The releases includes several important bugfixes for CloudWatchLogStorage
. We strongly recommend upgrading the dstack
server if it's configured to store logs in CloudWatch.
Deprecations
dstack logs --attach
is deprecated in favor ofdstack attach
and may be removed in the following minor releases.
What's Changed
- Check client-server compatibility according to new versioning policy by @r4victor in #1730
- [runner] fix MonotonicTimestamp by @un-def in #1728
- Gateway-in-server early prototype by @jvstme in #1718
- Implement dstack attach command by @r4victor in #1733
- Respect CloudWatch timestamp constraints by @un-def in #1732
- Add AMD examples with vLLM, Axolotl and Trl by @Bihan in #1693
- dstack-proxy naming tweaks by @jvstme in #1734
- Fix Failed to attach via Python API by @r4victor in #1739
- Support calling RunCollection.get_plan() without repo by @r4victor in #1741
Full Changelog: 0.18.15...0.18.16
0.18.15
Cluster placement groups
Instances of AWS cluster fleets are now provisioned into cluster placement groups for better connectivity. For example, when you create this fleet:
type: fleet
name: my-cluster-fleet
nodes: 4
placement: cluster
backends: [aws]
dstack
will automatically create a cluster placement group and use it to provision the instances.
On-prem and VM-based fleets improvements
- All available Nvidia driver capabilities are now requested by default, which makes it possible to run GPU workloads requiring OpenGL/Vulkan/RT/Video Codec SDK libraries. (#1714)
- Automatic container cleanup. Previously, when the run completed, either successfully or due to an error, its container was not deleted, which led to ever-increasing storage consumption. Now, only the last stopped container is preserved and is available until the next run is completed. (#1706)
Major bug fixes
- Fixed a bug where under some conditions logs wouldn't be uploaded to CloudWatch Logs due to size limits. (#1712)
- Fixed a bug that prevented running services on on-prem instances. (#1716)
Changelog
- Fix cli connection issue with TPU by @Bihan in #1705
- Rename
--default
to--yes
andno-default
to--no
indstack config
anddstack server
by @peterschmidt85 in #1709 - [CI] Fix shim/runner release versions by @un-def in #1704
- Document run diagnostic logs by @r4victor in #1710
- [shim] Add old container cleanup routine by @un-def in #1706
- Write events to CloudWatch in batches by @un-def in #1712
- [shim] Request all Nvidia driver capabilities by @un-def in #1714
- Added showing dstack version on the UI by @olgenn in #1717
- Add missing project SSH key to on-prem instances by @un-def in #1716
- Simplify handling missing
GatewayConfiguration
by @jvstme in #1724 - [shim] Fix container logs processing by @un-def in #1721
- Support AWS placement groups for cluster fleets by @r4victor in #1725
Full Changelog: 0.18.14...0.18.15
0.18.15rc1
On-prem and VM-based fleets improvements
- All available Nvidia driver capabilities are now requested by default, which makes it possible to run GPU workloads requiring OpenGL/Vulkan/RT/Video Codec SDK libraries.
- Automatic container cleanup. Previously, when the run completed, either successfully or due to an error, its container was not deleted, which led to ever-increasing storage consumption. Now, only the last stopped container is preserved and is available until the next run is completed.
Major bug fixes
- Fixed a bug where under some conditions logs wouldn't be uploaded to CloudWatch Logs due to size limits.
Changelog
- [UX] Rename
--default
to--yes
and--no-default
to--no
indstack config
anddstack server
by @peterschmidt85 in #1709 - Fix cli connection issue with TPU by @Bihan in #1705
- Fix
dstack-shim
anddstack-runner
release versions by @un-def in #1704 - Request all Nvidia driver capabilities by @un-def in #1714
- Add old container cleanup routine by @un-def in #1706
- Write events to CloudWatch in batches by @un-def in #1712
- [Docs] Document run diagnostic logs by @r4victor in #1710
- [Docs] Added the
server deployment
guide, updated theREADME.md
for the Docker Hub, fixed the scrolling issue by @peterschmidt85
Full changelog: 0.18.14...0.18.15rc1
0.18.14
Multi-replica server deployment
Previously, the dstack
server only supported deploying a single instance (replica). However, with 0.18.14, you can now deploy multiple replicas, enabling high availability and zero-downtime updates
Note
Multi-replica server deployment requires using Postgres instead of the default SQLite. To configure Postgres, set the DSTACK_DATABASE_URL
environment variable.
Make sure to update to version 0.18.14 before configuring multiple replicas.
Major bug-fixes
- [Bugfix]
dstack init --git-identity
doesn't accept backslashes in path on Windows by @un-def in #1686 - [Bugfix] Use
-tmpfs /dev/shm:rw,nosuid,nodev,exec,size=X
instead of--shm-size=X
@un-def in #1690 - [Bugfix]
dstack-shim
is not updated when fleet is recreated by @un-def in #1698
Other
- [Bugfix] Fix
SSHAttach.reuse_ports_lock()
when no grep matches by @un-def in #1700 - [Bugfix] Fix logger exception on instance provisioning timeout by @un-def in #1697
- [Internal] Add
JobProvisioningData.base_backend
by @r4victor in #1682 - [Internal] Add
Run.error
by @r4victor in #1684 - [Internal] Return server_version in
/api/server/get_info
by @r4victor in #1685 - [Internal] Allow gateway to connect to replicated server by @jvstme in #1688
- [Internal] Adjust gateway management for multiple server replicas by @r4victor in #1691
- [Internal] Skip gateway update if gateway was updated recently by @r4victor in #1695
- [Internal] Remove redundant
logger.error
by @r4victor in #1702
Full changelog: 0.18.13...0.18.14
0.18.13
Windows
You can now use the CLI on Windows (WSL 2 is not required).
Ensure that Git and OpenSSH are installed via Git for Windows.
During installation, select Git from the command line and also from 3-rd party software
(or Use Git and optional Unix tools from the Command Prompt
), and Use bundled OpenSSH
checkboxes.
Spot policy
Previously, dev environments used the on-demand
spot policy, while tasks and services used auto
. With this update, we've changed the default spot policy to always be on-demand
for all configurations. Users will now need to explicitly specify the spot policy if they want to use spot instances.
Troubleshooting
The documentation now includes a Troubleshooting guide with instructions on how to report issues.
Changelog
- [UX] Add Windows support by @un-def in #1675
- [UX] Changed the default
spot_policy
toon-demand
by @r4victor in #1657 and #1660 - [UI] Minor UI improvements by @olgenn in #1658
- [UX] Check SSH keys when SSH fleet creation before submission by @r4victor in #1661
- [Docs] Add TPU examples with Optimum TPU and vLLM by @Bihan in #1663
- [Troubleshooting] Do not auto-delete failed instances by @r4victor in #1665
- [Docs] Document SQLite to Postgres migration by @r4victor in #1678
- [Internal] Implement Postgres locking by @r4victor in #1651
- [Internal] Refactor
SSHTunnel
by @jvstme in #1669 - [Internal] Replace
String
withText
for long database columns by @r4victor in #1677 - [Internal] Take advisory lock on server init by @r4victor in #1674
All commits: 0.18.12...0.18.13
0.18.12
Features
Major bugfixes
- Fixed the order of CloudWatch log events in the web interface by @un-def in #1613
- Fixed a bug where CloudWatch log events might not be displayed in the web inferface for old runs by @un-def in #1652
- Prevent possible server freeze on SSH connections by @jvstme in #1627
Other changes
- [CLI] Show run name before detaching by @jvstme in #1607
- Increase time waiting for OCI Bare Metal instances by @jvstme in #1630
- Update lambda regions by @r4victor in #1634
- Change CloudWatch group check method by @un-def in #1615
- Add Postgres tests by @r4victor in #1628
- Fix lambda tests by @r4victor in #1635
- [Docs] Fixed a bug where search included non-existing pages that land to 404 by @peterschmidt85 in #1646
- [Docs] Introduce the Providers page by @peterschmidt85 in #1653
- [Docs] Update RunPod & DataCrunch setup guides by @jvstme in #1608
- [Docs] Add information about run log storage by @un-def in #1621
- [Internal] Update packer templates docs by @jvstme in #1619
Full changelog: 0.18.11...0.18.12