Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[k8s] Show all kubernetes clusters in optimizer table #4013

Merged
merged 4 commits into from
Oct 29, 2024

Conversation

Michaelvll
Copy link
Collaborator

Since we support multiple k8s clusters, people may expect to see multiple k8s clusters in their allowed_contexts that have the available GPUs.

Two clusters:

  1. my-h100-cluster-gke: 1 H100 node, 1 L4 node
  2. my-h100-cluster-eks: 1 H100 node
sky launch --gpus H100 echo 'Hello World'
Task from command: echo Hello World
== Optimizer ==
Estimated cost: $0.0 / hour

Considered resources (1 node):
------------------------------------------------------------------------------------------------------------------------
 CLOUD        INSTANCE                       vCPUs   Mem(GB)   ACCELERATORS   REGION/ZONE           COST ($)   CHOSEN   
------------------------------------------------------------------------------------------------------------------------
 Kubernetes   2CPU--8GB--1H100               2       8         H100:1         my-h100-cluster-gke   0.00          ✔     
 Kubernetes   2CPU--8GB--1H100               2       8         H100:1         my-h100-cluster-eks   0.00                
 Lambda       gpu_1x_h100_pcie               26      200       H100:1         us-east-1             2.49                
 Cudo         sapphire-rapids-h100_1x2v4gb   2       4         H100:1         ca-montreal-3         2.83                
 Fluidstack   H100_PCIE_80GB::1              28      180       H100:1         CANADA                2.89                
 RunPod       1x_H100_SECURE                 16      80        H100:1         CA                    4.49                
 Azure        Standard_NC40ads_H100_v5       40      320       H100:1         eastus                6.98                
------------------------------------------------------------------------------------------------------------------------

Launching a new cluster 'sky-b34d-gcpuser'. Proceed? [Y/n]: 
sky launch --gpus l4
== Optimizer ==
Estimated cost: $0.0 / hour

Considered resources (1 node):
----------------------------------------------------------------------------------------------------------
 CLOUD        INSTANCE         vCPUs   Mem(GB)   ACCELERATORS   REGION/ZONE           COST ($)   CHOSEN   
----------------------------------------------------------------------------------------------------------
 Kubernetes   2CPU--8GB--1L4   2       8         L4:1           my-h100-cluster-gke   0.00          ✔     
 RunPod       1x_L4_SECURE     4       24        L4:1           CA                    0.44                
 GCP          g2-standard-4    4       16        L4:1           us-east4-a            0.70                
 AWS          g6.xlarge        4       16        L4:1           us-east-1             0.80                
----------------------------------------------------------------------------------------------------------

Tested (run the relevant ones):

  • Code formatting: bash format.sh
  • Any manual or new tests for this PR (please specify below)
  • All smoke tests: pytest tests/test_smoke.py
  • Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
  • Backward compatibility tests: conda deactivate; bash -i tests/backward_compatibility_tests.sh

@Michaelvll Michaelvll marked this pull request as ready for review September 27, 2024 23:38
Copy link
Collaborator

@romilbhardwaj romilbhardwaj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Michaelvll! Tested on GKE + local cluster, works well.

'use_spot': resources.use_spot
}
if isinstance(resources.cloud, clouds.Kubernetes):
resource_key_dict['region'] = resources.region
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a quick comment here describing why we add region here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! Added.

@romilbhardwaj romilbhardwaj added this to the v0.7 milestone Oct 22, 2024
@romilbhardwaj
Copy link
Collaborator

Should we merge this @Michaelvll?

@Michaelvll
Copy link
Collaborator Author

Oops, missed the review. Will merge it soon

@Michaelvll Michaelvll added this pull request to the merge queue Oct 29, 2024
Merged via the queue into master with commit 1a6e2f6 Oct 29, 2024
20 checks passed
@Michaelvll Michaelvll deleted the show-multi-k8s-contexts-in-optimizer branch October 29, 2024 04:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants