Skip to content

Commit

Permalink
feat: Enable external api key for launch (#34)
Browse files Browse the repository at this point in the history 
Co-authored-by: Chris Van Pelt <vanpelt@wandb.com>
  • Loading branch information
@zacharyblasczyk @vanpelt
zacharyblasczyk and vanpelt authored Oct 23, 2023
1 parent 9d2ef9c commit 75c3d40
Show file tree
Hide file tree
Showing 9 changed files with 34 additions and 23 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/lint-test.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
# Python is required because `ct lint` runs Yamale (https://github.com/23andMe/Yamale) and
# yamllint (https://github.com/adrienverge/yamllint) which require Python
- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: "3.10"

Expand All @@ -41,9 +41,9 @@ jobs:
run: ct lint --config ct.yaml

- name: Create kind cluster
uses: helm/kind-action@v1.4.0
uses: helm/kind-action@v1.8.0
with:
version: v0.16.0
version: v0.20.0
if: steps.list-changed.outputs.changed == 'true'

- name: Run chart-testing (install)
Expand Down
2 changes: 1 addition & 1 deletion charts/launch-agent/Chart.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: launch-agent
icon: https://em-content.zobj.net/thumbs/240/apple/354/rocket_1f680.png
description: A Helm chart for running the W&B Launch Agent in Kubernetes
type: application
version: 0.10.2
version: 0.11.0
maintainers:
- name: wandb
email: support@wandb.com
Expand Down
37 changes: 21 additions & 16 deletions charts/launch-agent/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

This chart deploys the W&B Launch Agent to your Kubernetes cluster.

If you have not added `wandb` as a helm repo, please run:
If you have not added `wandb` as a helm repo, please run:

```bash
helm repo add wandb https://wandb.github.io/helm-charts
```
Expand All @@ -12,11 +13,13 @@ The launch agent is a Kubernetes Deployment that runs a container that connects
To deploy an agent, you will need to specify the following values in [`values.yaml`](values.yaml):

- `agent.apiKey`: Your W&B API key
- Note: `agent.useExternalWandbSecret` can be set to `true` if you would like to provide your api key external to this helm chart.
- `launchConfig`: The literal contents of a launch agent config file that will be used to configure the agent. See the [launch agent docs](https://docs.wandb.ai/guides/launch/run-agent) for more information.

You will likely want to modify the variable `agent.resources.limits.{cpu,mem}`, which default to `1000m`, and `1Gi` respectively.

By default, this chart will also install [volcano](https://volcano.sh)

- `volcano`: Set to `false` to disable volcano install (default: `true`)

You can modify the values directly in the `values.yaml` file or provide them as command line arguments when running `helm install`, for example:
Expand All @@ -32,20 +35,22 @@ helm upgrade --namespace=wandb --create-namespace --install wandb-launch wandb/l
```

## Chart variables

The table below describes all the available variables in the chart:

| Variable | Type | Required | Default | Description |
| ----------------------------- | -------------- | -------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| `agent.labels` | object | No | {} | Labels that will be added to the agent deployment. |
| `agent.apiKey` | string | **Yes** | "" | W&B API key to be used by the agent. |
| `agent.image` | string | No | `wandb/launch-agent-dev:latest` | Container image for the agent. |
| `agent.imagePullPolicy` | string | No | `Always` | Pull policy for the agent container image. |
| `agent.resources` | object | No | Limit to 1 CPU, 1Gi RAM | Pod spec resources block for the agent. |
| `namespace` | string | No | `wandb` | The namespace to deploy the agent into. |
| `additionalTargetNamespaces` | list(string) | No | [`wandb`,`default`] | A list of namespaces the agent can run jobs in. |
| `baseUrl` | string | No | `https://api.wandb.ai` | URL of your W&B server api. |
| `launchConfig` | mutiline string | **Yes** | `null` | his should be set to the literal contents of your launch agent config. |
| `volcano` | bool | No | `true` | Controls whether the volcano scheduler should be installed in your cluster along with the agent. Set to `false` to disable volcano installation. |
| `gitCreds` | mutiline string | No | `null` | Contents of a git credentials file. |
| `serviceAccount.annotations` | object | No | `null` | Annotations for the wandb service account. |
| `azureStorageAccessKey` | string | No | "" | Azure storage access key required for kaniko to acces build contexts in azure blob storage. |
| Variable | Type | Required | Default | Description |
| ------------------------------ | --------------- | -------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| `agent.labels` | object | No | {} | Labels that will be added to the agent deployment. |
| `agent.apiKey` | string | **Yes**<sup>\*</sup> | "" | W&B API key to be used by the agent. |
| `agent.useExternalWandbSecret` | bool | **false** | "" | Used to indicate you want to provide the api key secret external to this chart. |
| `agent.image` | string | No | `wandb/launch-agent-dev:latest` | Container image for the agent. |
| `agent.imagePullPolicy` | string | No | `Always` | Pull policy for the agent container image. |
| `agent.resources` | object | No | Limit to 1 CPU, 1Gi RAM | Pod spec resources block for the agent. |
| `namespace` | string | No | `wandb` | The namespace to deploy the agent into. |
| `additionalTargetNamespaces` | list(string) | No | [`wandb`,`default`] | A list of namespaces the agent can run jobs in. |
| `baseUrl` | string | No | `https://api.wandb.ai` | URL of your W&B server api. |
| `launchConfig` | mutiline string | **Yes** | `null` | his should be set to the literal contents of your launch agent config. |
| `volcano` | bool | No | `true` | Controls whether the volcano scheduler should be installed in your cluster along with the agent. Set to `false` to disable volcano installation. |
| `gitCreds` | mutiline string | No | `null` | Contents of a git credentials file. |
| `serviceAccount.annotations` | object | No | `null` | Annotations for the wandb service account. |
| `azureStorageAccessKey` | string | No | "" | Azure storage access key required for kaniko to acces build contexts in azure blob storage. |
1 change: 1 addition & 0 deletions charts/launch-agent/ci/basic-values.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
agent:
apiKey: "<an-api-key>"
useExternalWandbSecret: false
image: wandb/launch-agent:latest
imagePullPolicy: Always
resources:
Expand Down
3 changes: 3 additions & 0 deletions charts/launch-agent/templates/secret.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
{{- if not .Values.agent.useExternalWandbSecret }}
---
apiVersion: v1
kind: Secret
Expand All @@ -8,6 +9,8 @@ type: kubernetes.io/basic-auth
stringData:
password: {{ required "Please set agent.apiKey to a W&B API key" .Values.agent.apiKey }}
...
{{ end }}

{{- if .Values.gitCreds }}
---
apiVersion: v1
Expand Down
2 changes: 2 additions & 0 deletions charts/launch-agent/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ agent:
labels: {}
# W&B API key.
apiKey: ""
# Providing API key can be done external to this chart
useExternalWandbSecret: false
# Container image to use for the agent.
image: wandb/launch-agent:latest
# Image pull policy for agent image.
Expand Down
2 changes: 1 addition & 1 deletion charts/wandb/Chart.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
name: wandb
description: A Helm chart for deploying W&B to Kubernetes
type: application
version: 0.3.0
version: 0.3.1
appVersion: "0.44.1"
icon: https://wandb.ai/logo.svg
maintainers:
Expand Down
2 changes: 1 addition & 1 deletion charts/wandb/ci/basic-values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ mysql:
resources:
requests:
cpu: 100m
memory: 100Mi
memory: 1000Mi
2 changes: 1 addition & 1 deletion charts/wandb/templates/tests/test-connection.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ metadata:
spec:
containers:
- name: wandb-verify
image: python:3
image: python:3.10
env:
- name: WANDB_BASE_URL
value: http://{{ include "wandb.fullname" . }}:{{ .Values.service.port }}
Expand Down

0 comments on commit 75c3d40

Please sign in to comment.