Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add optional path var for instance level bucket path #251

Merged
merged 6 commits into from
Aug 23, 2024

Conversation

levinandrew
Copy link
Contributor

@levinandrew levinandrew commented Jul 31, 2024

  # module.wandb_infra.module.wandb.helm_release.wandb will be updated in-place
  ~ resource "helm_release" "wandb" {
        id                         = "wandb-cr"
      ~ metadata                   = [
          - {
              ...
              - values         = jsonencode(
                    {
                      - name = "wandb"
                      - spec = <<-EOT
                            "values":
                              "app": {}
                              "global":
                                "bucket":
                                  "kmsKey": "arn:aws:kms:us-west-2:770934259321:key/c88a4da3-aabd-4265-a57d-e4f0115d04b8"
                                  "name": "levin-aws-subpath-testing-file-storage-unbiased-chimp"
                                  "path": ""
                                  "provider": "s3"
                                  "region": "us-west-2"
                                "cloudProvider": "aws"
                         ...
                        EOT
                    }
                )
              - version        = "0.1.0"
            },
        ] -> (known after apply)
        name                       = "wandb-cr"
        # (26 unchanged attributes hidden)

      - set {
          - name  = "spec" -> null
          - type  = "string" -> null
          - value = <<-EOT
                "values":
                  "app": {}
                  "global":
                    "bucket":
                      "kmsKey": "arn:aws:kms:us-west-2:770934259321:key/c88a4da3-aabd-4265-a57d-e4f0115d04b8"
                      "name": "levin-aws-subpath-testing-file-storage-unbiased-chimp"
                      "path": ""
                      "provider": "s3"
                      "region": "us-west-2"
                    "cloudProvider": "aws"
 
            EOT -> null
        }
      + set {
          + name  = "spec"
          + type  = "string"
          + value = <<-EOT
                "values":
                  "app": {}
                  "global":
                    "bucket":
                      "kmsKey": "arn:aws:kms:us-west-2:770934259321:key/c88a4da3-aabd-4265-a57d-e4f0115d04b8"
                      "name": "levin-aws-subpath-testing-file-storage-unbiased-chimp"
                      "path": "nested/path"
                      "provider": "s3"
                      "region": "us-west-2"
                    "cloudProvider": "aws"
 ...
            EOT
        }

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

@levinandrew levinandrew marked this pull request as draft July 31, 2024 20:11
@levinandrew levinandrew changed the title Add optional path var for instance level bucket path feat: Add optional path var for instance level bucket path Jul 31, 2024
@levinandrew levinandrew marked this pull request as ready for review August 22, 2024 07:17
@zacharyblasczyk
Copy link
Contributor

@levinandrew what was the driver of this change for context?

CC @wandb/delivery-tooling-team

@zacharyblasczyk
Copy link
Contributor

The reason path is in the helm spec currently is because azure requires it for the storage account/ blob store.

Copy link
Contributor

@zacharyblasczyk zacharyblasczyk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming this is coming from a customer ask.

@levinandrew
Copy link
Contributor Author

Hey @zacharyblasczyk thanks for the review 🤜 🤛

This did come from a customer ask to be able to store all data at a path of the instance level bucket instead of its root. We are making similar changes for Azure and Google, and will handle the Azure case you commented on.

@levinandrew levinandrew merged commit 9ccc357 into main Aug 23, 2024
4 checks passed
@levinandrew levinandrew deleted the levinandrew/byob-subpath branch August 23, 2024 18:57
jsbroks pushed a commit that referenced this pull request Aug 23, 2024
## [4.23.0](v4.22.3...v4.23.0) (2024-08-23)

### Features

* Add optional path var for instance level bucket path ([#251](#251)) ([9ccc357](9ccc357))
@jsbroks
Copy link
Member

jsbroks commented Aug 23, 2024

This PR is included in version 4.23.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants