Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding Gitlab support #174

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

adding Gitlab support #174

wants to merge 8 commits into from

Conversation

valx
Copy link

@valx valx commented Oct 18, 2024

Main changes:

  • added support for Gitlab. The Gitlab pipeline files are in the folder template/{{.input_root_dir}}/.gitlab_ . Changes included:
    • Modified /mlops-stacks/databricks_template_schema.json prompt metadata by adding gitlab option
    • Searched all occurrences of input_cicd_platform and added a gitlab option
  • aded an integration test workspace variable input_databricks_test_workspace_host
  • in databricks bundle file, added mode: development to dev target

@valx valx marked this pull request as draft October 18, 2024 01:02
@valx valx marked this pull request as ready for review October 19, 2024 01:13
@@ -0,0 +1,15 @@
image: {{ .input_docker_image }}:latest

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

qq, why are these yml.tmpl name convention not consistent with what's in .github and .azure?
e.g in .github this is called
.github/workflows/{{.input_project_name}}-bundle-cd-prod.yml.tmpl

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 it would really help if we can follow similar naming conventions across providers

Copy link
Collaborator

@arpitjasa-db arpitjasa-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @valx ! Left a few comments, would you mind also generating a couple of Stacks projects from this PR and linking them in the PR? Would be good to verify this works with GitLab as expected, and the docs all look ok, as well as other providers remain unaffected.

* ``input_databricks_staging_workspace_host``: URL of staging Databricks workspace, used to run CI tests on PRs and preview config changes before they're deployed to production.
We encourage granting data scientists working on the current ML project non-admin (read) access to this workspace,
* ``input_cicd_platform`` : CI/CD platform of choice (GitHub Actions or GitHub Actions for GitHub Enterprise Servers or Azure DevOps or Gitlab)
* ``input_databricks_test_workspace_host``: URL of staging Databricks workspace, used to run CI tests on PRs before they're deployed to staging.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this? input_databricks_test_workspace_host is for dev or staging? If it's the same URL as staging we don't need another parameter right?

* ``input_cicd_platform`` : CI/CD platform of choice (GitHub Actions or GitHub Actions for GitHub Enterprise Servers or Azure DevOps)
* ``input_databricks_staging_workspace_host``: URL of staging Databricks workspace, used to run CI tests on PRs and preview config changes before they're deployed to production.
We encourage granting data scientists working on the current ML project non-admin (read) access to this workspace,
* ``input_cicd_platform`` : CI/CD platform of choice (GitHub Actions or GitHub Actions for GitHub Enterprise Servers or Azure DevOps or Gitlab)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* ``input_cicd_platform`` : CI/CD platform of choice (GitHub Actions or GitHub Actions for GitHub Enterprise Servers or Azure DevOps or Gitlab)
* ``input_cicd_platform`` : CI/CD platform of choice. Currently we support GitHub Actions, GitHub Actions for GitHub Enterprise Servers, Azure DevOps and GitLab.

"order": 20,
"type": "string",
"description": "\nDocker image for the execution of Gitlab pipelines",
"default": "registry.gitlab.com/ TODO /",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the TODO for here?

@@ -316,6 +331,44 @@
}
]
}
},
"input_docker_image": {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be an initialization parameter (generally we set this when it's used across several files) or can this be a different kind of parameter (job parameter, manually copy-pasted, etc.)?

@@ -0,0 +1,15 @@
image: {{ .input_docker_image }}:latest
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 it would really help if we can follow similar naming conventions across providers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants