Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code completion for DABs includes #819

Closed
wants to merge 6 commits into from
Closed

Code completion for DABs includes #819

wants to merge 6 commits into from

Conversation

fjakobs
Copy link
Collaborator

@fjakobs fjakobs commented Jul 25, 2023

Changes

This PR watches bundle config files and dynamically applies YAML code completion to all files included by the config.

Screen.Recording.2023-07-25.at.10.20.44.mov

@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:18 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:18 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:18 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:18 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:18 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:18 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:18 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:25 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:25 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:25 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:25 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:25 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:25 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:25 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 08:25 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 11:33 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 11:33 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 11:33 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 11:34 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 11:34 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 11:34 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 25, 2023 11:34 — with GitHub Actions Inactive
@codecov-commenter
Copy link

codecov-commenter commented Jul 25, 2023

Codecov Report

Merging #819 (3d733ca) into main (ae67ba2) will not change coverage.
Report is 1 commits behind head on main.
The diff coverage is n/a.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #819   +/-   ##
=======================================
  Coverage   79.78%   79.78%           
=======================================
  Files          47       47           
  Lines        1766     1766           
  Branches      366      366           
=======================================
  Hits         1409     1409           
  Misses        356      356           
  Partials        1        1           
Flag Coverage Δ
integration 61.04% <ø> (ø)
unit 66.76% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
js-sdk 79.78% <ø> (ø)


const folder = workspace.workspaceFolders?.[0];
const configFilePattern = new RelativePattern(
folder!,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return if no folder. Failing gracefully is better.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't we guaranteed to always have a workspace folder?

https://github.com/databricks/databricks-vscode/blob/main/packages/databricks-vscode/src/extension.ts#L71-L84

Still, I agree that we should make this code more defensive.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Just do not like the look of !

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I'll change it


// file watcher on all YAML files
const watcher = workspace.createFileSystemWatcher("**/*.{yml,yaml}");
watcher.onDidChange(async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use onDidCreate and onDidDelete

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also need a watcher on root bundle.yaml, so that we can update globs when the root is updated.
IMO we ONLY need the root watcher.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need an onDidCreate watcher for *.yml and an onDidChange watcher for {databricks|bundle}.{yaml|yml}.

That would be a bit more efficient but I feel that this code is easier to read.

We need the watcher on zonDidCreate in order to cover include patterns like *.yml.

Comment on lines +72 to +74
const resourceUri = Uri.parse(resource);
if (configFiles.has(resourceUri.path)) {
return rootConfigSchemaUri;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, why can't we just call the updateFileGlobs here? It is not a very expensive function (for reasonable bundles) and less compute intensive than watchers.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is executed whenever a file is opened. I'd like to keep that code path as fast as possible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This need not be too expensive. All we need to do is match the resourceUri with all the include paths in the root bundle.yaml.

That is mostly string matching and regex matching for globs. We shouldn't even need filesystem access if we have a single watcher on root bundle.yaml keeping the list of includes updated.

@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 09:53 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 09:53 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 09:53 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 09:53 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 09:53 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 09:53 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 09:53 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 09:53 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 10:01 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 10:01 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 10:01 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 10:01 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 10:01 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 10:01 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 10:01 — with GitHub Actions Inactive
@fjakobs fjakobs temporarily deployed to azure-prod-usr July 28, 2023 10:01 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr September 7, 2023 12:47 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr September 7, 2023 12:47 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr September 7, 2023 12:47 — with GitHub Actions Inactive
@kartikgupta-db kartikgupta-db temporarily deployed to azure-prod-usr September 7, 2023 12:47 — with GitHub Actions Inactive
@kartikgupta-db
Copy link
Contributor

Apparently we can have root config file including another file (a.yml) which in turn includes another file (b.yml). b.yml need not be explicitly included in the base root config.

Need to investigate, but putting this on hold for this release.

@kartikgupta-db
Copy link
Contributor

Closing in favour of #892

@kartikgupta-db kartikgupta-db deleted the dabs-include branch January 28, 2024 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants