Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add watch parameter to {module,repo}_ctx.path #23386

Closed
wants to merge 2 commits into from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented Aug 22, 2024

No description provided.

@github-actions github-actions bot added team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. awaiting-review PR is awaiting review from an assigned reviewer labels Aug 22, 2024
@fmeum
Copy link
Collaborator Author

fmeum commented Aug 22, 2024

FYI @aignas

@aignas
Copy link

aignas commented Aug 22, 2024

Will this be backported to 7.3.2 or an upcoming 7.4 release? Trying to understand about what kind of helpers/wrappers we need to maintain within rules_python, so that we can more easily support more bazel versions.

@fmeum
Copy link
Collaborator Author

fmeum commented Aug 22, 2024

@bazel-io fork 7.4.0

// This can never happen because we check it in the Starlark interpreter.
default -> throw new IllegalArgumentException("expected string or label for path");
};
if (shouldWatch == ShouldWatch.YES) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just maybeWatch(result, shouldWatch) here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment, this would be backwards incompatible (caught by one of the new tests).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(sorry for the delay)

You're talking about testPath_watchAuto_starlarkPathAndAbsolutePathNotWatched, right? I agree that's a change in behavior, but I don't know if it counts as backwards-incompatible. Didn't we do the same behavior change for repo rules?

Plus, we already have the condition where watch = 'auto' means that paths outside the workspace aren't watched (https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/bazel/repository/starlark/StarlarkBaseExternalContext.java;drc=6052ad65e4185f19944e5138e9b70f6a4eaf9e76;l=1457). Isn't that enough?

Copy link
Collaborator Author

@fmeum fmeum Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR title was misleading, it adds this parameter to both repo and module context, so we don't have prior art for this function to refer to.

I agree that the change is minor, but I think that it would end up breaking the code in https://github.com/bazelbuild/rules_python/pull/2135/files#diff-ca74c876e894b0bd25547ca666e562281e5c8fb2abc4a51446f410371224aea2 by reintroducing a dependency into the lockfile. Depending on a sibling label, manipulating the path and then obtaining a StarlarkPath for it via this method is the kind of existing "pattern" that changing this would break.

I'm not in love with that edge case and we could decide to point users to bazel_features instead, but it would still be breaking.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, I see. Could you add some more details in the PR description about the motivation of this change? It got lost in the commotion.

I could see an API where auto (or whatever the default value for [mr]ctx.path(watch=) ends up being) means "don't watch". Or just make it default to no. Or actually, maybe just make [mr]ctx.watch return the path?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, I remember now -- so rctx.path(label) used to watch, and rctx.path(path) and rctx.path(string) used to not watch.

This PR retains this old rctx.path(X) watch behavior (which is based on the type of X, not the actual location of X or whether it's rctx or mctx). It just allows the caller to be explicit about watching or not by specifying watch='yes' or watch='no'. The awkward part is that watch='auto' means a different thing for rctx.path() than rctx.read(), rctx.extract(), etc.

I wonder if we'd want rctx.path(X) to never watch (just use rctx.watch() instead). That's obviously backwards-incompatible because some people rely on rctx.path(label) watching, but maybe it's worth introducing an incompatible flag flip for? Undecided.

Also, the fact that rctx.read(label, watch='no') actually still watches is a bit of a subtle hiccup. I'll need to think about that too...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that I've thought more about it, I think that I would prefer the following changes:

  • ctx.path never watches and doesn't have a watch parameter. It only ever watched because that was the only way to watch anything at all. It doesn't make much sense that resolving a label to a path is linked to caring about the content at that path. ctx.watch can replace it.
  • ctx.read(label, watch = "no") shouldn't watch.

The latter clearly violates the intent, I would consider changing that behavior a bug fix and don't think it would break anyone.
The former is more impactful to change as it has been the only way to watch for a long time. rules_go would definitely need to change. Luckily ctx.watch can be trivially feature detected.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fully agree. Just a bit nervous about not having an incompatible flag for it. (so maybe not so "fully" 😅) Adding an incompatible flag for this one isn't super bad, right? It's just a single if in StarlarkBaseExternalContext so shouldn't be a lot of maintenance burden. And we can make it no-op in Bazel 9.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I agree the change for path should come with a flag. But since it will mostly require fixing in rulesets, not user projects, we would need to look into that regardless.

@fmeum fmeum changed the title Add watch parameter to module_ctx.path Add watch parameter to {module,repo}_ctx.path Aug 28, 2024
@fmeum
Copy link
Collaborator Author

fmeum commented Sep 25, 2024

Closing in favor of #23750

@fmeum fmeum closed this Sep 25, 2024
@fmeum fmeum deleted the watch-mctx-path branch September 25, 2024 09:37
@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants