Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[7.4.0] Fix the documentation for bazel fetch --configure #23723

Closed
wants to merge 1 commit into from
Closed

[7.4.0] Fix the documentation for bazel fetch --configure #23723

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 63 additions & 78 deletions site/en/extending/repo.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
Project: /_project.yaml
Book: /_book.yaml
Project: /\_project.yaml
Book: /\_book.yaml

# Repository Rules

{% include "_buttons.html" %}

This page covers how to create repository rules and provides examples for
more details.
This page covers how to define repository rules and provides examples for more
details.

An [external repository](/docs/external) is a rule that can be used only
in the `WORKSPACE` file and enables non-hermetic operation at the loading phase
Expand Down Expand Up @@ -70,8 +70,8 @@ If an attribute name starts with `_` it is private and users cannot set it.

## Implementation function

Every repository rule requires an `implementation` function. It contains the
actual logic of the rule and is executed strictly in the Loading Phase.
Every repo rule requires an `implementation` function. It contains the actual
logic of the rule and is executed strictly in the Loading Phase.

The function has exactly one input parameter, `repository_ctx`. The function
returns either `None` to signify that the rule is reproducible given the
Expand All @@ -81,11 +81,10 @@ example, for a rule tracking a git repository that would mean returning a
specific commit identifier instead of a floating branch that was originally
specified.

The input parameter `repository_ctx` can be used to
access attribute values, and non-hermetic functions (finding a binary,
executing a binary, creating a file in the repository or downloading a file
from the Internet). See [the library](/rules/lib/builtins/repository_ctx) for more
context. Example:
The input parameter `repository_ctx` can be used to access attribute values, and
non-hermetic functions (finding a binary, executing a binary, creating a file in
the repository or downloading a file from the Internet). See [the API
docs](/rules/lib/builtins/repository_ctx) for more context. Example:

```python
def _impl(repository_ctx):
Expand All @@ -98,86 +97,72 @@ local_repository = repository_rule(

## When is the implementation function executed?

The implementation function of a repository is executed when Bazel needs a
target from that repository, for example when another target (in another
repository) depends on it or if it is mentioned on the commmand line. The
implementation function is then expected to create the repository in the file
system. This is called "fetching" the repository.

In contrast to regular targets, repositories are not necessarily re-fetched when
something changes that would cause the repository to be different. This is
because there are things that Bazel either cannot detect changes to or it would
cause too much overhead on every build (for example, things that are fetched
from the network). Therefore, repositories are re-fetched only if one of the
following things changes:

* The parameters passed to the declaration of the repository in the
`WORKSPACE` file.
* The Starlark code comprising the implementation of the repository.
* The value of any environment variable passed to `repository_ctx`'s
The implementation function of a repo rule is executed when Bazel needs a target
from that repository, for example when another target (in another repo) depends
on it or if it is mentioned on the command line. The implementation function is
then expected to create the repo in the file system. This is called "fetching"
the repo.

In contrast to regular targets, repos are not necessarily re-fetched when
something changes that would cause the repo to be different. This is because
there are things that Bazel either cannot detect changes to or it would cause
too much overhead on every build (for example, things that are fetched from the
network). Therefore, repos are re-fetched only if one of the following things
changes:

- The attributes passed to the repo rule invocation.
- The Starlark code comprising the implementation of the repo rule.
- The value of any environment variable passed to `repository_ctx`'s
`getenv()` method or declared with the `environ` attribute of the
[`repository_rule`](/rules/lib/globals/bzl#repository_rule). The values
of these environment variables can be hard-wired on the command line with the
[`repository_rule`](/rules/lib/globals/bzl#repository_rule). The values of
these environment variables can be hard-wired on the command line with the
[`--repo_env`](/reference/command-line-reference#flag--repo_env) flag.
* The content of any file passed to the `read()`, `execute()` and similar
methods of `repository_ctx` which is referred to by a label (for example,
`//mypkg:label.txt` but not `mypkg/label.txt`)
* When `bazel sync` is executed.
- The existence, contents, and type of any paths being
[`watch`ed](/rules/lib/builtins/repository_ctx#watch) in the implementation
function of the repo rule.
- Certain other methods of `repository_ctx` with a `watch` parameter, such
as `read()`, `execute()`, and `extract()`, can also cause paths to be
watched.
- Similarly, [`repository_ctx.watch_tree`](/rules/lib/builtins/repository_ctx#watch_tree)
and [`path.readdir`](/rules/lib/builtins/path#readdir) can cause paths
to be watched in other ways.
- When `bazel fetch --force` is executed.

There are two parameters of `repository_rule` that control when the repositories
are re-fetched:

* If the `configure` flag is set, the repository is only re-fetched on
`bazel sync` when the` --configure` parameter is passed to it (if the
attribute is unset, this command will not cause a re-fetch)
* If the `local` flag is set, in addition to the above cases, the repository is
also re-fetched when the Bazel server restarts or when any file that affects
the declaration of the repository changes (e.g. the `WORKSPACE` file or a file
it loads), regardless of whether the changes resulted in a change to the
declaration of the repository or its code.

Non-local repositories are not re-fetched in these cases. This is because
these repositories are assumed to talk to the network or be otherwise
expensive.

## Restarting the implementation function

The implementation function can be _restarted_ while a repository is being
fetched if a dependency it requests is _missing_. In that case, the execution of
the implementation function will stop, the missing dependency is resolved and
the function will be re-executed after the dependency has been resolved. To
avoid unnecessary restarts (which are expensive, as network access might
have to be repeated), label arguments are prefetched, provided all
label arguments can be resolved to an existing file. Note that resolving
a path from a string or a label that was constructed only during execution
of the function might still cause a restart.
- If the `configure` flag is set, the repository is only re-fetched on `bazel
fetch` when the` --configure` parameter is passed to it (if the attribute is
unset, this command will not cause a re-fetch)
- If the `local` flag is set, in addition to the above cases, the repo is also
re-fetched when the Bazel server restarts.

## Forcing refetch of external repositories

Sometimes, an external repository can become outdated without any change to its
definition or dependencies. For example, a repository fetching sources might
follow a particular branch of a third-party repository, and new commits are
available on that branch. In this case, you can ask bazel to refetch all
external repositories unconditionally by calling `bazel sync`.

Moreover, some rules inspect the local machine and might become
outdated if the local machine was upgraded. Here you can ask bazel to
only refetch those external repositories where the
[`repository_rule`](/rules/lib/globals#repository_rule)
definition has the `configure` attribute set, use `bazel sync --configure`.
Sometimes, an external repo can become outdated without any change to its
definition or dependencies. For example, a repo fetching sources might follow a
particular branch of a third-party repository, and new commits are available on
that branch. In this case, you can ask bazel to refetch all external repos
unconditionally by calling `bazel fetch --force --all`.

Moreover, some repo rules inspect the local machine and might become outdated if
the local machine was upgraded. Here you can ask Bazel to only refetch those
external repos where the [`repository_rule`](/rules/lib/globals#repository_rule)
definition has the `configure` attribute set, use `bazel fetch --all
--configure`.

## Examples

- [C++ auto-configured toolchain](https://cs.opensource.google/bazel/bazel/+/master:tools/cpp/cc_configure.bzl;drc=644b7d41748e09eff9e47cbab2be2263bb71f29a;l=176):
it uses a repository rule to automatically create the
C++ configuration files for Bazel by looking for the local C++ compiler, the
environment and the flags the C++ compiler supports.
- [C++ auto-configured
toolchain](https://cs.opensource.google/bazel/bazel/+/master:tools/cpp/cc_configure.bzl;drc=644b7d41748e09eff9e47cbab2be2263bb71f29a;l=176):
it uses a repo rule to automatically create the C++ configuration files for
Bazel by looking for the local C++ compiler, the environment and the flags
the C++ compiler supports.

- [Go repositories](https://github.com/bazelbuild/rules_go/blob/67bc217b6210a0922d76d252472b87e9a6118fdf/go/private/go_repositories.bzl#L195)
uses several `repository_rule` to defines the list of dependencies
needed to use the Go rules.
uses several `repository_rule` to defines the list of dependencies needed to
use the Go rules.

- [rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external) creates
an external repository called `@maven` by default that generates build targets
for every Maven artifact in the transitive dependency tree.
- [rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external)
creates an external repository called `@maven` by default that generates
build targets for every Maven artifact in the transitive dependency tree.
Loading