diff --git a/repositories.bzl b/repositories.bzl index 2a7e8d5ec3e211..d1b7f1ca69a282 100644 --- a/repositories.bzl +++ b/repositories.bzl @@ -15,7 +15,7 @@ """ -load("//tools/build_defs/repo:http.bzl", "http_archive", "http_file") +load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file") load("//src/tools/bzlmod:utils.bzl", "get_canonical_repo_name") ################################################################################## diff --git a/site/en/concepts/build-ref.md b/site/en/concepts/build-ref.md index c9b87b38112116..849762c6653328 100644 --- a/site/en/concepts/build-ref.md +++ b/site/en/concepts/build-ref.md @@ -1,47 +1,37 @@ Project: /_project.yaml Book: /_book.yaml -# Workspaces, packages, and targets +# Repositories, workspaces, packages, and targets {% include "_buttons.html" %} -Bazel builds software from source code organized in a directory tree called a -workspace. Source files in the workspace are organized in a nested hierarchy of -packages, where each package is a directory that contains a set of related -source files and one `BUILD` file. The `BUILD` file specifies what software -outputs can be built from the source. - -## Workspace {:#workspace} - -A _workspace_ is a directory tree on your filesystem that contains the source -files for the software you want to build. Each workspace has a text file named -`WORKSPACE` which may be empty, or may contain references to [external -dependencies](/docs/external) required to build the outputs. - -Directories containing a file called `WORKSPACE` are considered the root of a -workspace. Therefore, Bazel ignores any directory trees in a workspace rooted at -a subdirectory containing a `WORKSPACE` file, as they form another workspace. - -Bazel also supports `WORKSPACE.bazel` file as an alias of `WORKSPACE` file. If -both files exist, `WORKSPACE.bazel` is used. +Bazel builds software from source code organized in directory trees called +repositories. A defined set of repositories comprises the workspace. Source +files in repositories are organized in a nested hierarchy of packages, where +each package is a directory that contains a set of related source files and one +`BUILD` file. The `BUILD` file specifies what software outputs can be built from +the source. ### Repositories {:#repositories} -Code is organized in _repositories_. The directory containing the `WORKSPACE` -file is the root of the main repository, also called `@`. Other, (external) -repositories are defined in the `WORKSPACE` file using workspace rules, or -generated from modules and extensions in the Bzlmod system. See [external +Source files used in a Bazel build are organized in _repositories_ (often +shortened to _repos_). A repo is a directory tree with a boundary marker file at +its root; such a boundary marker file could be `MODULE.bazel`, `REPO.bazel`, or +in legacy contexts, `WORKSPACE` or `WORKSPACE.bazel`. + +The repo in which the current Bazel command is being run is called the _main +repo_. Other, (external) repos are defined by _repo rules_; see [external dependencies overview](/external/overview) for more information. -The workspace rules bundled with Bazel are documented in the [Workspace -Rules](/reference/be/workspace) section in the [Build -Encyclopedia](/reference/be/overview) and the documentation on [embedded -Starlark repository rules](/rules/lib/repo/index). +## Workspace {:#workspace} + +A _workspace_ is the environment shared by all Bazel commands run from the same +main repo. It encompasses the main repo and the set of all defined external +repos. -As external repositories are repositories themselves, they often contain a -`WORKSPACE` file as well. However, these additional `WORKSPACE` files are -ignored by Bazel. In particular, repositories depended upon transitively are not -added automatically. +Note that historically the concepts of "repository" and "workspace" have been +conflated; the term "workspace" has often been used to refer to the main +repository, and sometimes even used as a synonym of "repository". ## Packages {:#packages} diff --git a/site/en/concepts/platforms.md b/site/en/concepts/platforms.md index edb3e5daf63f0e..2563eb438613ed 100644 --- a/site/en/concepts/platforms.md +++ b/site/en/concepts/platforms.md @@ -175,7 +175,7 @@ For your project to build, you need to check the following: 1. The toolchains you want to use must exist. If using stock toolchains, the language owners should include instructions for how to register them. If writing your own custom toolchains, you need to [register](https://bazel.build/extending/toolchains#registering-building-toolchains) them in your - `WORKSPACE` or with [`--extra_toolchains`](https://bazel.build/reference/command-line-reference#flag--extra_toolchains). + `MODULE.bazel` file or with [`--extra_toolchains`](https://bazel.build/reference/command-line-reference#flag--extra_toolchains). 1. `select()`s and [configuration transitions][Starlark transitions] must resolve properly. See [select()](#select) and [Transitions](#transitions). @@ -374,8 +374,8 @@ When building `$ bazel build //:myproject --platforms=//:myplatform`, Bazel automatically selects a toolchain that can run on the build machine and build binaries for `//:myplatform`. This is known as *toolchain resolution*. -The set of available toolchains can be registered in the `WORKSPACE` with -[`register_toolchains`][register_toolchains Function] or at the +The set of available toolchains can be registered in the `MODULE.bazel` file +with [`register_toolchains`][register_toolchains Function] or at the command line with [`--extra_toolchains`][extra_toolchains Flag]. For more information see [here][Toolchains]. @@ -419,7 +419,7 @@ contact [bazel-dev]{: .external}. [Platforms examples]: https://github.com/hlopko/bazel_platforms_examples [platform mappings design]: https://docs.google.com/document/d/1Vg_tPgiZbSrvXcJ403vZVAGlsWhH9BUDrAxMOYnO0Ls/edit [platform Rule]: /reference/be/platforms-and-toolchains#platform -[register_toolchains Function]: /rules/lib/globals/workspace#register_toolchains +[register_toolchains Function]: /rules/lib/globals/module#register_toolchains [Rust rules]: https://github.com/bazelbuild/rules_rust [select()]: /docs/configurable-attributes [select() Platforms]: /docs/configurable-attributes#platforms diff --git a/site/en/configure/attributes.md b/site/en/configure/attributes.md index 23df7bf606fadd..43a8dbe96fb459 100644 --- a/site/en/configure/attributes.md +++ b/site/en/configure/attributes.md @@ -916,9 +916,12 @@ def selecty_genrule(name, select_cmd): ### Why doesn't select() work with bind()? {:#faq-select-bind} -Because [`bind()`](/reference/be/workspace#bind) is a WORKSPACE rule, not a BUILD rule. +First of all, do not use `bind()`. It is deprecated in favor of `alias()`. -Workspace rules do not have a specific configuration, and aren't evaluated in +The technical answer is that [`bind()`](/reference/be/workspace#bind) is a repo +rule, not a BUILD rule. + +Repo rules do not have a specific configuration, and aren't evaluated in the same way as BUILD rules. Therefore, a `select()` in a `bind()` can't actually evaluate to any specific branch. @@ -927,8 +930,6 @@ the `actual` attribute, to perform this type of run-time determination. This works correctly, since `alias()` is a BUILD rule, and is evaluated with a specific configuration. -You can even have a `bind()` target point to an `alias()`, if needed. - ```sh $ cat WORKSPACE workspace(name = "myapp") @@ -957,6 +958,8 @@ With this setup, you can pass `--define ssl_library=alternative`, and any target that depends on either `//:ssl` or `//external:ssl` will see the alternative located at `@alternative//:ssl`. +But really, stop using `bind()`. + ### Why doesn't my select() choose what I expect? {:#faq-select-choose-condition} If `//myapp:foo` has a `select()` that doesn't choose the condition you expect, diff --git a/site/en/configure/windows.md b/site/en/configure/windows.md index 90730e24c6dfa8..32a4127422e9f7 100644 --- a/site/en/configure/windows.md +++ b/site/en/configure/windows.md @@ -268,7 +268,7 @@ To enable the Clang toolchain for building C++, there are several situations. --extra_toolchains=@local_config_cc//:cc-toolchain-x64_windows-clang-cl --extra_execution_platforms=//:x64_windows-clang-cl ``` - * Register the platform and toolchain in your `WORKSPACE` file: + * Register the platform and toolchain in your `MODULE.bazel` file: ``` register_execution_platforms( diff --git a/site/en/contribute/codebase.md b/site/en/contribute/codebase.md index 3189a5b587a670..8744adc2da3df0 100644 --- a/site/en/contribute/codebase.md +++ b/site/en/contribute/codebase.md @@ -47,10 +47,10 @@ and some are after (`-c opt`); the former kind is called a "startup option" and affects the server process as a whole, whereas the latter kind, the "command option", only affects a single command. -Each server instance has a single associated source tree ("workspace") and each -workspace usually has a single active server instance. This can be circumvented -by specifying a custom output base (see the "Directory layout" section for more -information). +Each server instance has a single associated workspace (collection of source +trees known as "repositories") and each workspace usually has a single active +server instance. This can be circumvented by specifying a custom output base +(see the "Directory layout" section for more information). Bazel is distributed as a single ELF executable that is also a valid .zip file. When you type `bazel`, the above ELF executable implemented in C++ (the @@ -96,8 +96,9 @@ from the client are handled by `GrpcServerImpl.run()`. Bazel creates a somewhat complicated set of directories during a build. A full description is available in [Output directory layout](/remote/output-directories). -The "workspace" is the source tree Bazel is run in. It usually corresponds to -something you checked out from source control. +The "main repo" is the source tree Bazel is run in. It usually corresponds to +something you checked out from source control. The root of this directory is +known as the "workspace root". Bazel puts all of its data under the "output user root". This is usually `$HOME/.cache/bazel/_bazel_${USER}`, but can be overridden using the @@ -226,12 +227,11 @@ Bazel, in contrast, supports projects whose source code spans multiple repositories. The repository from which Bazel is invoked is called the "main repository", the others are called "external repositories". -A repository is marked by a file called `WORKSPACE` (or `WORKSPACE.bazel`) in -its root directory. This file contains information that is "global" to the whole -build, for example, the set of available external repositories. It works like a -regular Starlark file which means that one can `load()` other Starlark files. -This is commonly used to pull in repositories that are needed by a repository -that's explicitly referenced (we call this the "`deps.bzl` pattern") +A repository is marked by a repo boundary file (`MODULE.bazel`, `REPO.bazel`, or +in legacy contexts, `WORKSPACE` or `WORKSPACE.bazel`) in its root directory. The +main repo is the source tree where you're invoking Bazel from. External repos +are defined in various ways; see [external dependencies +overview](/external/overview) for more information. Code of external repositories is symlinked or downloaded under `$OUTPUT_BASE/external`. @@ -239,9 +239,7 @@ Code of external repositories is symlinked or downloaded under When running the build, the whole source tree needs to be pieced together; this is done by `SymlinkForest`, which symlinks every package in the main repository to `$EXECROOT` and every external repository to either `$EXECROOT/external` or -`$EXECROOT/..` (the former of course makes it impossible to have a package -called `external` in the main repository; that's why we are migrating away from -it) +`$EXECROOT/..`. ### Packages {:#packages} @@ -277,7 +275,8 @@ Globbing is implemented in the following classes: the legacy globber in order to avoid "Skyframe restarts" (described below) The `Package` class itself contains some members that are exclusively used to -parse the WORKSPACE file and which do not make sense for real packages. This is +parse the "external" package (related to external dependencies) and which do not +make sense for real packages. This is a design flaw because objects describing regular packages should not contain fields that describe something else. These include: @@ -285,10 +284,10 @@ fields that describe something else. These include: * The registered toolchains * The registered execution platforms -Ideally, there would be more separation between parsing the WORKSPACE file from -parsing regular packages so that `Package`does not need to cater for the needs -of both. This is unfortunately difficult to do because the two are intertwined -quite deeply. +Ideally, there would be more separation between parsing the "external" package +from parsing regular packages so that `Package` does not need to cater for the +needs of both. This is unfortunately difficult to do because the two are +intertwined quite deeply. ### Labels, Targets, and Rules {:#labels-targets-rules} @@ -417,25 +416,21 @@ implementation used in Bazel is currently an interpreter. Starlark is used in several contexts, including: -1. **The `BUILD` language.** This is where new rules are defined. Starlark code - running in this context only has access to the contents of the `BUILD` file - itself and `.bzl` files loaded by it. -2. **Rule definitions.** This is how new rules (such as support for a new - language) are defined. Starlark code running in this context has access to - the configuration and data provided by its direct dependencies (more on this - later). -3. **The WORKSPACE file.** This is where external repositories (code that's not - in the main source tree) are defined. -4. **Repository rule definitions.** This is where new external repository types - are defined. Starlark code running in this context can run arbitrary code on - the machine where Bazel is running, and reach outside the workspace. +1. **`BUILD` files.** This is where new build targets are defined. Starlark + code running in this context only has access to the contents of the `BUILD` + file itself and `.bzl` files loaded by it. +2. **The `MODULE.bazel` file.** This is where external dependencies are + defined. Starlark code running in this context only has very limited access + to a few predefined directives. +3. **`.bzl` files.** This is where new build rules, repo rules, module + extensions are defined. Starlark code here can define new functions and load + from other `.bzl` files. The dialects available for `BUILD` and `.bzl` files are slightly different because they express different things. A list of differences is available [here](/rules/language#differences-between-build-and-bzl-files). -More information about Starlark is available -[here](/rules/language). +More information about Starlark is available [here](/rules/language). ## The loading/analysis phase {:#loading-phase} @@ -721,7 +716,7 @@ time to load. Execution platforms are specified in one of the following ways: -1. In the WORKSPACE file using the `register_execution_platforms()` function +1. In the MODULE.bazel file using the `register_execution_platforms()` function 2. On the command line using the --extra\_execution\_platforms command line option @@ -736,7 +731,7 @@ yet. The set of toolchains to be used for a configured target is determined by `ToolchainResolutionFunction`. It is a function of: -* The set of registered toolchains (in the WORKSPACE file and the +* The set of registered toolchains (in the MODULE.bazel file and the configuration) * The desired execution and target platforms (in the configuration) * The set of toolchain types that are required by the configured target (in @@ -1444,6 +1439,11 @@ This is implemented in the `build.lib.buildeventservice` and ## External repositories {:#external-repos} +Note: The information in this section is out of date, as code in this area has +undergone extensive change in the past couple of years. Please refer to +[external dependencies overview](/external/overview) for more up-to-date +information. + Whereas Bazel was originally designed to be used in a monorepo (a single source tree containing everything one needs to build), Bazel lives in a world where this is not necessarily true. "External repositories" are an abstraction used to @@ -1515,23 +1515,6 @@ invalidated when the definition of the repository they are in changes. Thus, `FileStateValue`s for an artifact in an external repository need to depend on their external repository. This is handled by `ExternalFilesHelper`. -### Managed directories {:#managed-directories} - -Sometimes, external repositories need to modify files under the workspace root -(such as a package manager that houses the downloaded packages in a subdirectory of -the source tree). This is at odds with the assumption Bazel makes that source -files are only modified by the user and not by itself and allows packages to -refer to every directory under the workspace root. In order to make this kind of -external repository work, Bazel does two things: - -1. Allows the user to specify subdirectories of the workspace Bazel is not - allowed to reach into. They are listed in a file called `.bazelignore` and - the functionality is implemented in `BlacklistedPackagePrefixesFunction`. -2. We encode the mapping from the subdirectory of the workspace to the external - repository it is handled by into `ManagedDirectoriesKnowledge` and handle - `FileStateValue`s referring to them in the same way as those for regular - external repositories. - ### Repository mappings {:#repo-mappings} It can happen that multiple repositories want to depend on the same repository, diff --git a/site/en/docs/configurable-attributes.md b/site/en/docs/configurable-attributes.md index 8e7db46df0be2c..9929bbf4b0976a 100644 --- a/site/en/docs/configurable-attributes.md +++ b/site/en/docs/configurable-attributes.md @@ -916,9 +916,12 @@ def selecty_genrule(name, select_cmd): ### Why doesn't select() work with bind()? {:#faq-select-bind} -Because [`bind()`](/reference/be/workspace#bind) is a WORKSPACE rule, not a BUILD rule. +First of all, do not use `bind()`. It is deprecated in favor of `alias()`. -Workspace rules do not have a specific configuration, and aren't evaluated in +The technical answer is that [`bind()`](/reference/be/workspace#bind) is a repo +rule, not a BUILD rule. + +Repo rules do not have a specific configuration, and aren't evaluated in the same way as BUILD rules. Therefore, a `select()` in a `bind()` can't actually evaluate to any specific branch. @@ -957,6 +960,8 @@ With this setup, you can pass `--define ssl_library=alternative`, and any target that depends on either `//:ssl` or `//external:ssl` will see the alternative located at `@alternative//:ssl`. +But really, stop using `bind()`. + ### Why doesn't my select() choose what I expect? {:#faq-select-choose-condition} If `//myapp:foo` has a `select()` that doesn't choose the condition you expect, diff --git a/site/en/extending/macros.md b/site/en/extending/macros.md index a6ba77c18360f0..a0f73905abadef 100644 --- a/site/en/extending/macros.md +++ b/site/en/extending/macros.md @@ -138,8 +138,7 @@ def my_macro(name, visibility=None): If you need to know the package name (for example, which `BUILD` file is calling the macro), use the function [native.package_name()](/rules/lib/toplevel/native#package_name). -Note that `native` can only be used in `.bzl` files, and not in `WORKSPACE` or -`BUILD` files. +Note that `native` can only be used in `.bzl` files, and not in `BUILD` files. ## Label resolution in macros {:#label-resolution} @@ -164,7 +163,7 @@ def my_cc_wrapper(name, deps = [], **kwargs): Label("//config:needs_foo"): [ # Due to the use of Label, this label will resolve to the correct target # even if the canonical name of @dep_of_my_ruleset should be different - # in the main workspace, such as due to repo mappings. + # in the main repo, such as due to repo mappings. Label("@dep_of_my_ruleset//tools:foo"), ], "//conditions:default": [], diff --git a/site/en/extending/repo.md b/site/en/extending/repo.md index e617d625bde305..78b47b5dab4606 100644 --- a/site/en/extending/repo.md +++ b/site/en/extending/repo.md @@ -5,45 +5,44 @@ Book: /_book.yaml {% include "_buttons.html" %} -This page covers how to create repository rules and provides examples for -more details. +This page covers how to define repository rules and provides examples for more +details. -An [external repository](/docs/external) is a rule that can be used only -in the `WORKSPACE` file and enables non-hermetic operation at the loading phase -of Bazel. Each external repository rule creates its own workspace, with its -own `BUILD` files and artifacts. They can be used to depend on third-party -libraries (such as Maven packaged libraries) but also to generate `BUILD` files -specific to the host Bazel is running on. +An [external repository](/external/overview#repository) is a directory tree, +containing source files usable in a Bazel build, which is generated on demand by +running its corresponding **repo rule**. Repos can be defined in a multitude of +ways, but ultimately, each repo is defined by invoking a repo rule, just as +build targets are defined by invoking build rules. They can be used to depend on +third-party libraries (such as Maven packaged libraries) but also to generate +`BUILD` files specific to the host Bazel is running on. -## Repository rule creation +## Repository rule definition In a `.bzl` file, use the -[repository_rule](/rules/lib/globals/bzl#repository_rule) function to create a new -repository rule and store it in a global variable. +[repository_rule](/rules/lib/globals/bzl#repository_rule) function to define a +new repo rule and store it in a global variable. After a repo rule is defined, +it can be invoked as a function to define repos. This invocation is usually +performed from inside a [module extension](/external/extension) implementation +function. -A custom repository rule can be used just like a native repository rule. It -has a mandatory `name` attribute and every target present in its build files -can be referred as `@//package:target` where `` is the value of the -`name` attribute. - -The rule is loaded when you explicitly build it, or if it is a dependency of -the build. In this case, Bazel will execute its `implementation` function. This -function describe how to create the repository, its content and `BUILD` files. +The two major components of a repo rule definition are its attribute schema and +implementation function. The attribute schema determines the names and types of +attributes passed to a repo rule invocation, and the implementation function is +run when the repo needs to be fetched. ## Attributes -Attribute are rule arguments passed as a dict to the `attrs` rule argument. -The attributes and their types are defined are listed when you define a -repository rule. An example definining `url` and `sha256` attributes as -strings: +Attributes are arguments passed to the repo rule invocation. The schema of +attributes accepted by a repo rule is specified using the `attrs` argument when +the repo rule is defined with a call to `repository_rule`. An example defining +`url` and `sha256` attributes as strings: ```python -local_repository = repository_rule( +http_archive = repository_rule( implementation=_impl, - local=True, attrs={ - "url": attr.string(mandatory=True) - "sha256": attr.string(mandatory=True) + "url": attr.string(mandatory=True), + "sha256": attr.string(mandatory=True), } ) ``` @@ -57,35 +56,29 @@ def _impl(repository_ctx): checksum = repository_ctx.attr.sha256 ``` -All `repository_rule`s have implicitly defined attributes (just like build -rules). The two implicit attributes are `name` (just like for build rules) and -`repo_mapping`. The name of a repository rule is accessible with -`repository_ctx.name`. The meaning of `repo_mapping` is the same as for the -native repository rules -[`local_repository`](https://bazel.build/reference/be/workspace#local_repository.repo_mapping) -and -[`new_local_repository`](https://bazel.build/reference/be/workspace#new_local_repository.repo_mapping). - -If an attribute name starts with `_` it is private and users cannot set it. +All `repository_rule`s have the implicitly defined attribute `name`. This is a +string attribute that behaves somewhat magically: when specified as an input to +a repo rule invocation, it takes an apparent repo name; but when read from the +repo rule's implementation function using `repository_ctx.attr.name`, it returns +the canonical repo name. ## Implementation function -Every repository rule requires an `implementation` function. It contains the -actual logic of the rule and is executed strictly in the Loading Phase. +Every repo rule requires an `implementation` function. It contains the actual +logic of the rule and is executed strictly in the Loading Phase. The function has exactly one input parameter, `repository_ctx`. The function returns either `None` to signify that the rule is reproducible given the specified parameters, or a dict with a set of parameters for that rule that -would turn that rule into a reproducible one generating the same repository. For +would turn that rule into a reproducible one generating the same repo. For example, for a rule tracking a git repository that would mean returning a specific commit identifier instead of a floating branch that was originally specified. -The input parameter `repository_ctx` can be used to -access attribute values, and non-hermetic functions (finding a binary, -executing a binary, creating a file in the repository or downloading a file -from the Internet). See [the library](/rules/lib/builtins/repository_ctx) for more -context. Example: +The input parameter `repository_ctx` can be used to access attribute values, and +non-hermetic functions (finding a binary, executing a binary, creating a file in +the repository or downloading a file from the Internet). See [the API +docs](/rules/lib/builtins/repository_ctx) for more context. Example: ```python def _impl(repository_ctx): @@ -98,86 +91,72 @@ local_repository = repository_rule( ## When is the implementation function executed? -The implementation function of a repository is executed when Bazel needs a -target from that repository, for example when another target (in another -repository) depends on it or if it is mentioned on the commmand line. The -implementation function is then expected to create the repository in the file -system. This is called "fetching" the repository. - -In contrast to regular targets, repositories are not necessarily re-fetched when -something changes that would cause the repository to be different. This is -because there are things that Bazel either cannot detect changes to or it would -cause too much overhead on every build (for example, things that are fetched -from the network). Therefore, repositories are re-fetched only if one of the -following things changes: - -* The parameters passed to the declaration of the repository in the - `WORKSPACE` file. -* The Starlark code comprising the implementation of the repository. -* The value of any environment variable passed to `repository_ctx`'s - `getenv()` method or declared with the `environ` attribute of the - [`repository_rule`](/rules/lib/globals/bzl#repository_rule). The values - of these environment variables can be hard-wired on the command line with the - [`--repo_env`](/reference/command-line-reference#flag--repo_env) flag. -* The content of any file passed to the `read()`, `execute()` and similar - methods of `repository_ctx` which is referred to by a label (for example, - `//mypkg:label.txt` but not `mypkg/label.txt`) -* When `bazel sync` is executed. +The implementation function of a repo rule is executed when Bazel needs a target +from that repository, for example when another target (in another repo) depends +on it or if it is mentioned on the command line. The implementation function is +then expected to create the repo in the file system. This is called "fetching" +the repo. + +In contrast to regular targets, repos are not necessarily re-fetched when +something changes that would cause the repo to be different. This is because +there are things that Bazel either cannot detect changes to or it would cause +too much overhead on every build (for example, things that are fetched from the +network). Therefore, repos are re-fetched only if one of the following things +changes: + +* The attributes passed to the repo rule invocation. +* The Starlark code comprising the implementation of the repo rule. +* The value of any environment variable passed to `repository_ctx`'s + `getenv()` method or declared with the `environ` attribute of the + [`repository_rule`](/rules/lib/globals/bzl#repository_rule). The values of + these environment variables can be hard-wired on the command line with the + [`--repo_env`](/reference/command-line-reference#flag--repo_env) flag. +* The existence, contents, and type of any paths being + [`watch`ed](/rules/lib/builtins/repository_ctx#watch) in the implementation + function of the repo rule. + * Certain other methods of `repository_ctx` with a `watch` parameter, such + as `read()`, `execute()`, and `extract()`, can also cause paths to be + watched. + * Similarly, [`repository_ctx.watch_tree`](/rules/lib/builtins/repository_ctx#watch_tree) + and [`path.readdir`](/rules/lib/builtins/path#readdir) can cause paths + to be watched in other ways. +* When `bazel fetch --force` is executed. There are two parameters of `repository_rule` that control when the repositories are re-fetched: -* If the `configure` flag is set, the repository is only re-fetched on - `bazel sync` when the` --configure` parameter is passed to it (if the - attribute is unset, this command will not cause a re-fetch) -* If the `local` flag is set, in addition to the above cases, the repository is - also re-fetched when the Bazel server restarts or when any file that affects - the declaration of the repository changes (e.g. the `WORKSPACE` file or a file - it loads), regardless of whether the changes resulted in a change to the - declaration of the repository or its code. - - Non-local repositories are not re-fetched in these cases. This is because - these repositories are assumed to talk to the network or be otherwise - expensive. - -## Restarting the implementation function - -The implementation function can be _restarted_ while a repository is being -fetched if a dependency it requests is _missing_. In that case, the execution of -the implementation function will stop, the missing dependency is resolved and -the function will be re-executed after the dependency has been resolved. To -avoid unnecessary restarts (which are expensive, as network access might -have to be repeated), label arguments are prefetched, provided all -label arguments can be resolved to an existing file. Note that resolving -a path from a string or a label that was constructed only during execution -of the function might still cause a restart. - -## Forcing refetch of external repositories - -Sometimes, an external repository can become outdated without any change to its -definition or dependencies. For example, a repository fetching sources might -follow a particular branch of a third-party repository, and new commits are -available on that branch. In this case, you can ask bazel to refetch all -external repositories unconditionally by calling `bazel sync`. - -Moreover, some rules inspect the local machine and might become -outdated if the local machine was upgraded. Here you can ask bazel to -only refetch those external repositories where the -[`repository_rule`](/rules/lib/globals#repository_rule) -definition has the `configure` attribute set, use `bazel sync --configure`. +* If the `configure` flag is set, the repository is re-fetched on `bazel + fetch --force --configure` (non-`configure` repositories are not + re-fetched). +* If the `local` flag is set, in addition to the above cases, the repo is also + re-fetched when the Bazel server restarts. + +## Forcing refetch of external repos + +Sometimes, an external repo can become outdated without any change to its +definition or dependencies. For example, a repo fetching sources might follow a +particular branch of a third-party repository, and new commits are available on +that branch. In this case, you can ask bazel to refetch all external repos +unconditionally by calling `bazel fetch --force --all`. +Moreover, some repo rules inspect the local machine and might become outdated if +the local machine was upgraded. Here you can ask Bazel to only refetch those +external repos where the [`repository_rule`](/rules/lib/globals#repository_rule) +definition has the `configure` attribute set, use `bazel fetch --force +--configure`. ## Examples -- [C++ auto-configured toolchain](https://cs.opensource.google/bazel/bazel/+/master:tools/cpp/cc_configure.bzl;drc=644b7d41748e09eff9e47cbab2be2263bb71f29a;l=176): -it uses a repository rule to automatically create the -C++ configuration files for Bazel by looking for the local C++ compiler, the -environment and the flags the C++ compiler supports. +- [C++ auto-configured + toolchain](https://cs.opensource.google/bazel/bazel/+/master:tools/cpp/cc_configure.bzl;drc=644b7d41748e09eff9e47cbab2be2263bb71f29a;l=176): + it uses a repo rule to automatically create the C++ configuration files for + Bazel by looking for the local C++ compiler, the environment and the flags + the C++ compiler supports. -- [Go repositories](https://github.com/bazelbuild/rules_go/blob/67bc217b6210a0922d76d252472b87e9a6118fdf/go/private/go_repositories.bzl#L195) - uses several `repository_rule` to defines the list of dependencies - needed to use the Go rules. +- [Go repositories](https://github.com/bazelbuild/rules_go/blob/67bc217b6210a0922d76d252472b87e9a6118fdf/go/private/go_repositories.bzl#L195) + uses several `repository_rule` to defines the list of dependencies needed to + use the Go rules. -- [rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external) creates - an external repository called `@maven` by default that generates build targets - for every Maven artifact in the transitive dependency tree. +- [rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external) + creates an external repository called `@maven` by default that generates + build targets for every Maven artifact in the transitive dependency tree. \ No newline at end of file diff --git a/site/en/extending/toolchains.md b/site/en/extending/toolchains.md index 6e9fb91d825352..6dac718713a025 100644 --- a/site/en/extending/toolchains.md +++ b/site/en/extending/toolchains.md @@ -430,7 +430,7 @@ be able to run on the execution platform. At this point all the building blocks are assembled, and you just need to make the toolchains available to Bazel's resolution procedure. This is done by -registering the toolchain, either in a `WORKSPACE` file using +registering the toolchain, either in a `MODULE.bazel` file using `register_toolchains()`, or by passing the toolchains' labels on the command line using the `--extra_toolchains` flag. @@ -495,10 +495,11 @@ selected toolchain for each toolchain type as well as a selected execution platform for the current target. The available execution platforms and toolchains are gathered from the -`WORKSPACE` file via -[`register_execution_platforms`](/rules/lib/globals/workspace#register_execution_platforms) +external dependency graph via +[`register_execution_platforms`](/rules/lib/globals/module#register_execution_platforms) and -[`register_toolchains`](/rules/lib/globals/workspace#register_toolchains). +[`register_toolchains`](/rules/lib/globals/module#register_toolchains) calls in +`MODULE.bazel`` files. Additional execution platforms and toolchains may also be specified on the command line via [`--extra_execution_platforms`](/reference/command-line-reference#flag--extra_execution_platforms) @@ -511,10 +512,19 @@ with preference given to earlier items in the list. The set of available toolchains, in priority order, is created from `--extra_toolchains` and `register_toolchains`: -1. Toolchains registered using `--extra_toolchains` are added first. - 1. Within these, the **last** toolchain has highest priority. -2. Toolchains registered using `register_toolchains` - 1. Within these, the **first** mentioned toolchain has highest priority. +1. Toolchains registered using `--extra_toolchains` are added first. (Within + these, the **last** toolchain has highest priority.) +2. Toolchains registered using `register_toolchains` in the transitive external + dependency graph, in the following order: (Within these, the **first** + mentioned toolchain has highest priority.) + 1. Toolchains registered by the root module (as in, the `MODULE.bazel` at the + workspace root); + 2. Toolchains registered in the user's `WORKSPACE` file, including in any + macros invoked from there; + 3. Toolchains registered by non-root modules (as in, dependencies specified by + the root module, and their dependencies, and so forth); + 4. Toolchains registered in the "WORKSPACE suffix"; this is only used by + certain native rules bundled with the Bazel installation. **NOTE:** [Pseudo-targets like `:all`, `:*`, and `/...`](/run/build#specifying-build-targets) are ordered by Bazel's package diff --git a/site/en/external/module.md b/site/en/external/module.md index 4bf6dee1832c37..fa8ba89190ca3e 100644 --- a/site/en/external/module.md +++ b/site/en/external/module.md @@ -10,10 +10,9 @@ which publishes metadata about other modules that it depends on. This is analogous to familiar concepts in other dependency management systems, such as a Maven *artifact*, an npm *package*, a Go *module*, or a Cargo *crate*. -A module must have a `MODULE.bazel` file at its repo root (next to the -`WORKSPACE` file). This file is the module's manifest, declaring its name, -version, list of direct dependencies, and other information. For a basic -example: +A module must have a `MODULE.bazel` file at its repo root. This file is the +module's manifest, declaring its name, version, list of direct dependencies, and +other information. For a basic example: ```python module(name = "my-module", version = "1.0") @@ -22,7 +21,8 @@ bazel_dep(name = "rules_cc", version = "0.0.1") bazel_dep(name = "protobuf", version = "3.19.0") ``` -See the [full list](/rules/lib/globals/module) of directives available in `MODULE.bazel` files. +See the [full list](/rules/lib/globals/module) of directives available in +`MODULE.bazel` files. To perform module resolution, Bazel starts by reading the root module's `MODULE.bazel` file, and then repeatedly requests any dependency's diff --git a/site/en/external/overview.md b/site/en/external/overview.md index 3ea4a449430dc5..e24891fc7ce708 100644 --- a/site/en/external/overview.md +++ b/site/en/external/overview.md @@ -38,9 +38,13 @@ directory. The repository in which the current Bazel command is being run. +The root of the main repository is also known as the +**workspace root**. + ### Workspace {:#workspace} -The environment shared by all Bazel commands run in the same main repository. +The environment shared by all Bazel commands run in the same main repository. It +encompasses the main repo and the set of all defined external repos. Note that historically the concepts of "repository" and "workspace" have been conflated; the term "workspace" has often been used to refer to the main diff --git a/site/en/external/registry.md b/site/en/external/registry.md index 2a064df2740ba2..ab27efd2c88b0d 100644 --- a/site/en/external/registry.md +++ b/site/en/external/registry.md @@ -21,7 +21,15 @@ An index registry must follow the format below: * `/bazel_registry.json`: A JSON file containing metadata for the registry like: - * `mirrors`: specifying the list of mirrors to use for source archives + * `mirrors`: specifying the list of mirrors to use for source archives. + The mirrored URL is a concatenation of the mirror itself, and the + source URL of the module specified by its `source.json` file sans the + protocol. For example, if a module's source URL is + `https://foo.com/bar/baz`, and `mirrors` contains + `["https://mirror1.com/", "https://example.com/mirror2/"]`, then the + URLs Bazel will try in order are `https://mirror1.com/foo.com/bar/baz`, + `https://example.com/mirror2/foo.com/bar/baz`, and finally the original + source URL itself `https://foo.com/bar/baz`. * `module_base_path`: specifying the base path for modules with `local_repository` type in the `source.json` file * `/modules`: A directory containing a subdirectory for each module in this diff --git a/site/en/query/language.md b/site/en/query/language.md index d015d36da6b557..b53e7678b70ee0 100644 --- a/site/en/query/language.md +++ b/site/en/query/language.md @@ -1504,24 +1504,8 @@ full syntax for ### Querying with external repositories {:#querying-external-repositories} -If the build depends on rules from external repositories (defined in the -WORKSPACE file) then query results will include these dependencies. For -example, if `//foo:bar` depends on `//external:some-lib` -and `//external:some-lib` is bound to `@other-repo//baz:lib`, then -`bazel query 'deps(//foo:bar)'` will list both `@other-repo//baz:lib` and -`//external:some-lib` as dependencies. - -External repositories themselves are not dependencies of a build. That is, in -the example above, `//external:other-repo` is not a dependency. It -can be queried for as a member of the `//external` package, though, -for example: - -``` - # Querying over all members of //external returns the repository. - bazel query 'kind(http_archive, //external:*)' - //external:other-repo - - # ...but the repository is not a dependency. - bazel query 'kind(http_archive, deps(//foo:bar))' - INFO: Empty results -``` +If the build depends on rules from [external repositories](/external/overview) +then query results will include these dependencies. For +example, if `//foo:bar` depends on `@other-repo//baz:lib`, then +`bazel query 'deps(//foo:bar)'` will list `@other-repo//baz:lib` as a +dependency. diff --git a/site/en/query/quickstart.md b/site/en/query/quickstart.md index f8e5bc6c5e5b99..329d336ba2c97d 100644 --- a/site/en/query/quickstart.md +++ b/site/en/query/quickstart.md @@ -74,7 +74,7 @@ bazelqueryguide │ │ ├── Review.java │ │ └── BUILD │ └── Runner.java -└── WORKSPACE +└── MODULE.bazel ``` Throughout this tutorial, unless directed otherwise, try not to look in the `BUILD` files to find the information you need and instead solely use the query function. diff --git a/site/en/remote/output-directories.md b/site/en/remote/output-directories.md index 49d42a94e4c9da..dd0a2bec6d412c 100644 --- a/site/en/remote/output-directories.md +++ b/site/en/remote/output-directories.md @@ -26,9 +26,10 @@ Requirements for an output directory layout: The solution that's currently implemented: -* Bazel must be invoked from a directory containing a WORKSPACE file (the - "_workspace directory_"), or a subdirectory thereof. It reports an error if it - is not. +* Bazel must be invoked from a directory containing a repo boundary file, or a + subdirectory thereof. In other words, Bazel must be invoked from inside a + [repository](../external/overview#repository). Otherwise, an error is + reported. * The _outputRoot_ directory defaults to `${XDG_CACHE_HOME}/bazel` (or `~/.cache/bazel`, if the `XDG_CACHE_HOME` environment variable is not set) on Linux, `/private/var/tmp` on macOS, and on Windows it defaults to `%HOME%` if @@ -43,11 +44,11 @@ The solution that's currently implemented: installation manifest. * Beneath the `outputUserRoot` directory, an `outputBase` directory is also created whose name is the MD5 hash of the path name of the workspace - directory. So, for example, if Bazel is running in the workspace directory + root. So, for example, if Bazel is running in the workspace root `/home/user/src/my-project` (or in a directory symlinked to that one), then an output base directory is created called: `/home/user/.cache/bazel/_bazel_user/7ffd56a6e4cb724ea575aba15733d113`. You - can also run `echo -n $(pwd) | md5sum` in a Bazel workspace to get the MD5. + can also run `echo -n $(pwd) | md5sum` in the workspace root to get the MD5. * You can use Bazel's `--output_base` startup option to override the default output base directory. For example, `bazel --output_base=/tmp/bazel/output build x/y:z`. @@ -59,14 +60,14 @@ The symlinks for "bazel-<workspace-name>", "bazel-out", "bazel-testlogs", and "bazel-bin" are put in the workspace directory; these symlinks point to some directories inside a target-specific directory inside the output directory. These symlinks are only for the user's convenience, as Bazel itself does not -use them. Also, this is done only if the workspace directory is writable. +use them. Also, this is done only if the workspace root is writable. ## Layout diagram {:#layout-diagram} The directories are laid out as follows: 
-<workspace-name>/                         <== The workspace directory
+<workspace-name>/                         <== The workspace root
   bazel-my-project => <...my-project>     <== Symlink to execRoot
   bazel-out => <...bin>                   <== Convenience symlink to outputPath
   bazel-bin => <...bin>                   <== Convenience symlink to most recent written bin dir $(BINDIR)
@@ -80,7 +81,7 @@ The directories are laid out as follows:
         _embedded_binaries/               <== Contains binaries and scripts unpacked from the data section of
                                               the bazel executable on first run (such as helper scripts and the
                                               main Java file BazelServer_deploy.jar)
-    7ffd56a6e4cb724ea575aba15733d113/     <== Hash of the client's workspace directory (such as
+    7ffd56a6e4cb724ea575aba15733d113/     <== Hash of the client's workspace root (such as
                                               /home/user/src/my-project): outputBase
       action_cache/                       <== Action cache directory hierarchy
                                               This contains the persistent record of the file
@@ -103,7 +104,7 @@ The directories are laid out as follows:
                                               are created, are intentionally hidden from the action.
                                               Every action can access its inputs and outputs relative
                                               to the execroot directory.
-        <workspace-name>/                 <== Working tree for the Bazel build & root of symlink forest: execRoot
+        _main/                            <== Working tree for the Bazel build & root of symlink forest: execRoot
           _bin/                           <== Helper tools are linked from or copied to here.
 
           bazel-out/                      <== All actual output of the build is under here: outputPath
@@ -117,7 +118,7 @@ The directories are laid out as follows:
                                               //foo/bar:baz
                 foo/bar/baz.runfiles/     <== The runfiles symlink farm for the //foo/bar:baz executable.
                   MANIFEST
-                  <workspace-name>/
+                  _main/
                     ...
               genfiles/                   <== Bazel puts generated source for the target configuration here:
                                               $(GENDIR)
diff --git a/site/en/start/java.md b/site/en/start/java.md
index 56e904a9000d27..459cae0e661a99 100644
--- a/site/en/start/java.md
+++ b/site/en/start/java.md
@@ -70,7 +70,7 @@ java-tutorial
 │                   │   └── Runner.java
 │                   ├── Greeting.java
 │                   └── ProjectRunner.java
-└── WORKSPACE
+└── MODULE.bazel
 ```
 
 ## Build with Bazel
@@ -81,7 +81,7 @@ Before you can build a project, you need to set up its workspace. A workspace is
 a directory that holds your project's source files and Bazel's build outputs. It
 also contains files that Bazel recognizes as special:
 
-*  The `WORKSPACE` file, which identifies the directory and its contents as a
+*  The `MODULE.bazel` file, which identifies the directory and its contents as a
    Bazel workspace and lives at the root of the project's directory structure,
 
 *  One or more `BUILD` files, which tell Bazel how to build different parts of
@@ -89,7 +89,7 @@ also contains files that Bazel recognizes as special:
    is a *package*. You will learn about packages later in this tutorial.)
 
 To designate a directory as a Bazel workspace, create an empty file named
-`WORKSPACE` in that directory.
+`MODULE.bazel` in that directory.
 
 When Bazel builds the project, all inputs and dependencies must be in the same
 workspace. Files residing in different workspaces are independent of one
diff --git a/src/main/java/com/google/devtools/build/lib/starlarkbuildapi/repository/RepositoryModuleApi.java b/src/main/java/com/google/devtools/build/lib/starlarkbuildapi/repository/RepositoryModuleApi.java
index 1bd5e6f5fdbc5c..aace92ac766fc1 100644
--- a/src/main/java/com/google/devtools/build/lib/starlarkbuildapi/repository/RepositoryModuleApi.java
+++ b/src/main/java/com/google/devtools/build/lib/starlarkbuildapi/repository/RepositoryModuleApi.java
@@ -40,8 +40,10 @@ public interface RepositoryModuleApi {
   @StarlarkMethod(
       name = "repository_rule",
       doc =
-          "Creates a new repository rule. Store it in a global value, so that it can be loaded and "
-              + "called from the WORKSPACE file.",
+          "Creates a new repository rule. Store it in a global value, so that it can be loaded and"
+              + " called from a module extension"
+              + " implementation function, or used by use_repo_rule.",
       parameters = {
         @Param(
             name = "implementation",
@@ -132,7 +134,8 @@ StarlarkCallable repositoryRule(
       name = "module_extension",
       doc =
           "Creates a new module extension. Store it in a global value, so that it can be exported"
-              + " and used in a MODULE.bazel file.",
+              + " and used in a MODULE.bazel file with use_extension.",
       parameters = {
         @Param(
             name = "implementation",
diff --git a/src/test/tools/bzlmod/MODULE.bazel.lock b/src/test/tools/bzlmod/MODULE.bazel.lock
index 555b1e58d4f790..0f1004e77937c9 100644
--- a/src/test/tools/bzlmod/MODULE.bazel.lock
+++ b/src/test/tools/bzlmod/MODULE.bazel.lock
@@ -108,7 +108,7 @@
     },
     "@@rules_jvm_external~//:extensions.bzl%maven": {
       "general": {
-        "bzlTransitiveDigest": "nCJBnRI68ndNaTFBoEY+PPV6B9hLz1V4M3YjjxCQFhk=",
+        "bzlTransitiveDigest": "VW3qd5jCZXYbR9xpSwrhGQ04GCmEIIFPVERY34HHvFE=",
         "usagesDigest": "Ibxz+lEeSRsbUXF0hvMKg1hWMFUL0S0/LBkSsDDs6lg=",
         "recordedFileInputs": {
           "@@rules_jvm_external~//rules_jvm_external_deps_install.json": "10442a5ae27d9ff4c2003e5ab71643bf0d8b48dcf968b4173fa274c3232a8c06"
@@ -1132,7 +1132,7 @@
     },
     "@@rules_jvm_external~//:non-module-deps.bzl%non_module_deps": {
       "general": {
-        "bzlTransitiveDigest": "Lxr/sC3jorQDfvqpluvFC7JhOi41kaZ6ESxDVC2LZ5s=",
+        "bzlTransitiveDigest": "ZOivBbbZUakRexeLO/N26oX4Bcph6HHnqNmfxt7yoCc=",
         "usagesDigest": "hwl83iGGV39M7yBqqlvKbiCEgo0wLkrXn8FcWAMOt1A=",
         "recordedFileInputs": {},
         "recordedDirentsInputs": {},
@@ -1160,7 +1160,7 @@
     },
     "@@rules_python~//python/extensions:python.bzl%python": {
       "general": {
-        "bzlTransitiveDigest": "M7E/O6D+cJQoC4ZmcJ+cmXXJT2JMff2FRpQyHtoDQCQ=",
+        "bzlTransitiveDigest": "lbXqTyC4ahBb81TIrIp+2d3sWnlurVNqSeAaLJknLUs=",
         "usagesDigest": "sdqp9yj+PY7j/QqeodNDzIPDsgW02WGMxb76v6YZfBM=",
         "recordedFileInputs": {},
         "recordedDirentsInputs": {},
@@ -1190,7 +1190,7 @@
     },
     "@@rules_python~//python/extensions/private:internal_deps.bzl%internal_deps": {
       "general": {
-        "bzlTransitiveDigest": "X9KNWFFpQp2+YidRYA7/EpZiRW/JtKz7Qh788dYYBlM=",
+        "bzlTransitiveDigest": "b6FMQSdoZ1QOssw14AW8bWDn2BvywI4FVkLbO2nTMsE=",
         "usagesDigest": "ckBX0TASrGH+IURVs99U68Ub17sjGKe25FfdEE51DWg=",
         "recordedFileInputs": {},
         "recordedDirentsInputs": {},
diff --git a/tools/build_defs/repo/http.bzl b/tools/build_defs/repo/http.bzl
index ebb01b4690c97d..b640fcc47b856c 100644
--- a/tools/build_defs/repo/http.bzl
+++ b/tools/build_defs/repo/http.bzl
@@ -22,19 +22,24 @@
 
 ### Setup
 
-To use these rules, load them in your `WORKSPACE` file as follows:
+To use these rules in a module extension, load them in your .bzl file and then call them from your
+extension's implementation function. For example, to use `http_archive`:
 
 ```python
-load(
-    "@bazel_tools//tools/build_defs/repo:http.bzl",
-    "http_archive",
-    "http_file",
-    "http_jar",
-)
+load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
+
+def _my_extension_impl(mctx):
+  http_archive(name = "foo", urls = [...])
+
+my_extension = module_extension(implementation = _my_extension_impl)
 ```
 
-These rules are improved versions of the native http rules and will eventually
-replace the native rules.
+Alternatively, you can directly call these repo rules in your MODULE.bazel file with
+`use_repo_rule`:
+
+```python
+http_archive = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
+http_archive(name = "foo", urls = [...])
 """
 
 load(