Allow git_override to specify pathspec for --recurse_submodules

[jacky8hyf]

Description of the feature request:

The feature request

We request to add an attribute to git_override:

git_override(
    module_name = "kleaf",
    remote = "https://android.googlesource.com/kernel/superproject",
    commit = "1c03b74f03bb7a45856b8e131a857c238af12033",
    init_submodules = True,
    recurse_submodules_pathspec = ":(attr:ddk)"
)

... or any solution that can allow us to pull only submodules with the given attribute to achieve the behavior of git clone --recurse-submodules=':(attr:ddk)' ...

Which category does this issue belong to?

External Dependency

What underlying problem are you trying to solve with this feature?

Background
We (the Kleaf team) are working on a feature that needs to pull a subtree of a source tree as a module that builds a build number on ci.android.com. The way we design it is the following:

• From ci.android.com, for a given build number, infer the superproject SHA of the kernel superproject
• Add the following to MODULE.bazel:

git_override(
    module_name = "kleaf",
    remote = "https://android.googlesource.com/kernel/superproject",
    commit = "1c03b74f03bb7a45856b8e131a857c238af12033",
    init_submodules = True,
)

However, this will pull the whole superproject. We would like to be able to pull only projects with certain .gitattributes (example. These git attributes were originally from the groups of a repo manifest (example)

So, to pull groups=ddk projects, with repo we would do

repo init --groups=ddk ...

With Git submodules we would do

git clone --recurse-submodules=':(attr:ddk)' ...

However, there is no way to specify --recurse-submodules=<pathspec> in git_override.

Alternatives

One may suggest us to use use_repo_rule and implement our own repo_rule instead. However, I need a full module so that bazel will read its MODULE.bazel and resolve the dependencies, not just a single repository.

We also come up with some hacky alternative solutions, example:

git_override(
    module_name = "kleaf",
    remote = "https://android.googlesource.com/kernel/superproject",
    commit = "1c03b74f03bb7a45856b8e131a857c238af12033",
    patch_cmds = [
        "tools/filter_submodules.sh -g ddk",
    ],
)

where the tools/filter_submodules.sh contains:

GROUP=ddk # <value of -g, default to ddk>

git rm -r $(git check-attr $GROUP -- $(git submodule status | cut -f2 -d' ') | \
    grep -v "$GROUP: set" | cut -f1 -d':')

git commit -am'Remove unrelated projects'

git submodule update --init --recursive

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 7.0.2

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

Git pathspecs: https://css-tricks.com/git-pathspecs-and-how-to-use-them/#aa-attr
Git submodules: https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---recurse-submodulesltpathspecgt

Any other information, logs, or outputs that you want to share?

No response

[meteorcloudy]

@jacky8hyf It's hard for our team to make time for this, do you mind submitting a PR for this? Thanks!

[jacky8hyf]

Thanks for the reply. Let me evaluate the importance of this feature to our design and properly prioritize this.