Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase to v2.47.0 #5196

Merged
merged 427 commits into from
Oct 8, 2024
Merged
This pull request is big! We’re only showing the most recent 250 commits.

Commits on Oct 8, 2024

  1. path-walk: introduce an object walk by path

    In anticipation of a few planned applications, introduce the most basic form
    of a path-walk API. It currently assumes that there are no UNINTERESTING
    objects, and does not include any complicated filters. It calls a function
    pointer on groups of tree and blob objects as grouped by path. This only
    includes objects the first time they are discovered, so an object that
    appears at multiple paths will not be included in two batches.
    
    There are many future adaptations that could be made, but they are left for
    future updates when consumers are ready to take advantage of those features.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    685d130 View commit details
    Browse the repository at this point in the history
  2. t6601: add helper for testing path-walk API

    Add some tests based on the current behavior, doing interesting checks
    for different sets of branches, ranges, and the --boundary option. This
    sets a baseline for the behavior and we can extend it as new options are
    introduced.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f10a957 View commit details
    Browse the repository at this point in the history
  3. path-walk: allow consumer to specify object types

    We add the ability to filter the object types in the path-walk API so
    the callback function is called fewer times.
    
    This adds the ability to ask for the commits in a list, as well. Future
    changes will add the ability to visit annotated tags.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    118b06d View commit details
    Browse the repository at this point in the history
  4. path-walk: allow visiting tags

    In anticipation of using the path-walk API to analyze tags or include
    them in a pack-file, add the ability to walk the tags that were included
    in the revision walk.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    731998c View commit details
    Browse the repository at this point in the history
  5. revision: create mark_trees_uninteresting_dense()

    The sparse tree walk algorithm was created in d5d2e93 (revision:
    implement sparse algorithm, 2019-01-16) and involves using the
    mark_trees_uninteresting_sparse() method. This method takes a repository
    and an oidset of tree IDs, some of which have the UNINTERESTING flag and
    some of which do not.
    
    Create a method that has an equivalent set of preconditions but uses a
    "dense" walk (recursively visits all reachable trees, as long as they
    have not previously been marked UNINTERESTING). This is an important
    difference from mark_tree_uninteresting(), which short-circuits if the
    given tree has the UNINTERESTING flag.
    
    A use of this method will be added in a later change, with a condition
    set whether the sparse or dense approach should be used.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    9e7da31 View commit details
    Browse the repository at this point in the history
  6. path-walk: add prune_all_uninteresting option

    This option causes the path-walk API to act like the sparse tree-walk
    algorithm implemented by mark_trees_uninteresting_sparse() in
    list-objects.c.
    
    Starting from the commits marked as UNINTERESTING, their root trees and
    all objects reachable from those trees are UNINTERSTING, at least as we
    walk path-by-path. When we reach a path where all objects associated
    with that path are marked UNINTERESTING, then do no continue walking the
    children of that path.
    
    We need to be careful to pass the UNINTERESTING flag in a deep way on
    the UNINTERESTING objects before we start the path-walk, or else the
    depth-first search for the path-walk API may accidentally report some
    objects as interesting.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ec1a5d4 View commit details
    Browse the repository at this point in the history
  7. pack-objects: extract should_attempt_deltas()

    This will be helpful in a future change.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    4578489 View commit details
    Browse the repository at this point in the history
  8. pack-objects: add --path-walk option

    In order to more easily compute delta bases among objects that appear at the
    exact same path, add a --path-walk option to 'git pack-objects'.
    
    This option will use the path-walk API instead of the object walk given by
    the revision machinery. Since objects will be provided in batches
    representing a common path, those objects can be tested for delta bases
    immediately instead of waiting for a sort of the full object list by
    name-hash. This has multiple benefits, including avoiding collisions by
    name-hash.
    
    The objects marked as UNINTERESTING are included in these batches, so we
    are guaranteeing some locality to find good delta bases.
    
    After the individual passes are done on a per-path basis, the default
    name-hash is used to find other opportunistic delta bases that did not
    match exactly by the full path name.
    
    RFC TODO: It is important to note that this option is inherently
    incompatible with using a bitmap index. This walk probably also does not
    work with other advanced features, such as delta islands.
    
    Getting ahead of myself, this option compares well with --full-name-hash
    when the packfile is large enough, but also performs at least as well as
    the default in all cases that I've seen.
    
    RFC TODO: this should probably be recording the batch locations to another
    list so they could be processed in a second phase using threads.
    
    RFC TODO: list some examples of how this outperforms previous pack-objects
    strategies. (This is coming in later commits that include performance
    test changes.)
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    454cfdd View commit details
    Browse the repository at this point in the history
  9. pack-objects: introduce GIT_TEST_PACK_PATH_WALK

    There are many tests that validate whether 'git pack-objects' works as
    expected. Instead of duplicating these tests, add a new test environment
    variable, GIT_TEST_PACK_PATH_WALK, that implies --path-walk by default
    when specified.
    
    This was useful in testing the implementation of the --path-walk
    implementation, especially in conjunction with test such as:
    
     - t0411-clone-from-partial.sh : One test fetches from a repo that does
       not have the boundary objects. This causes the path-based walk to
       fail. Disable the variable for this test.
    
     - t5306-pack-nobase.sh : Similar to t0411, one test fetches from a repo
       without a boundary object.
    
     - t5310-pack-bitmaps.sh : One test compares the case when packing with
       bitmaps to the case when packing without them. Since we disable the
       test variable when writing bitmaps, this causes a difference in the
       object list (the --path-walk option adds an extra object). Specify
       --no-path-walk in both processes for the comparison. Another test
       checks for a specific delta base, but when computing dynamically
       without using bitmaps, the base object it too small to be considered
       in the delta calculations so no base is used.
    
     - t5316-pack-delta-depth.sh : This script cares about certain delta
       choices and their chain lengths. The --path-walk option changes how
       these chains are selected, and thus changes the results of this test.
    
     - t5322-pack-objects-sparse.sh : This demonstrates the effectiveness of
       the --sparse option and how it combines with --path-walk.
    
     - t5332-multi-pack-reuse.sh : This test verifies that the preferred
       pack is used for delta reuse when possible. The --path-walk option is
       not currently aware of the preferred pack at all, so finds a
       different delta base.
    
     - t7406-submodule-update.sh : When using the variable, the --depth
       option collides with the --path-walk feature, resulting in a warning
       message. Disable the variable so this warning does not appear.
    
    I want to call out one specific test change that is only temporary:
    
     - t5530-upload-pack-error.sh : One test cares specifically about an
       "unable to read" error message. Since the current implementation
       performs delta calculations within the path-walk API callback, a
       different "unable to get size" error message appears. When this
       is changed in a future refactoring, this test change can be reverted.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    82abe40 View commit details
    Browse the repository at this point in the history
  10. repack: add --path-walk option

    Since 'git pack-objects' supports a --path-walk option, allow passing it
    through in 'git repack'. This presents interesting testing opportunities for
    comparing the different repacking strategies against each other.
    
    Add the --path-walk option to the performance tests in p5313.
    
    For the microsoft/fluentui repo [1] checked out at a specific commit [2],
    the results are very interesting:
    
    Test                                           this tree
    ------------------------------------------------------------------
    5313.2: thin pack                              0.40(0.47+0.04)
    5313.3: thin pack size                                    1.2M
    5313.4: thin pack with --full-name-hash        0.09(0.10+0.04)
    5313.5: thin pack size with --full-name-hash             22.8K
    5313.6: thin pack with --path-walk             0.08(0.06+0.02)
    5313.7: thin pack size with --path-walk                  20.8K
    5313.8: big pack                               2.16(8.43+0.23)
    5313.9: big pack size                                    17.7M
    5313.10: big pack with --full-name-hash        1.42(3.06+0.21)
    5313.11: big pack size with --full-name-hash             18.0M
    5313.12: big pack with --path-walk             2.21(8.39+0.24)
    5313.13: big pack size with --path-walk                  17.8M
    5313.14: repack                                98.05(662.37+2.64)
    5313.15: repack size                                    449.1K
    5313.16: repack with --full-name-hash          33.95(129.44+2.63)
    5313.17: repack size with --full-name-hash              182.9K
    5313.18: repack with --path-walk               106.21(121.58+0.82)
    5313.19: repack size with --path-walk                   159.6K
    
    [1] https://github.com/microsoft/fluentui
    [2] e70848ebac1cd720875bccaa3026f4a9ed700e08
    
    This repo suffers from having a lot of paths that collide in the name
    hash, so examining them in groups by path leads to better deltas. Also,
    in this case, the single-threaded implementation is competitive with the
    full repack. This is saving time diffing files that have significant
    differences from each other.
    
    A similar, but private, repo has even more extremes in the thin packs:
    
    Test                                           this tree
    --------------------------------------------------------------
    5313.2: thin pack                              2.39(2.91+0.10)
    5313.3: thin pack size                                    4.5M
    5313.4: thin pack with --full-name-hash        0.29(0.47+0.12)
    5313.5: thin pack size with --full-name-hash             15.5K
    5313.6: thin pack with --path-walk             0.35(0.31+0.04)
    5313.7: thin pack size with --path-walk                  14.2K
    
    Notice, however, that while the --full-name-hash version is working
    quite well in these cases for the thin pack, it does poorly for some
    other standard cases, such as this test on the Linux kernel repository:
    
    Test                                           this tree
    --------------------------------------------------------------
    5313.2: thin pack                              0.01(0.00+0.00)
    5313.3: thin pack size                                     310
    5313.4: thin pack with --full-name-hash        0.00(0.00+0.00)
    5313.5: thin pack size with --full-name-hash              1.4K
    5313.6: thin pack with --path-walk             0.00(0.00+0.00)
    5313.7: thin pack size with --path-walk                    310
    
    Here, the --full-name-hash option does much worse than the default name
    hash, but the path-walk option does exactly as well.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f07075e View commit details
    Browse the repository at this point in the history
  11. pack-objects: enable --path-walk via config

    Users may want to enable the --path-walk option for 'git pack-objects' by
    default, especially underneath commands like 'git push' or 'git repack'.
    
    This should be limited to client repositories, since the --path-walk option
    disables bitmap walks, so would be bad to include in Git servers when
    serving fetches and clones. There is potential that it may be helpful to
    consider when repacking the repository, to take advantage of improved deltas
    across historical versions of the same files.
    
    Much like how "pack.useSparse" was introduced and included in
    "feature.experimental" before being enabled by default, use the repository
    settings infrastructure to make the new "pack.usePathWalk" config enabled by
    "feature.experimental" and "feature.manyFiles".
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    bf51d09 View commit details
    Browse the repository at this point in the history
  12. scalar: enable path-walk during push via config

    Repositories registered with Scalar are expected to be client-only
    repositories that are rather large. This means that they are more likely to
    be good candidates for using the --path-walk option when running 'git
    pack-objects', especially under the hood of 'git push'. Enable this config
    in Scalar repositories.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    2d533c9 View commit details
    Browse the repository at this point in the history
  13. pack-objects: refactor path-walk delta phase

    Previously, the --path-walk option to 'git pack-objects' would compute
    deltas inline with the path-walk logic. This would make the progress
    indicator look like it is taking a long time to enumerate objects, and
    then very quickly computed deltas.
    
    Instead of computing deltas on each region of objects organized by tree,
    store a list of regions corresponding to these groups. These can later
    be pulled from the list for delta compression before doing the "global"
    delta search.
    
    This presents a new progress indicator that can be used in tests to
    verify that this stage is happening.
    
    The current implementation is not integrated with threads, but could be
    done in a future update.
    
    Since we do not attempt to sort objects by size until after exploring
    all trees, we can remove the previous change to t5530 due to a different
    error message appearing first.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    2f07cd7 View commit details
    Browse the repository at this point in the history
  14. pack-objects: thread the path-based compression

    Adapting the implementation of ll_find_deltas(), create a threaded
    version of the --path-walk compression step in 'git pack-objects'.
    
    This involves adding a 'regions' member to the thread_params struct,
    allowing each thread to own a section of paths. We can simplify the way
    jobs are split because there is no value in extending the batch based on
    name-hash the way sections of the object entry array are attempted to be
    grouped. We re-use the 'list_size' and 'remaining' items for the purpose
    of borrowing work in progress from other "victim" threads when a thread
    has finished its batch of work more quickly.
    
    Using the Git repository as a test repo, the p5313 performance test
    shows that the resulting size of the repo is the same, but the threaded
    implementation gives gains of varying degrees depending on the number of
    objects being packed. (This was tested on a 16-core machine.)
    
    Test                                    HEAD~1    HEAD
    -------------------------------------------------------------
    5313.6: thin pack with --path-walk        0.01    0.01  +0.0%
    5313.7: thin pack size with --path-walk    475     475  +0.0%
    5313.12: big pack with --path-walk        1.99    1.87  -6.0%
    5313.13: big pack size with --path-walk  14.4M   14.3M  -0.4%
    5313.18: repack with --path-walk         98.14   41.46 -57.8%
    5313.19: repack size with --path-walk   197.2M  197.3M  +0.0%
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    cd6e3a6 View commit details
    Browse the repository at this point in the history
  15. path-walk API: avoid adding a root tree more than once

    When adding tree objects, we are very careful to avoid adding the same
    tree object more than once. There was one small gap in that logic,
    though: when adding a root tree object. Two refs can easily share the
    same root tree object, and we should still not add it more than once.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8b02eeb View commit details
    Browse the repository at this point in the history
  16. backfill: add builtin boilerplate

    In anticipation of implementing 'git backfill', populate the necessary files
    with the boilerplate of a new builtin.
    
    RFC TODO: When preparing this for a full implementation, make sure it is
    based on the newest standards introduced by [1].
    
    [1] https://lore.kernel.org/git/xmqqjzfq2f0f.fsf@gitster.g/T/#m606036ea2e75a6d6819d6b5c90e729643b0ff7f7
        [PATCH 1/3] builtin: add a repository parameter for builtin functions
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a288333 View commit details
    Browse the repository at this point in the history
  17. backfill: basic functionality and tests

    The default behavior of 'git backfill' is to fetch all missing blobs that
    are reachable from HEAD. Document and test this behavior.
    
    The implementation is a very simple use of the path-walk API, initializing
    the revision walk at HEAD to start the path-walk from all commits reachable
    from HEAD. Ignore the object arrays that correspond to tree entries,
    assuming that they are all present already.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e61870b View commit details
    Browse the repository at this point in the history
  18. backfill: add --batch-size=<n> option

    Users may want to specify a minimum batch size for their needs. This is only
    a minimum: the path-walk API provides a list of OIDs that correspond to the
    same path, and thus it is optimal to allow delta compression across those
    objects in a single server request.
    
    We could consider limiting the request to have a maximum batch size in the
    future.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    c7df6b4 View commit details
    Browse the repository at this point in the history
  19. backfill: add --sparse option

    One way to significantly reduce the cost of a Git clone and later fetches is
    to use a blobless partial clone and combine that with a sparse-checkout that
    reduces the paths that need to be populated in the working directory. Not
    only does this reduce the cost of clones and fetches, the sparse-checkout
    reduces the number of objects needed to download from a promisor remote.
    
    However, history investigations can be expensie as computing blob diffs will
    trigger promisor remote requests for one object at a time. This can be
    avoided by downloading the blobs needed for the given sparse-checkout using
    'git backfill' and its new '--sparse' mode, at a time that the user is
    willing to pay that extra cost.
    
    Note that this is distinctly different from the '--filter=sparse:<oid>'
    option, as this assumes that the partial clone has all reachable trees and
    we are using client-side logic to avoid downloading blobs outside of the
    sparse-checkout cone. This avoids the server-side cost of walking trees
    while also achieving a similar goal. It also downloads in batches based on
    similar path names, presenting a resumable download if things are
    interrupted.
    
    This augments the path-walk API to have a possibly-NULL 'pl' member that may
    point to a 'struct pattern_list'. This could be more general than the
    sparse-checkout definition at HEAD, but 'git backfill --sparse' is currently
    the only consumer.
    
    Be sure to test this in both cone mode and not cone mode. Cone mode has the
    benefit that the path-walk can skip certain paths once they would expand
    beyond the sparse-checkout.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    938bdf9 View commit details
    Browse the repository at this point in the history
  20. survey: stub in new experimental 'git-survey' command

    Start work on a new 'git survey' command to scan the repository
    for monorepo performance and scaling problems.  The goal is to
    measure the various known "dimensions of scale" and serve as a
    foundation for adding additional measurements as we learn more
    about Git monorepo scaling problems.
    
    The initial goal is to complement the scanning and analysis performed
    by the GO-based 'git-sizer' (https://github.com/github/git-sizer) tool.
    It is hoped that by creating a builtin command, we may be able to take
    advantage of internal Git data structures and code that is not
    accessible from GO to gain further insight into potential scaling
    problems.
    
    Co-authored-by: Derrick Stolee <stolee@gmail.com>
    Signed-off-by: Jeff Hostetler <git@jeffhostetler.com>
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    2 people authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6d2fb8b View commit details
    Browse the repository at this point in the history
  21. backfill: assume --sparse when sparse-checkout is enabled

    The previous change introduced the '--[no-]sparse' option for the 'git
    backfill' command, but did not assume it as enabled by default. However,
    this is likely the behavior that users will most often want to happen.
    Without this default, users with a small sparse-checkout may be confused
    when 'git backfill' downloads every version of every object in the full
    history.
    
    However, this is left as a separate change so this decision can be reviewed
    independently of the value of the '--[no-]sparse' option.
    
    Add a test of adding the '--sparse' option to a repo without sparse-checkout
    to make it clear that supplying it without a sparse-checkout is an error.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b8cebf0 View commit details
    Browse the repository at this point in the history
  22. survey: add command line opts to select references

    By default we will scan all references in "refs/heads/", "refs/tags/"
    and "refs/remotes/".
    
    Add command line opts let the use ask for all refs or a subset of them
    and to include a detached HEAD.
    
    Signed-off-by: Jeff Hostetler <git@jeffhostetler.com>
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    Jeff Hostetler authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    4e61f93 View commit details
    Browse the repository at this point in the history
  23. backfill: mark it as experimental

    This is a highly useful command, and we want it to get some testing "in
    the wild". However, the patches have not yet been reviewed on the Git
    mailing list, and are therefore subject to change. By marking the
    command as experimental, users will be warned to pay attention to those
    changes.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    51b8ae8 View commit details
    Browse the repository at this point in the history
  24. survey: start pretty printing data in table form

    When 'git survey' provides information to the user, this will be presented
    in one of two formats: plaintext and JSON. The JSON implementation will be
    delayed until the functionality is complete for the plaintext format.
    
    The most important parts of the plaintext format are headers specifying the
    different sections of the report and tables providing concreted data.
    
    Create a custom table data structure that allows specifying a list of
    strings for the row values. When printing the table, check each column for
    the maximum width so we can create a table of the correct size from the
    start.
    
    The table structure is designed to be flexible to the different kinds of
    output that will be implemented in future changes.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3c9a3dd View commit details
    Browse the repository at this point in the history
  25. survey: add object count summary

    At the moment, nothing is obvious about the reason for the use of the
    path-walk API, but this will become more prevelant in future iterations. For
    now, use the path-walk API to sum up the counts of each kind of object.
    
    For example, this is the reachable object summary output for my local repo:
    
    REACHABLE OBJECT SUMMARY
    ========================
    Object Type |  Count
    ------------+-------
           Tags |   1343
        Commits | 179344
          Trees | 314350
          Blobs | 184030
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    dc4a5f7 View commit details
    Browse the repository at this point in the history
  26. survey: summarize total sizes by object type

    Now that we have explored objects by count, we can expand that a bit more to
    summarize the data for the on-disk and inflated size of those objects. This
    information is helpful for diagnosing both why disk space (and perhaps
    clone or fetch times) is growing but also why certain operations are slow
    because the inflated size of the abstract objects that must be processed is
    so large.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    61f5ff5 View commit details
    Browse the repository at this point in the history
  27. survey: show progress during object walk

    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    0714ece View commit details
    Browse the repository at this point in the history
  28. survey: add ability to track prioritized lists

    In future changes, we will make use of these methods. The intention is to
    keep track of the top contributors according to some metric. We don't want
    to store all of the entries and do a sort at the end, so track a
    constant-size table and remove rows that get pushed out depending on the
    chosen sorting algorithm.
    
    Co-authored-by: Jeff Hostetler <git@jeffhostetler.com>
    Signed-off-by; Jeff Hostetler <git@jeffhostetler.com>
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    2 people authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    76ae6b5 View commit details
    Browse the repository at this point in the history
  29. survey: add report of "largest" paths

    Since we are already walking our reachable objects using the path-walk API,
    let's now collect lists of the paths that contribute most to different
    metrics. Specifically, we care about
    
     * Number of versions.
     * Total size on disk.
     * Total inflated size (no delta or zlib compression).
    
    This information can be critical to discovering which parts of the
    repository are causing the most growth, especially on-disk size. Different
    packing strategies might help compress data more efficiently, but the toal
    inflated size is a representation of the raw size of all snapshots of those
    paths. Even when stored efficiently on disk, that size represents how much
    information must be processed to complete a command such as 'git blame'.
    
    Since the on-disk size is likely to be fragile, stop testing the exact
    output of 'git survey' and check that the correct set of headers is
    output.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1eef810 View commit details
    Browse the repository at this point in the history
  30. survey: add --top=<N> option and config

    The 'git survey' builtin provides several detail tables, such as "top
    files by on-disk size". The size of these tables defaults to 100,
    currently.
    
    Allow the user to specify this number via a new --top=<N> option or the
    new survey.top config key.
    
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    910d865 View commit details
    Browse the repository at this point in the history
  31. line-log: protect inner strbuf from free

    The output_prefix() method in line-log.c may call a function pointer via
    the diff_options struct. This function pointer returns a strbuf struct
    and then its buffer is passed back. However, that implies that the
    consumer is responsible to free the string. This is especially true
    because the default behavior is to duplicate the empty string.
    
    The existing functions used in the output_prefix pointer include:
    
     1. idiff_prefix_cb() in diff-lib.c. This returns the data pointer, so
        the value exists across multiple calls.
    
     2. diff_output_prefix_callback() in graph.c. This uses a static strbuf
        struct, so it reuses buffers across calls. These should not be
        freed.
    
     3. output_prefix_cb() in range-diff.c. This is similar to the
        diff-lib.c case.
    
    In each case, we should not be freeing this buffer. We can convert the
    output_prefix() function to return a const char pointer and stop freeing
    the result.
    
    This choice is essentially the opposite of what was done in 394affd
    (line-log: always allocate the output prefix, 2024-06-07).
    
    This was discovered via 'valgrind' while investigating a public report
    of a bug in 'git log --graph -L' [1].
    
    [1] git-for-windows#5185
    
    This issue would have been caught by the new test, when Git is compiled
    with ASan to catch these double frees.
    
    Co-authored-by: Jeff King <peff@peff.net>
    Signed-off-by: Jeff King <peff@peff.net>
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    2 people authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1b6847f View commit details
    Browse the repository at this point in the history
  32. survey: clearly note the experimental nature in the output

    While this command is definitely something we _want_, chances are that
    upstreaming this will require substantial changes.
    
    We still want to be able to experiment with this before that, to focus
    on what we need out of this command: To assist with diagnosing issues
    with large repositories, as well as to help monitoring the growth and
    the associated painpoints of such repositories.
    
    To that end, we are about to integrate this command into
    `microsoft/git`, to get the tool into the hands of users who need it
    most, with the idea to iterate in close collaboration between these
    users and the developers familar with Git's internals.
    
    However, we will definitely want to avoid letting anybody have the
    impression that this command, its exact inner workings, as well as its
    output format, are anywhere close to stable. To make that fact utterly
    clear (and thereby protect the freedom to iterate and innovate freely
    before upstreaming the command), let's mark its output as experimental
    in all-caps, as the first thing we do.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a563945 View commit details
    Browse the repository at this point in the history
  33. line-log: remove output_prefix()

    Now that output_prefix() returns a const char * type, it matches the
    behavior of diff_line_prefix() and no longer needs to exist on its own.
    
    Signed-off-by: Jeff King <peff@peff.net>
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    peff authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    eff0ca0 View commit details
    Browse the repository at this point in the history
  34. diff: modify output_prefix function pointer

    The uses of the output_prefix function pointer in the diff_options
    struct is currently difficult to work with by returning a pointer to a
    strbuf. There is only one use that cares about the length of the string,
    which appears to be the only justification of the return type.
    
    We already noticed confusing memory issues around this return type, so
    use a const char * return type to make it clear that the caller does not
    own this string buffer.
    
    Signed-off-by: Jeff King <peff@peff.net>
    Signed-off-by: Derrick Stolee <stolee@gmail.com>
    peff authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    db93c66 View commit details
    Browse the repository at this point in the history
  35. Merge branch 'safe-PATH-lookup-in-gitk-on-Windows'

    This topic branch extends the protections introduced for Git GUI's
    CVE-2022-41953 to cover `gitk`, too.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f77097a View commit details
    Browse the repository at this point in the history
  36. Merge 'remote-hg-prerequisites' into HEAD

    These fixes were necessary for Sverre Rabbelier's remote-hg to work,
    but for some magic reason they are not necessary for the current
    remote-hg. Makes you wonder how that one gets away with it.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d3c7781 View commit details
    Browse the repository at this point in the history
  37. Merge branch 'drive-prefix'

    This topic branch allows us to specify absolute paths without the drive
    prefix e.g. when cloning.
    
    Example:
    
    	C:\Users\me> git clone https://github.com/git/git \upstream-git
    
    This will clone into a new directory C:\upstream-git, in line with how
    Windows interprets absolute paths.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3259881 View commit details
    Browse the repository at this point in the history
  38. Merge branch 'dont-clean-junctions'

    This topic branch teaches `git clean` to respect NTFS junctions and Unix
    bind mounts: it will now stop at those boundaries.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    4b5f1af View commit details
    Browse the repository at this point in the history
  39. Merge branch 'msys2-python'

    In MSYS2, we have two Python interpreters at our disposal, so we can
    include the Python stuff in the build.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    4731663 View commit details
    Browse the repository at this point in the history
  40. Merge branch 'mimalloc-v2.0.9'

    This topic vendors in mimalloc v2.0.9, a fast allocator that allows Git
    for Windows to perform efficiently.
    
    Switch Git for Windows to using mimalloc instead of nedmalloc
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f6ea2a7 View commit details
    Browse the repository at this point in the history
  41. Merge pull request git-for-windows#2375 from assarbad/reintroduce-sid…

    …eband-config
    
    Config option to disable side-band-64k for transport
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1351770 View commit details
    Browse the repository at this point in the history
  42. Merge pull request git-for-windows#2405 from dscho/mingw-setsockopt

    Make sure `errno` is set when socket operations fail
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    5630f19 View commit details
    Browse the repository at this point in the history
  43. Merge pull request git-for-windows#2449 from dscho/mingw-getcwd-and-s…

    …ymlinks
    
    Do resolve symlinks in `getcwd()`
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ae5ba9c View commit details
    Browse the repository at this point in the history
  44. Merge pull request git-for-windows#2488 from bmueller84/master

    mingw: fix fatal error working on mapped network drives on Windows
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6600b1a View commit details
    Browse the repository at this point in the history
  45. Merge pull request git-for-windows#2501 from jeffhostetler/clink-debu…

    …g-curl
    
    clink.pl: fix MSVC compile script to handle libcurl-d.lib
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3f7fa1d View commit details
    Browse the repository at this point in the history
  46. Merge pull request git-for-windows#2504 from dscho/access-repo-via-ju…

    …nction
    
    Handle `git add <file>` where <file> traverses an NTFS junction
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e99cdcd View commit details
    Browse the repository at this point in the history
  47. Merge pull request git-for-windows#2506 from dscho/issue-2283

    Allow running Git directly from `C:\Program Files\Git\mingw64\bin\git.exe`
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1587bd1 View commit details
    Browse the repository at this point in the history
  48. Merge pull request git-for-windows#2535 from dscho/schannel-revoke-be…

    …st-effort
    
    Introduce and use the new "best effort" strategy for Secure Channel revoke checking
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3059223 View commit details
    Browse the repository at this point in the history
  49. Merge pull request git-for-windows#2618 from dscho/avoid-d/f-conflict…

    …-in-vs/master
    
    ci: avoid d/f conflict in vs/master
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    270f7f4 View commit details
    Browse the repository at this point in the history
  50. Merge 'add-p-many-files'

    This topic branch allows `add -p` and `add -i` with a large number of
    files. It is kind of a hack that was never really meant to be
    upstreamed. Let's see if we can do better in the built-in `add -p`.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    876026a View commit details
    Browse the repository at this point in the history
  51. Merge pull request git-for-windows#2730 from dscho/crlf-aware-git-add-i

    git add -i: handle CR/LF line endings in the interactive input
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    54386d6 View commit details
    Browse the repository at this point in the history
  52. Merge pull request git-for-windows#2714 from lbonanomi/crlf-scissors

    Rationalize line endings for scissors-cleanup
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6a320e0 View commit details
    Browse the repository at this point in the history
  53. Merge pull request git-for-windows#2655 from jglathe/jg/t0014_trace_e…

    …xtra_info
    
    t/t0014: fix: eliminate additional lines from trace
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    7ff3d47 View commit details
    Browse the repository at this point in the history
  54. Merge 'git-gui/js/intent-to-add'

    This merges the current version of the patch that tries to address Git
    GUI's problems with intent-to-add files.
    
    This patch will likely be improved substantially before it is merged
    into Git GUI's main branch, but we want to have _something_ resembling a
    fix already in Git for Windows v2.29.0.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6f512cd View commit details
    Browse the repository at this point in the history
  55. Merge pull request git-for-windows#2974 from derrickstolee/maintenanc…

    …e-and-headless
    
    Include Windows-specific maintenance and headless-git
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f0e44dd View commit details
    Browse the repository at this point in the history
  56. cmake: install headless-git. (git-for-windows#4338)

    Even if CMake is not the canonical way to build Git for Windows, but
    CMake support merely exists in Git to support building Git for Windows
    using Visual Studio, we should include `headless-git` in such a scenario
    when installing the binaries to a given location.
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    4877c51 View commit details
    Browse the repository at this point in the history
  57. Merge pull request git-for-windows#2351 from PhilipOakley/vcpkg-tip

    Vcpkg Install: detect lack of working Git, and note possible vcpkg time outs
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    54be4aa View commit details
    Browse the repository at this point in the history
  58. Merge pull request git-for-windows#2915 from dennisameling/windows-ar…

    …m64-support
    
    Windows arm64 support
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1984428 View commit details
    Browse the repository at this point in the history
  59. Merge pull request git-for-windows#3327 from dennisameling/fix-host-cpu

    cmake(): allow setting HOST_CPU for cross-compilation
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    bb450f4 View commit details
    Browse the repository at this point in the history
  60. Merge pull request git-for-windows#3165 from dscho/increase-allowed-l…

    …ength-of-interpreter-path
    
    mingw: allow for longer paths in `parse_interpreter()`
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    24e816a View commit details
    Browse the repository at this point in the history
  61. Merge pull request git-for-windows#3220 from dscho/there-is-no-vs/mas…

    …ter-anymore
    
    Let the documentation reflect that there is no vs/master anymore
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    82f250d View commit details
    Browse the repository at this point in the history
  62. Merge pull request git-for-windows#3293 from pascalmuller/http-suppor…

    …t-automatically-sending-client-certificate
    
    http: Add support for enabling automatic sending of SSL client certificate
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    88864c0 View commit details
    Browse the repository at this point in the history
  63. Merge pull request git-for-windows#3349 from vdye/feature/ci-subtree-…

    …tests
    
    Add `contrib/subtree` test execution to CI builds
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    85472a9 View commit details
    Browse the repository at this point in the history
  64. Merge pull request git-for-windows#3306 from PhilipOakley/vs-sln

    Make Git for Windows start builds in modern Visual Studio
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e7ee159 View commit details
    Browse the repository at this point in the history
  65. Merge pull request git-for-windows#3417 from dscho/initialize-core.sy…

    …mlinks-earlier
    
    init: respect core.symlinks before copying the templates
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    2f8fcbb View commit details
    Browse the repository at this point in the history
  66. Merge pull request git-for-windows#3533 from PhilipOakley/hashliteral_t

    Begin `unsigned long`->`size_t` conversion to support large files on Windows
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    9805f8a View commit details
    Browse the repository at this point in the history
  67. Merge pull request git-for-windows#3791: Various fixes around `safe.d…

    …irectory`
    
    The first three commits are rebased versions of those in gitgitgadget#1215. These allow the following:
    
    1. Fix `git config --global foo.bar <path>` from allowing the `<path>`. As a bonus, users with a config value starting with `/` will not get a warning about "old-style" paths needing a "`%(prefix)/`".
    
    2. When in WSL, the path starts with `/` so it needs to be interpolated properly. Update the warning to include `%(prefix)/` to get the right value for WSL users. (This is specifically for using Git for Windows from Git Bash, but in a WSL directory.)
    
    3. When using WSL, the ownership check fails and reports an error message. This is noisy, and happens even if the user has marked the path with `safe.directory`. Remove that error message.
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1908d17 View commit details
    Browse the repository at this point in the history
  68. Merge pull request git-for-windows#3751 from rkitover/native-term

    mingw: set $env:TERM=xterm-256color for newer OSes
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3968416 View commit details
    Browse the repository at this point in the history
  69. Merge pull request git-for-windows#3875 from 1480c1/wine/detect_msys_tty

    winansi: check result before using Name for pty
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d5a7a36 View commit details
    Browse the repository at this point in the history
  70. Merge branch 'optionally-dont-append-atomically-on-windows'

    Fix append failure issue under remote directories git-for-windows#2753
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    887a326 View commit details
    Browse the repository at this point in the history
  71. Merge branch 'fsync-object-files-always'

    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    43392d7 View commit details
    Browse the repository at this point in the history
  72. Merge branch 'ci-fixes'

    Backport a couple fixes to make the CI build run again (so much for
    reproducible builds...).
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    047f7b4 View commit details
    Browse the repository at this point in the history
  73. Merge pull request git-for-windows#3942 from rimrul/mingw-tsaware

    MinGW: link as terminal server aware
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8505d76 View commit details
    Browse the repository at this point in the history
  74. Fix Windows version resources (git-for-windows#4092)

    Add `FileVersion`, which is a required string ([Microsoft
    documentation](https://learn.microsoft.com/en-us/windows/win32/menurc/versioninfo-resource))
    in the `StringFileInfo` block.
    As not all required strings were present in the block, none were being
    included.
    Fixes git-for-windows#4090
    
    After including the `FileVersion` string, all other defined strings are
    now being included on executables.
    
    File version information for `git.exe` has changed from:
    ```
    PS C:\Program Files\Git\bin> [System.Diagnostics.FileVersionInfo]::GetVersionInfo("C:\Data\git-sdk-64\usr\src\git\git.exe") | Select-Object *
    
    FileVersionRaw     : 2.38.1.1
    ProductVersionRaw  : 2.38.1.1
    Comments           :
    CompanyName        :
    FileBuildPart      : 1
    FileDescription    :
    FileMajorPart      : 2
    FileMinorPart      : 38
    FileName           : C:\Data\git-sdk-64\usr\src\git\git.exe
    FilePrivatePart    : 1
    FileVersion        :
    InternalName       :
    IsDebug            : False
    IsPatched          : False
    IsPrivateBuild     : False
    IsPreRelease       : False
    IsSpecialBuild     : False
    Language           : English (United States)
    LegalCopyright     :
    LegalTrademarks    :
    OriginalFilename   :
    PrivateBuild       :
    ProductBuildPart   : 1
    ProductMajorPart   : 2
    ProductMinorPart   : 38
    ProductName        :
    ProductPrivatePart : 1
    ProductVersion     :
    SpecialBuild       :
    ```
    
    To the following:
    ```
    PS C:\Program Files\Git\bin> [System.Diagnostics.FileVersionInfo]::GetVersionInfo("C:\Data\git-sdk-64\usr\src\git\git.exe") | Select-Object *
    
    FileVersionRaw     : 2.38.1.1
    ProductVersionRaw  : 2.38.1.1
    Comments           :
    CompanyName        : The Git Development Community
    FileBuildPart      : 1
    FileDescription    : Git for Windows
    FileMajorPart      : 2
    FileMinorPart      : 38
    FileName           : C:\Data\git-sdk-64\usr\src\git\git.exe
    FilePrivatePart    : 1
    FileVersion        : 2.38.1.windows.1.10.g6ed65a6fab
    InternalName       : git
    IsDebug            : False
    IsPatched          : False
    IsPrivateBuild     : False
    IsPreRelease       : False
    IsSpecialBuild     : False
    Language           : English (United States)
    LegalCopyright     :
    LegalTrademarks    :
    OriginalFilename   : git.exe
    PrivateBuild       :
    ProductBuildPart   : 1
    ProductMajorPart   : 2
    ProductMinorPart   : 38
    ProductName        : Git
    ProductPrivatePart : 1
    ProductVersion     : 2.38.1.windows.1.10.g6ed65a6fab
    SpecialBuild       :
    ```
    
    I wasn't really expecting `GIT_VERSION` to contain the Git commit, I was
    hoping for just `2.38.1` or `2.38.1.1`, at least for the `FileVersion`
    string.
    
    Anybody know if it's possible to concatenate the `MAJOR`, `MINOR`,
    `MICRO`, and `PATCHLEVEL` fields with dots, or if there's another
    variable that can be used (with or without `PATCHLEVEL`)?
    Alternatively, use the complete `GIT_VERSION` for both `FileVersion` and
    `ProductVersion`.
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    9af5c1d View commit details
    Browse the repository at this point in the history
  75. Merge branch 'builtin-swap-functions'

    Do prefer GCC's built-in bit-swap functions.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ff8b985 View commit details
    Browse the repository at this point in the history
  76. Add support for CLANGARM64 target (git-for-windows#3916)

    **This requires an ARM64-machine with Windows 11 installed (which
    supports x64 emulation for MSYS2)**
    
    ### The main idea
    
    - Use the main MSYS2/git-sdk-64 setup, which works on Windows 11 on ARM
    thanks to x64-emulation
    - Configure the official `clangarm64` MSYS2 repo
    - Install `mingw-w64-clang-aarch64-toolchain` which contains the
    ARM64-native Clang compiler
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e8c362b View commit details
    Browse the repository at this point in the history
  77. Fix global repository field not being cleared (git-for-windows#4083)

    It is checked for w.r.t. global repository struct down in the callstack
    in compatibility layer for MinGW before being assigned in the function
    that `free()`'d it.
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    56b7a4a View commit details
    Browse the repository at this point in the history
  78. Skip linking the "dashed" git-<command>s for built-ins (git-for-win…

    …dows#4252)
    
    It is merely a historical wart that, say, `git-commit` exists in the
    `libexec/git-core/` directory, a tribute to the original idea to let Git
    be essentially a bunch of Unix shell scripts revolving around very few
    "plumbing" (AKA low-level) commands.
    
    Git has evolved a lot from there. These days, most of Git's
    functionality is contained within the `git` executable, in the form of
    "built-in" commands.
    
    To accommodate for scripts that use the "dashed" form of Git commands,
    even today, Git provides hard-links that make the `git` executable
    available as, say, `git-commit`, just in case that an old script has not
    been updated to invoke `git commit`.
    
    Those hard-links do not come cheap: they take about half a minute for
    every build of Git on Windows, they are mistaken for taking up huge
    amounts of space by some Windows Explorer versions that do not
    understand hard-links, and therefore many a "bug" report had to be
    addressed.
    
    The "dashed form" has been officially deprecated in Git version 1.5.4,
    which was released on February 2nd, 2008, i.e. a very long time ago.
    This deprecation was never finalized by skipping these hard-links, but
    we can start the process now, in Git for Windows.
    
    This addresses the concern raised in
    git-for-windows#4185 (comment)
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d49561f View commit details
    Browse the repository at this point in the history
  79. Git GUI: fix Repository>Explore Working Copy (git-for-windows#4357)

    This is a companion PR of prati0100/git-gui#95
    
    Since Git v2.39.1, we are a bit more stringent in searching the PATH. In
    particular, we specifically require the `.exe` suffix.
    
    However, the `Repository>Explore Working Copy` command asks for
    `explorer.exe` to be found on the `PATH`, which _already_ has that
    suffix.
    
    Let's unstartle the PATH-finding logic about this scenario.
    
    This fixes git-for-windows#4356
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ebb4d3f View commit details
    Browse the repository at this point in the history
  80. Lazy load libcurl, allowing for an SSL/TLS backend-specific libcurl (g…

    …it-for-windows#4410)
    
    As per
    git-for-windows#4350 (comment),
    the major block for upgrading Git for Windows' OpenSSL from v1.1 to v3
    is the tricky part where such an upgrade would break `git fetch`/`git
    clone` and `git push` because the libcurl depends on the OpenSSL DLL,
    and the major version bump will _change_ the file name of said DLL.
    
    To overcome that, the plan is to build libcurl flavors for each
    supported SSL/TLS backend, aligning with the way MSYS2 builds libcurl,
    then switch Git for Windows' SDK to the Secure Channel-flavored libcurl,
    and teach Git to look for the specific flavor of libcurl corresponding
    to the `http.sslBackend` setting (if that was configured).
    
    Here is the PR to teach Git that trick.
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1bdf694 View commit details
    Browse the repository at this point in the history
  81. Merge branch 'nano-server'

    This patch adds a GitHub workflow (to be triggered manually) to allow
    for conveniently verifying that Git and Scalar still work as intended in
    Windows Nano Server (a relatively small container base image that is
    frequently used where a "small Windows" is needed, e.g. in automation
    ;-))
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    47a1180 View commit details
    Browse the repository at this point in the history
  82. Additional error checks for issuing the windows.appendAtomically warn…

    …ing (git-for-windows#4528)
    
    Another (hopefully clean) PR for showing the error warning about atomic
    append on windows after failure on APFS, which returns EBADF not EINVAL.
    
    Signed-off-by: David Lomas <dl3@pale-eds.co.uk>
    Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    df5f41d View commit details
    Browse the repository at this point in the history
  83. win32: use native ANSI sequence processing, if possible (git-for-wind…

    …ows#4700)
    
    Windows 10 version 1511 (also known as Anniversary Update), according to
    https://learn.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences
    introduced native support for ANSI sequence processing. This allows
    using colors from the entire 24-bit color range.
    
    All we need to do is test whether the console's "virtual processing
    support" can be enabled. If it can, we do not even need to start the
    `console_thread` to handle ANSI sequences.
    
    Incidentally, this addresses
    git-for-windows#3712.
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    4da4c9f View commit details
    Browse the repository at this point in the history
  84. ARM64: Embed manifest properly (git-for-windows#4718)

    Teach our ARM64 based builds to embed the manifest file correctly.
    
    This fixes git-for-windows#4707
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8b72a3c View commit details
    Browse the repository at this point in the history
  85. Configuration menu
    Copy the full SHA
    744024e View commit details
    Browse the repository at this point in the history
  86. Merge branch 'run-t5601-and-t7406-with-symlinks-on-windows-10'

    This topic branch contains a patch that made it into Git for Windows
    v2.45.1 but not into Git v2.45.1 (because the latter does not come with
    symlink support on Windows).
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    519ce46 View commit details
    Browse the repository at this point in the history
  87. Merge branch 'Fix-i686-build-with-GCC-v14'

    This fixes a long-time compile warning turned error by GCC v14.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f3a25a8 View commit details
    Browse the repository at this point in the history
  88. Merge branch 'Fallback-to-AppData-if-XDG-CONFIG-HOME-is-unset'

    This topic branch adds support for a more Windows-native user-wide
    config file than `XDG_CONFIG_HOME` (or `~/.config/`) will ever be.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b32ad2e View commit details
    Browse the repository at this point in the history
  89. Merge branch 'run-command-be-helpful-when-Git-LFS-fails-on-Windows-7'

    Since Git LFS v3.5.x implicitly dropped Windows 7 support, we now want
    users to be advised _what_ is going wrong on that Windows version. This
    topic branch goes out of its way to provide users with such guidance.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    15972c5 View commit details
    Browse the repository at this point in the history
  90. pack-objects: create new name-hash algorithm (git-for-windows#5157)

    This is an updated version of gitgitgadget#1785, intended for early
    consumption into Git for Windows.
    
    The idea here is to add a new `--full-name-hash` option to `git
    pack-objects` and `git repack`. This adjusts the name-hash value used
    for finding delta bases in such a way that uses the full path name with
    a lower likelihood of collisions than the default name-hash algorithm.
    In many repositories with name-hash collisions and many versions of
    those paths, this can significantly reduce the size of a full repack. It
    can also help in certain cases of `git push`, but only if the pack is
    already artificially inflated by name-hash collisions; cases that find
    "sibling" deltas as better choices become worse with `--full-name-hash`.
    
    Thus, this option is currently recommended for full repacks of large
    repos, and on client machines without reachability bitmaps.
    
    Some care is taken to ignore this option when using bitmaps, either
    writing bitmaps or using a bitmap walk during reads. The bitmap file
    format contains name-hash values, but no way to indicate which function
    is used, so compatibility is a concern for bitmaps. Future work could
    explore this idea.
    
    After this PR is merged, then the more-involved `--path-walk` option may
    be considered.
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1f75682 View commit details
    Browse the repository at this point in the history
  91. Add path walk API and its use in 'git pack-objects' (git-for-windows#…

    …5171)
    
    This is a follow up to git-for-windows#5157 as well as motivated by the RFC in
    gitgitgadget#1786.
    
    We have ways of walking all objects, but it is focused on visiting a
    single commit and then expanding the new trees and blobs reachable from
    that commit that have not been visited yet. This means that objects
    arrive without any locality based on their path.
    
    Add a new "path walk API" that focuses on walking objects in batches
    according to their type and path. This will walk all annotated tags, all
    commits, all root trees, and then start a depth-first search among all
    paths in the repo to collect trees and blobs in batches.
    
    The most important application for this is being fast-tracked to Git for
    Windows: `git pack-objects --path-walk`. This application of the path
    walk API discovers the objects to pack via this batched walk, and
    automatically groups objects that appear at a common path so they can be
    checked for delta comparisons.
    
    This use completely avoids any name-hash collisions (even the collisions
    that sometimes occur with the new `--full-name-hash` option) and can be
    much faster to compute since the first pass of delta calculations does
    not waste time on objects that are unlikely to be diffable.
    
    Some statistics are available in the commit messages.
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    68d135b View commit details
    Browse the repository at this point in the history
  92. path-walk API: avoid adding a root tree more than once (git-for-windo…

    …ws#5195)
    
    When adding tree objects, we are very careful to avoid adding the same
    tree object more than once. There was one small gap in that logic,
    though: when adding a root tree object. Two refs can easily share the
    same root tree object, and we should still not add it more than once.
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a0b4e54 View commit details
    Browse the repository at this point in the history
  93. Introduce 'git backfill' to get missing blobs in a partial clone (git…

    …-for-windows#5172)
    
    This change introduces the `git backfill` command which uses the path
    walk API to download missing blobs in a blobless partial clone.
    
    By downloading blobs that correspond to the same file path at the same
    time, we hope to maximize the potential benefits of delta compression
    against multiple versions.
    
    These downloads occur in a configurable batch size, presenting a
    mechanism to perform "resumable" clones: `git clone --filter=blob:none`
    gets the commits and trees, then `git backfill` will download all
    missing blobs. If `git backfill` is interrupted partway through, it can
    be restarted and will redownload only the missing objects.
    
    When combining blobless partial clones with sparse-checkout, `git
    backfill` will assume its `--sparse` option and download only the blobs
    within the sparse-checkout. Users may want to do this as the repo size
    will still be smaller than the full repo size, but commands like `git
    blame` or `git log -L` will not suffer from many one-by-one blob
    downloads.
    
    Future directions should consider adding a pathspec or file prefix to
    further focus which paths are being downloaded in a batch.
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8e758c4 View commit details
    Browse the repository at this point in the history
  94. Add experimental 'git survey' builtin (git-for-windows#5174)

    This introduces `git survey` to Git for Windows ahead of upstream for
    the express purpose of getting the path-based analysis in the hands of
    more folks.
    
    The inspiration of this builtin is
    [`git-sizer`](https://github.com/github/git-sizer), but since that
    command relies on `git cat-file --batch` to get the contents of objects,
    it has limits to how much information it can provide.
    
    This is mostly a rewrite of the `git survey` builtin that was introduced
    into the `microsoft/git` fork in microsoft#667. That version had a
    lot more bells and whistles, including an analysis much closer to what
    `git-sizer` provides.
    
    The biggest difference in this version is that this one is focused on
    using the path-walk API in order to visit batches of objects based on a
    common path. This allows identifying, for instance, the path that is
    contributing the most to the on-disk size across all versions at that
    path.
    
    For example, here are the top ten paths contributing to my local Git
    repository (which includes `microsoft/git` and `gitster/git`):
    
    ```
    TOP FILES BY DISK SIZE
    ============================================================================
                                        Path | Count | Disk Size | Inflated Size
    -----------------------------------------+-------+-----------+--------------
                           whats-cooking.txt |  1373 |  11637459 |      37226854
                 t/helper/test-gvfs-protocol |     2 |   6847105 |      17233072
                          git-rebase--helper |     1 |   6027849 |      15269664
                              compat/mingw.c |  6111 |   5194453 |     463466970
                 t/helper/test-parse-options |     1 |   3420385 |       8807968
                      t/helper/test-pkt-line |     1 |   3408661 |       8778960
          t/helper/test-dump-untracked-cache |     1 |   3408645 |       8780816
                t/helper/test-dump-fsmonitor |     1 |   3406639 |       8776656
                                    po/vi.po |   104 |   1376337 |      51441603
                                    po/de.po |   210 |   1360112 |      71198603
    ```
    
    This kind of analysis has been helpful in identifying the reasons for
    growth in a few internal monorepos. Those findings motivated the changes
    in git-for-windows#5157 and git-for-windows#5171.
    
    With this early version in Git for Windows, we can expand the reach of
    the experimental tool in advance of it being contributed to the upstream
    project.
    
    Unfortunately, this will mean that in the next `microsoft/git` rebase,
    @jeffhostetler's version will need to be pulled out since there are
    enough conflicts. These conflicts include how tables are stored and
    generated, as the version in this PR is slightly more general to allow
    for different kinds of data.
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b5869d3 View commit details
    Browse the repository at this point in the history
  95. Fix git log --graph -u hangs (git-for-windows#5193)

    This fixes git-for-windows#5185 by
    backporting gitgitgadget#1806 (which, sadly,
    seems not to have made it into Git v2.47.0).
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a8e465c View commit details
    Browse the repository at this point in the history
  96. Win32: make FILETIME conversion functions public

    We will use them in the upcoming "FSCache" patches (to accelerate
    sequential lstat() calls).
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    2e6b871 View commit details
    Browse the repository at this point in the history
  97. Merge branch 'ready-for-upstream'

    This is the branch thicket of patches in Git for Windows that are
    considered ready for upstream. To keep them in a ready-to-submit shape,
    they are kept as close to the beginning of the branch thicket as
    possible.
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    0d79092 View commit details
    Browse the repository at this point in the history
  98. Win32: dirent.c: Move opendir down

    Move opendir down in preparation for the next patch.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    dcfad57 View commit details
    Browse the repository at this point in the history
  99. mingw: make the dirent implementation pluggable

    Emulating the POSIX `dirent` API on Windows via
    `FindFirstFile()`/`FindNextFile()` is pretty staightforward, however,
    most of the information provided in the `WIN32_FIND_DATA` structure is
    thrown away in the process. A more sophisticated implementation may
    cache this data, e.g. for later reuse in calls to `lstat()`.
    
    Make the `dirent` implementation pluggable so that it can be switched at
    runtime, e.g. based on a config option.
    
    Define a base DIR structure with pointers to `readdir()`/`closedir()`
    that match the `opendir()` implementation (similar to vtable pointers in
    Object-Oriented Programming). Define `readdir()`/`closedir()` so that
    they call the function pointers in the `DIR` structure. This allows to
    choose the `opendir()` implementation on a call-by-call basis.
    
    Make the fixed-size `dirent.d_name` buffer a flex array, as `d_name` may
    be implementation specific (e.g. a caching implementation may allocate a
    `struct dirent` with _just_ the size needed to hold the `d_name` in
    question).
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    0077575 View commit details
    Browse the repository at this point in the history
  100. Win32: make the lstat implementation pluggable

    Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
    slow. Windows operating system APIs seem to be much better at scanning the
    status of entire directories than checking single files. A caching
    implementation may improve performance by bulk-reading entire directories
    or reusing data obtained via opendir / readdir.
    
    Make the lstat implementation pluggable so that it can be switched at
    runtime, e.g. based on a config option.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    95bccd8 View commit details
    Browse the repository at this point in the history
  101. mingw: add infrastructure for read-only file system level caches

    Add a macro to mark code sections that only read from the file system,
    along with a config option and documentation.
    
    This facilitates implementation of relatively simple file system level
    caches without the need to synchronize with the file system.
    
    Enable read-only sections for 'git status' and preload_index.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    98e220a View commit details
    Browse the repository at this point in the history
  102. mingw: add a cache below mingw's lstat and dirent implementations

    Checking the work tree status is quite slow on Windows, due to slow
    `lstat()` emulation (git calls `lstat()` once for each file in the
    index). Windows operating system APIs seem to be much better at scanning
    the status of entire directories than checking single files.
    
    Add an `lstat()` implementation that uses a cache for lstat data. Cache
    misses read the entire parent directory and add it to the cache.
    Subsequent `lstat()` calls for the same directory are served directly
    from the cache.
    
    Also implement `opendir()`/`readdir()`/`closedir()` so that they create
    and use directory listings in the cache.
    
    The cache doesn't track file system changes and doesn't plug into any
    modifying file APIs, so it has to be explicitly enabled for git functions
    that don't modify the working copy.
    
    Note: in an earlier version of this patch, the cache was always active and
    tracked file system changes via ReadDirectoryChangesW. However, this was
    much more complex and had negative impact on the performance of modifying
    git commands such as 'git checkout'.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    028efda View commit details
    Browse the repository at this point in the history
  103. fscache: load directories only once

    If multiple threads access a directory that is not yet in the cache, the
    directory will be loaded by each thread. Only one of the results is added
    to the cache, all others are leaked. This wastes performance and memory.
    
    On cache miss, add a future object to the cache to indicate that the
    directory is currently being loaded. Subsequent threads register themselves
    with the future object and wait. When the first thread has loaded the
    directory, it replaces the future object with the result and notifies
    waiting threads.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d4f0444 View commit details
    Browse the repository at this point in the history
  104. fscache: add key for GIT_TRACE_FSCACHE

    Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    jeffhostetler authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e45f8ad View commit details
    Browse the repository at this point in the history
  105. fscache: remember not-found directories

    Teach FSCACHE to remember "not found" directories.
    
    This is a performance optimization.
    
    FSCACHE is a performance optimization available for Windows.  It
    intercepts Posix-style lstat() calls into an in-memory directory
    using FindFirst/FindNext.  It improves performance on Windows by
    catching the first lstat() call in a directory, using FindFirst/
    FindNext to read the list of files (and attribute data) for the
    entire directory into the cache, and short-cut subsequent lstat()
    calls in the same directory.  This gives a major performance
    boost on Windows.
    
    However, it does not remember "not found" directories.  When STATUS
    runs and there are missing directories, the lstat() interception
    fails to find the parent directory and simply return ENOENT for the
    file -- it does not remember that the FindFirst on the directory
    failed. Thus subsequent lstat() calls in the same directory, each
    re-attempt the FindFirst.  This completely defeats any performance
    gains.
    
    This can be seen by doing a sparse-checkout on a large repo and
    then doing a read-tree to reset the skip-worktree bits and then
    running status.
    
    This change reduced status times for my very large repo by 60%.
    
    Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    jeffhostetler authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d00800e View commit details
    Browse the repository at this point in the history
  106. dir.c: make add_excludes aware of fscache during status

    Teach read_directory_recursive() and add_excludes() to
    be aware of optional fscache and avoid trying to open()
    and fstat() non-existant ".gitignore" files in every
    directory in the worktree.
    
    The current code in add_excludes() calls open() and then
    fstat() for a ".gitignore" file in each directory present
    in the worktree.  Change that when fscache is enabled to
    call lstat() first and if present, call open().
    
    This seems backwards because both lstat needs to do more
    work than fstat.  But when fscache is enabled, fscache will
    already know if the .gitignore file exists and can completely
    avoid the IO calls.  This works because of the lstat diversion
    to mingw_lstat when fscache is enabled.
    
    This reduced status times on a 350K file enlistment of the
    Windows repo on a NVMe SSD by 0.25 seconds.
    
    Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
    jeffhostetler authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    fda16c8 View commit details
    Browse the repository at this point in the history
  107. fscache: add a test for the dir-not-found optimization

    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ade3ba7 View commit details
    Browse the repository at this point in the history
  108. add: use preload-index and fscache for performance

    Teach "add" to use preload-index and fscache features
    to improve performance on very large repositories.
    
    During an "add", a call is made to run_diff_files()
    which calls check_remove() for each index-entry.  This
    calls lstat().  On Windows, the fscache code intercepts
    the lstat() calls and builds a private cache using the
    FindFirst/FindNext routines, which are much faster.
    
    Somewhat independent of this, is the preload-index code
    which distributes some of the start-up costs across
    multiple threads.
    
    We need to keep the call to read_cache() before parsing the
    pathspecs (and hence cannot use the pathspecs to limit any preload)
    because parse_pathspec() is using the index to determine whether a
    pathspec is, in fact, in a submodule. If we would not read the index
    first, parse_pathspec() would not error out on a path that is inside
    a submodule, and t7400-submodule-basic.sh would fail with
    
    	not ok 47 - do not add files from a submodule
    
    We still want the nice preload performance boost, though, so we simply
    call read_cache_preload(&pathspecs) after parsing the pathspecs.
    
    Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    jeffhostetler authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    53de33f View commit details
    Browse the repository at this point in the history
  109. fscache: make fscache_enabled() public

    Make fscache_enabled() function public rather than static.
    Remove unneeded fscache_is_enabled() function.
    Change is_fscache_enabled() macro to call fscache_enabled().
    
    is_fscache_enabled() now takes a pathname so that the answer
    is more precise and mean "is fscache enabled for this pathname",
    since fscache only stores repo-relative paths and not absolute
    paths, we can avoid attempting lookups for absolute paths.
    
    Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
    jeffhostetler authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3826f0d View commit details
    Browse the repository at this point in the history
  110. dir.c: regression fix for add_excludes with fscache

    Fix regression described in:
    git-for-windows#1392
    
    which was introduced in:
    git-for-windows@b235337
    
    Problem Symptoms
    ================
    When the user has a .gitignore file that is a symlink, the fscache
    optimization introduced above caused the stat-data from the symlink,
    rather that of the target file, to be returned.  Later when the ignore
    file was read, the buffer length did not match the stat.st_size field
    and we called die("cannot use <path> as an exclude file")
    
    Optimization Rationale
    ======================
    The above optimization calls lstat() before open() primarily to ask
    fscache if the file exists.  It gets the current stat-data as a side
    effect essentially for free (since we already have it in memory).
    If the file does not exist, it does not need to call open().  And
    since very few directories have .gitignore files, we can greatly
    reduce time spent in the filesystem.
    
    Discussion of Fix
    =================
    The above optimization calls lstat() rather than stat() because the
    fscache only intercepts lstat() calls.  Calls to stat() stay directed
    to the mingw_stat() completly bypassing fscache.  Furthermore, calls
    to mingw_stat() always call {open, fstat, close} so that symlinks are
    properly dereferenced, which adds *additional* open/close calls on top
    of what the original code in dir.c is doing.
    
    Since the problem only manifests for symlinks, we add code to overwrite
    the stat-data when the path is a symlink.  This preserves the effect of
    the performance gains provided by the fscache in the normal case.
    
    Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
    jeffhostetler authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    eeead24 View commit details
    Browse the repository at this point in the history
  111. fetch-pack.c: enable fscache for stats under .git/objects

    When I do git fetch, git call file stats under .git/objects for each
    refs. This takes time when there are many refs.
    
    By enabling fscache, git takes file stats by directory traversing and that
    improved the speed of fetch-pack for repository having large number of
    refs.
    
    In my windows workstation, this improves the time of `git fetch` for
    chromium repository like below. I took stats 3 times.
    
    * With this patch
    TotalSeconds: 9.9825165
    TotalSeconds: 9.1862075
    TotalSeconds: 10.1956256
    Avg: 9.78811653333333
    
    * Without this patch
    TotalSeconds: 15.8406702
    TotalSeconds: 15.6248053
    TotalSeconds: 15.2085938
    Avg: 15.5580231
    
    Signed-off-by: Takuto Ikuta <tikuta@chromium.org>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    Takuto Ikuta authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    7973ab2 View commit details
    Browse the repository at this point in the history
  112. checkout.c: enable fscache for checkout again

    This is retry of git-for-windows#1419.
    
    I added flush_fscache macro to flush cached stats after disk writing
    with tests for regression reported in git-for-windows#1438 and git-for-windows#1442.
    
    git checkout checks each file path in sorted order, so cache flushing does not
    make performance worse unless we have large number of modified files in
    a directory containing many files.
    
    Using chromium repository, I tested `git checkout .` performance when I
    delete 10 files in different directories.
    With this patch:
    TotalSeconds: 4.307272
    TotalSeconds: 4.4863595
    TotalSeconds: 4.2975562
    Avg: 4.36372923333333
    
    Without this patch:
    TotalSeconds: 20.9705431
    TotalSeconds: 22.4867685
    TotalSeconds: 18.8968292
    Avg: 20.7847136
    
    I confirmed this patch passed all tests in t/ with core_fscache=1.
    
    Signed-off-by: Takuto Ikuta <tikuta@chromium.org>
    Takuto Ikuta authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    bb923bb View commit details
    Browse the repository at this point in the history
  113. Enable the filesystem cache (fscache) in refresh_index().

    On file systems that support it, this can dramatically speed up operations
    like add, commit, describe, rebase, reset, rm that would otherwise have to
    lstat() every file to "re-match" the stat information in the index to that
    of the file system.
    
    On a synthetic repo with 1M files, "git reset" dropped from 52.02 seconds to
    14.42 seconds for a savings of 72%.
    
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6f5ebad View commit details
    Browse the repository at this point in the history
  114. fscache: use FindFirstFileExW to avoid retrieving the short name

    Use FindFirstFileExW with FindExInfoBasic to avoid forcing NTFS to look up
    the short name.  Also switch to a larger (64K vs 4K) buffer using
    FIND_FIRST_EX_LARGE_FETCH to minimize round trips to the kernel.
    
    In a repo with ~200K files, this drops warm cache status times from 3.19
    seconds to 2.67 seconds for a 16% savings.
    
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    5517a9f View commit details
    Browse the repository at this point in the history
  115. status: disable and free fscache at the end of the status command

    At the end of the status command, disable and free the fscache so that we
    don't leak the memory and so that we can dump the fscache statistics.
    
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    2a5d448 View commit details
    Browse the repository at this point in the history
  116. fscache: add GIT_TEST_FSCACHE support

    Add support to fscache to enable running the entire test suite with the
    fscache enabled.
    
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e931185 View commit details
    Browse the repository at this point in the history
  117. mem_pool: add GIT_TRACE_MEMPOOL support

    Add tracing around initializing and discarding mempools. In discard report
    on the amount of memory unused in the current block to help tune setting
    the initial_size.
    
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3257dfe View commit details
    Browse the repository at this point in the history
  118. fscache: fscache takes an initial size

    Update enable_fscache() to take an optional initial size parameter which is
    used to initialize the hashmap so that it can avoid having to rehash as
    additional entries are added.
    
    Add a separate disable_fscache() macro to make the code clearer and easier
    to read.
    
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    9e1e228 View commit details
    Browse the repository at this point in the history
  119. fscache: update fscache to be thread specific instead of global

    The threading model for fscache has been to have a single, global cache.
    This puts requirements on it to be thread safe so that callers like
    preload-index can call it from multiple threads.  This was implemented
    with a single mutex and completion events which introduces contention
    between the calling threads.
    
    Simplify the threading model by making fscache thread specific.  This allows
    us to remove the global mutex and synchronization events entirely and instead
    associate a fscache with every thread that requests one. This works well with
    the current multi-threading which divides the cache entries into blocks with
    a separate thread processing each block.
    
    At the end of each worker thread, if there is a fscache on the primary
    thread, merge the cached results from the worker into the primary thread
    cache. This enables us to reuse the cache later especially when scanning for
    untracked files.
    
    In testing, this reduced the time spent in preload_index() by about 25% and
    also reduced the CPU utilization significantly.  On a repo with ~200K files,
    it reduced overall status times by ~12%.
    
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    daf7e83 View commit details
    Browse the repository at this point in the history
  120. fscache: add fscache hit statistics

    Track fscache hits and misses for lstat and opendir requests.  Reporting of
    statistics is done when the cache is disabled for the last time and freed
    and is only reported if GIT_TRACE_FSCACHE is set.
    
    Sample output is:
    
    11:33:11.836428 compat/win32/fscache.c:433 fscache: lstat 3775, opendir 263, total requests/misses 4052/269
    
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    94dd824 View commit details
    Browse the repository at this point in the history
  121. fscache: teach fscache to use mempool

    Now that the fscache is single threaded, take advantage of the mem_pool as
    the allocator to significantly reduce the cost of allocations and frees.
    
    With the reduced cost of free, in future patches, we can start freeing the
    fscache at the end of commands instead of just leaking it.
    
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b8f8cdc View commit details
    Browse the repository at this point in the history
  122. fscache: remember the reparse tag for each entry

    We will use this in the next commit to implement an FSCache-aware
    version of is_mount_point().
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8cf3d40 View commit details
    Browse the repository at this point in the history
  123. fscache: make fscache_enable() thread safe

    The recent change to make fscache thread specific relied on fscache_enable()
    being called first from the primary thread before being called in parallel
    from worker threads.  Make that more robust and protect it with a critical
    section to avoid any issues.
    
    Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f34dae8 View commit details
    Browse the repository at this point in the history
  124. fscache: teach fscache to use NtQueryDirectoryFile

    Using FindFirstFileExW() requires the OS to allocate a 64K buffer for each
    directory and then free it when we call FindClose().  Update fscache to call
    the underlying kernel API NtQueryDirectoryFile so that we can do the buffer
    management ourselves.  That allows us to allocate a single buffer for the
    lifetime of the cache and reuse it for each directory.
    
    This change improves performance of 'git status' by 18% in a repo with ~200K
    files and 30k folders.
    
    Documentation for NtQueryDirectoryFile can be found at:
    
    https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntifs/nf-ntifs-ntquerydirectoryfile
    https://docs.microsoft.com/en-us/windows/desktop/FileIO/file-attribute-constants
    https://docs.microsoft.com/en-us/windows/desktop/fileio/reparse-point-tags
    
    To determine if the specified directory is a symbolic link, inspect the
    FileAttributes member to see if the FILE_ATTRIBUTE_REPARSE_POINT flag is
    set. If so, EaSize will contain the reparse tag (this is a so far
    undocumented feature, but confirmed by the NTFS developers). To
    determine if the reparse point is a symbolic link (and not some other
    form of reparse point), test whether the tag value equals the value
    IO_REPARSE_TAG_SYMLINK.
    
    The NtQueryDirectoryFile() call works best (and on Windows 8.1 and
    earlier, it works *only*) with buffer sizes up to 64kB. Which is 32k
    wide characters, so let's use that as our buffer size.
    
    Signed-off-by: Ben Peart <benpeart@microsoft.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    benpeart authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6330375 View commit details
    Browse the repository at this point in the history
  125. unpack-trees: enable fscache for sparse-checkout

    When updating the skip-worktree bits in the index to align with new
    values in a sparse-checkout file, Git scans the entire working
    directory with lstat() calls. In a sparse-checkout, many of these
    lstat() calls are for paths that do not exist.
    
    Enable the fscache feature during this scan. Since enable_fscache()
    calls nest, the disable_fscache() method decrements a counter and
    would only clear the cache if that counter reaches zero.
    
    In a local test of a repo with ~2.2 million paths, updating the index
    with git read-tree -m -u HEAD with a sparse-checkout file containing
    only /.gitattributes improved from 2-3 minutes to ~6 seconds.
    
    Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3629591 View commit details
    Browse the repository at this point in the history
  126. git-gui: provide question helper for retry fallback on Windows

    Make use of the new environment variable GIT_ASK_YESNO to support the
    recently implemented fallback in case unlink, rename or rmdir fail for
    files in use on Windows. The added dialog will present a yes/no question
    to the the user which will currently be used by the windows compat layer
    to let the user retry a failed file operation.
    
    Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
    hvoigt authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8b37641 View commit details
    Browse the repository at this point in the history
  127. fscache: implement an FSCache-aware is_mount_point()

    When FSCache is active, we can cache the reparse tag and use it directly
    to determine whether a path refers to an NTFS junction, without any
    additional, costly I/O.
    
    Note: this change only makes a difference with the next commit, which
    will make use of the FSCache in `git clean` (contingent on
    `core.fscache` set, of course).
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d5c7454 View commit details
    Browse the repository at this point in the history
  128. git gui: set GIT_ASKPASS=git-gui--askpass if not set yet

    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3158c66 View commit details
    Browse the repository at this point in the history
  129. clean: make use of FSCache

    The `git clean` command needs to enumerate plenty of files and
    directories, and can therefore benefit from the FSCache.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    75ce329 View commit details
    Browse the repository at this point in the history
  130. gitk: Unicode file name support

    Assumes file names in git tree objects are UTF-8 encoded.
    
    On most unix systems, the system encoding (and thus the TCL system
    encoding) will be UTF-8, so file names will be displayed correctly.
    
    On Windows, it is impossible to set the system encoding to UTF-8.
    Changing the TCL system encoding (via 'encoding system ...', e.g. in the
    startup code) is explicitly discouraged by the TCL docs.
    
    Change gitk functions dealing with file names to always convert
    from and to UTF-8.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    76d79d9 View commit details
    Browse the repository at this point in the history
  131. git-gui--askyesno: fix funny text wrapping

    The text wrapping seems to be aligned to the right side of the Yes
    button, leaving an awful lot of empty space.
    
    Let's try to counter this by using pixel units.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    4ab6eca View commit details
    Browse the repository at this point in the history
  132. gitk: Use an external icon file on Windows

    Git for Windows now ships with the new Git icon from git-scm.com. Use that
    icon file if it exists instead of the old procedurally drawn one.
    
    This patch was sent upstream but so far no decision on its inclusion was
    made, so commit it to our fork.
    
    Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
    sschuberth authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    0c82cc9 View commit details
    Browse the repository at this point in the history
  133. git-gui--askyesno: allow overriding the window title

    "Question?" is maybe not the most informative thing to ask. In the
    absence of better information, it is the best we can do, of course.
    
    However, Git for Windows' auto updater just learned the trick to use
    git-gui--askyesno to ask the user whether to update now or not. And in
    this scripted scenario, we can easily pass a command-line option to
    change the window title.
    
    So let's support that with the new `--title <title>` option.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    dde11f4 View commit details
    Browse the repository at this point in the history
  134. gitk: fix arrow keys in input fields with Tcl/Tk >= 8.6

    Tcl/Tk 8.6 introduced new events for the cursor left/right keys and
    apparently changed the behavior of the previous event.
    
    Let's work around that by using the new events when we are running with
    Tcl/Tk 8.6 or later.
    
    This fixes git-for-windows#495
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    28d40b9 View commit details
    Browse the repository at this point in the history
  135. git-gui--askyesno (mingw): use Git for Windows' icon, if available

    For additional GUI goodness.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e6058c8 View commit details
    Browse the repository at this point in the history
  136. Merge 'git-gui' into HEAD

    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a99ea8c View commit details
    Browse the repository at this point in the history
  137. gitk: make the "list references" default window width wider

    When using remotes (with git-flow especially), the remote reference names
    are almost always wordwrapped in the "list references" window because it's
    somewhat narrow by default. It's possible to resize it with a mouse,
    but it's annoying to have to do this every time, especially on Windows 10,
    where the window border seems to be only one (1) pixel wide, thus making
    the grabbing of the window border tricky.
    
    Signed-off-by: James J. Raden <james.raden@gmail.com>
    sidecut authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    24be74a View commit details
    Browse the repository at this point in the history
  138. Merge branch 'git-gui-askyesno'

    These changes are necessary to support better Git for Windows' new
    auto-update feature.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b0cdda0 View commit details
    Browse the repository at this point in the history
  139. Merge 'gitk' into HEAD

    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f7d8568 View commit details
    Browse the repository at this point in the history
  140. Merge branch 'fscache'

    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a96aa60 View commit details
    Browse the repository at this point in the history
  141. Merge pull request git-for-windows#994 from jeffhostetler/jeffhostetl…

    …er/fscache_nfd
    
    fscache: add not-found directory cache to fscache
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    fa1bafd View commit details
    Browse the repository at this point in the history
  142. Merge pull request git-for-windows#971 from jeffhostetler/jeffhostetl…

    …er/add_preload_fscache
    
    add: use preload-index and fscache for performance
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    95e4f9f View commit details
    Browse the repository at this point in the history
  143. Merge pull request git-for-windows#1344 from jeffhostetler/perf_add_e…

    …xcludes_with_fscache
    
    dir.c: make add_excludes aware of fscache during status
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a70f212 View commit details
    Browse the repository at this point in the history
  144. Merge pull request git-for-windows#1426 from atetubou/fetch_pack

    fetch-pack.c: enable fscache for stats under .git/objects
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6ca479f View commit details
    Browse the repository at this point in the history
  145. Merge pull request git-for-windows#1468 from atetubou/fscache_checkou…

    …t_flush
    
    checkout.c: enable fscache for checkout again
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8b5ce62 View commit details
    Browse the repository at this point in the history
  146. Merge pull request git-for-windows#1827 from benpeart/fscache_refresh…

    …_index
    
    Enable the filesystem cache (fscache) in refresh_index().
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    61c18ed View commit details
    Browse the repository at this point in the history
  147. Merge pull request git-for-windows#1908 from benpeart/FindFirstFileEx…

    …-gfw
    
    fscache: use FindFirstFileExW to avoid retrieving the short name
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    de71e31 View commit details
    Browse the repository at this point in the history
  148. Merge pull request git-for-windows#1909 from benpeart/free-fscache-af…

    …ter-status-gfw
    
    status: disable and free fscache at the end of the status command
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    00f3380 View commit details
    Browse the repository at this point in the history
  149. Merge pull request git-for-windows#1911 from benpeart/git_test_fscach…

    …e-gfw
    
    fscache: add GIT_TEST_FSCACHE support
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3aac229 View commit details
    Browse the repository at this point in the history
  150. Merge pull request git-for-windows#1910 from benpeart/fscache_statist…

    …ics-gfw
    
    fscache: add fscache hit statistics
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    466ba3b View commit details
    Browse the repository at this point in the history
  151. Merge remote-tracking branch 'benpeart/fscache-per-thread-gfw'

    This brings substantial wins in performance because the FSCache is now
    per-thread, being merged to the primary thread only at the end, so we do
    not have to lock (except while merging).
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    77b1db5 View commit details
    Browse the repository at this point in the history
  152. Merge pull request git-for-windows#1934 from benpeart/fscache-thread-…

    …safe-enable-gfw
    
    fscache: make fscache_enable() thread safe
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f2bd2da View commit details
    Browse the repository at this point in the history
  153. Merge pull request git-for-windows#1937 from benpeart/fscache-NtQuery…

    …DirectoryFile-gfw
    
     fscache: teach fscache to use NtQueryDirectoryFile
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    51cf6a9 View commit details
    Browse the repository at this point in the history
  154. Merge branch 'fscache-and-sparse-checkout'

    When updating the skip-worktree bits in the index to align with new
    values in a sparse-checkout file, Git scans the entire working
    directory with lstat() calls. In a sparse-checkout, many of these
    lstat() calls are for paths that do not exist.
    
    Enable the fscache feature during this scan.
    
    In a local test of a repo with ~2.2 million paths, updating the index
    with `git read-tree -m -u HEAD` with a sparse-checkout file containing
    only `/.gitattributes` improved from 2-3 minutes to 15-20 seconds.
    
    More work could be done to stop running lstat() calls when recursing
    into directories that are known to not exist.
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b0add8c View commit details
    Browse the repository at this point in the history
  155. pack-objects (mingw): demonstrate a segmentation fault with large deltas

    There is a problem in the way 9ac3f0e (pack-objects: fix
    performance issues on packing large deltas, 2018-07-22) initializes that
    mutex in the `packing_data` struct. The problem manifests in a
    segmentation fault on Windows, when a mutex (AKA critical section) is
    accessed without being initialized. (With pthreads, you apparently do
    not really have to initialize them?)
    
    This was reported in git-for-windows#1839.
    
    Signed-off-by: Doug Kelly <dougk.ff7@gmail.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    vangdfang authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    bf28c7f View commit details
    Browse the repository at this point in the history
  156. mingw: support long paths

    Windows paths are typically limited to MAX_PATH = 260 characters, even
    though the underlying NTFS file system supports paths up to 32,767 chars.
    This limitation is also evident in Windows Explorer, cmd.exe and many
    other applications (including IDEs).
    
    Particularly annoying is that most Windows APIs return bogus error codes
    if a relative path only barely exceeds MAX_PATH in conjunction with the
    current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
    infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.
    
    Many Windows wide char APIs support longer than MAX_PATH paths through the
    file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
    Notable exceptions include functions dealing with executables and the
    current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
    well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).
    
    Introduce a handle_long_path function to check the length of a specified
    path properly (and fail with ENAMETOOLONG), and to optionally expand long
    paths using the '\\?\' file namespace prefix. Short paths will not be
    modified, so we don't need to worry about device names (NUL, CON, AUX).
    
    Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
    limited to MAX_PATH (at least not on Win7), so we can use it to do the
    heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
    '..', and make an absolute path).
    
    Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
    limit.
    
    Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
    that support long paths.
    
    While improved error checking is always active, long paths support must be
    explicitly enabled via 'core.longpaths' option. This is to prevent end
    users to shoot themselves in the foot by checking out files that Windows
    Explorer, cmd/bash or their favorite IDE cannot handle.
    
    Test suite:
    Test the case is when the full pathname length of a dir is close
    to 260 (MAX_PATH).
    Bug report and an original reproducer by Andrey Rogozhnikov:
    msysgit#122 (comment)
    
    [jes: adjusted test number to avoid conflicts, added support for
    chdir(), etc]
    
    Thanks-to: Martin W. Kirst <maki@bitkings.de>
    Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
    Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Stepan Kasal <kasal@ucw.cz>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    0b30bc0 View commit details
    Browse the repository at this point in the history
  157. Merge branch 'dont-clean-junctions-fscache'

    We already avoid traversing NTFS junction points in `git clean -dfx`.
    With this topic branch, we do that when the FSCache is enabled, too.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e3535ad View commit details
    Browse the repository at this point in the history
  158. Win32: fix 'lstat("dir/")' with long paths

    Use a suffciently large buffer to strip the trailing slash.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    0a9dc09 View commit details
    Browse the repository at this point in the history
  159. win32(long path support): leave drive-less absolute paths intact

    When trying to ensure that long paths are handled correctly, we
    first normalize absolute paths as we encounter them.
    
    However, if the path is a so-called "drive-less" absolute path, i.e. if
    it is relative to the current drive but _does_ start with a directory
    separator, we would want the normalized path to be such a drive-less
    absolute path, too.
    
    Let's do that, being careful to still include the drive prefix when we
    need to go through the `\\?\` dance (because there, the drive prefix is
    absolutely required).
    
    This fixes git-for-windows#4586.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    5ff1ebe View commit details
    Browse the repository at this point in the history
  160. compat/fsmonitor/fsm-*-win32: support long paths

    Update wchar_t buffers to use MAX_LONG_PATH instead of MAX_PATH and call
    xutftowcs_long_path() in the Win32 backend source files.
    
    Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
    jeffhostetler authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e4527a8 View commit details
    Browse the repository at this point in the history
  161. mingw: Support git_terminal_prompt with more terminals

    The `git_terminal_prompt()` function expects the terminal window to be
    attached to a Win32 Console. However, this is not the case with terminal
    windows other than `cmd.exe`'s, e.g. with MSys2's own `mintty`.
    
    Non-cmd terminals such as `mintty` still have to have a Win32 Console
    to be proper console programs, but have to hide the Win32 Console to
    be able to provide more flexibility (such as being resizeable not only
    vertically but also horizontally). By writing to that Win32 Console,
    `git_terminal_prompt()` manages only to send the prompt to nowhere and
    to wait for input from a Console to which the user has no access.
    
    This commit introduces a function specifically to support `mintty` -- or
    other terminals that are compatible with MSys2's `/dev/tty` emulation. We
    use the `TERM` environment variable as an indicator for that: if the value
    starts with "xterm" (such as `mintty`'s "xterm_256color"), we prefer to
    let `xterm_prompt()` handle the user interaction.
    
    The most prominent user of `git_terminal_prompt()` is certainly
    `git-remote-https.exe`. It is an interesting use case because both
    `stdin` and `stdout` are redirected when Git calls said executable, yet
    it still wants to access the terminal.
    
    When running inside a `mintty`, the terminal is not accessible to the
    `git-remote-https.exe` program, though, because it is a MinGW program
    and the `mintty` terminal is not backed by a Win32 console.
    
    To solve that problem, we simply call out to the shell -- which is an
    *MSys2* program and can therefore access `/dev/tty`.
    
    Helped-by: nalla <nalla@hamal.uberspace.de>
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    67337c8 View commit details
    Browse the repository at this point in the history
  162. compat/terminal.c: only use the Windows console if bash 'read -r' fails

    Accessing the Windows console through the special CONIN$ / CONOUT$ devices
    doesn't work properly for non-ASCII usernames an passwords.
    
    It also doesn't work for terminal emulators that hide the native console
    window (such as mintty), and 'TERM=xterm*' is not necessarily a reliable
    indicator for such terminals.
    
    The new shell_prompt() function, on the other hand, works fine for both
    MSys1 and MSys2, in native console windows as well as mintty, and properly
    supports Unicode. It just needs bash on the path (for 'read -s', which is
    bash-specific).
    
    On Windows, try to use the shell to read from the terminal. If that fails
    with ENOENT (i.e. bash was not found), use CONIN/OUT as fallback.
    
    Note: To test this, create a UTF-8 credential file with non-ASCII chars,
    e.g. in git-bash: 'echo url=http://täst.com > cred.txt'. Then in git-cmd,
    'git credential fill <cred.txt' works (shell version), while calling git
    without the git-wrapper (i.e. 'mingw64\bin\git credential fill <cred.txt')
    mangles non-ASCII chars in both console output and input.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b5dc26a View commit details
    Browse the repository at this point in the history
  163. strbuf_readlink: don't call readlink twice if hint is the exact link …

    …size
    
    strbuf_readlink() calls readlink() twice if the hint argument specifies the
    exact size of the link target (e.g. by passing stat.st_size as returned by
    lstat()). This is necessary because 'readlink(..., hint) == hint' could
    mean that the buffer was too small.
    
    Use hint + 1 as buffer size to prevent this.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    2c87e7b View commit details
    Browse the repository at this point in the history
  164. clean: suggest using core.longPaths if paths are too long to remove

    On Windows, git repositories may have extra files which need cleaned
    (e.g., a build directory) that may be arbitrarily deep. Suggest using
    `core.longPaths` if such situations are encountered.
    
    Fixes: git-for-windows#2715
    Signed-off-by: Ben Boeckel <mathstuf@gmail.com>
    mathstuf authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a36b695 View commit details
    Browse the repository at this point in the history
  165. mingw (git_terminal_prompt): do fall back to CONIN$/CONOUT$ method

    To support Git Bash running in a MinTTY, we use a dirty trick to access
    the MSYS2 pseudo terminal: we execute a Bash snippet that accesses
    /dev/tty.
    
    The idea was to fall back to writing to/reading from CONOUT$/CONIN$ if
    that Bash call failed because Bash was not found.
    
    However, we should fall back even in other error conditions, because we
    have not successfully read the user input. Let's make it so.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1f391dd View commit details
    Browse the repository at this point in the history
  166. strbuf_readlink: support link targets that exceed PATH_MAX

    strbuf_readlink() refuses to read link targets that exceed PATH_MAX (even
    if a sufficient size was specified by the caller).
    
    As some platforms support longer paths, remove this restriction (similar
    to strbuf_getcwd()).
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    5bea09f View commit details
    Browse the repository at this point in the history
  167. lockfile.c: use is_dir_sep() instead of hardcoded '/' checks

    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    512490f View commit details
    Browse the repository at this point in the history
  168. Win32: don't call GetFileAttributes twice in mingw_lstat()

    GetFileAttributes cannot handle paths with trailing dir separator. The
    current [l]stat implementation calls GetFileAttributes twice if the path
    has trailing slashes (first with the original path passed to [l]stat, and
    and a second time with a path copy with trailing '/' removed).
    
    With Unicode conversion, we get the length of the path for free and also
    have a (wide char) buffer that can be modified.
    
    Remove trailing directory separators before calling the Win32 API.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    bd1d088 View commit details
    Browse the repository at this point in the history
  169. Win32: implement stat() with symlink support

    With respect to symlinks, the current stat() implementation is almost the
    same as lstat(): except for the file type (st_mode & S_IFMT), it returns
    information about the link rather than the target.
    
    Implement stat by opening the file with as little permissions as possible
    and calling GetFileInformationByHandle on it. This way, all link resoltion
    is handled by the Windows file system layer.
    
    If symlinks are disabled, use lstat() as before, but fail with ELOOP if a
    symlink would have to be resolved.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6d21967 View commit details
    Browse the repository at this point in the history
  170. Win32: remove separate do_lstat() function

    With the new mingw_stat() implementation, do_lstat() is only called from
    mingw_lstat() (with follow == 0). Remove the extra function and the old
    mingw_stat()-specific (follow == 1) logic.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a32c452 View commit details
    Browse the repository at this point in the history
  171. Win32: let mingw_lstat() error early upon problems with reparse points

    When obtaining lstat information for reparse points, we need to call
    FindFirstFile() in addition to GetFileInformationEx() to obtain the type
    of the reparse point (symlink, mount point etc.). However, currently there
    is no error handling whatsoever if FindFirstFile() fails.
    
    Call FindFirstFile() before modifying the stat *buf output parameter and
    error out if the call fails.
    
    Note: The FindFirstFile() return value includes all the data that we get
    from GetFileAttributesEx(), so we could replace GetFileAttributesEx() with
    FindFirstFile(). We don't do that because GetFileAttributesEx() is about
    twice as fast for single files. I.e. we only pay the extra cost of calling
    FindFirstFile() in the rare case that we encounter a reparse point.
    
    Note: The indentation of the remaining reparse point code will be fixed in
    the next patch.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    04ac44d View commit details
    Browse the repository at this point in the history
  172. mingw: teach fscache and dirent about symlinks

    Move S_IFLNK detection to file_attr_to_st_mode() and reuse it in fscache.
    
    Implement DT_LNK detection in dirent.c and the fscache readdir version.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    797b60f View commit details
    Browse the repository at this point in the history
  173. Win32: lstat(): return adequate stat.st_size for symlinks

    Git typically doesn't trust the stat.st_size member of symlinks (e.g. see
    strbuf_readlink()). However, some functions take shortcuts if st_size is 0
    (e.g. diff_populate_filespec()).
    
    In mingw_lstat() and fscache_lstat(), make sure to return an adequate size.
    
    The extra overhead of opening and reading the reparse point to calculate
    the exact size is not necessary, as git doesn't rely on the value anyway.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    147a648 View commit details
    Browse the repository at this point in the history
  174. Win32: factor out retry logic

    The retry pattern is duplicated in three places. It also seems to be too
    hard to use: mingw_unlink() and mingw_rmdir() duplicate the code to retry,
    and both of them do so incompletely. They also do not restore errno if the
    user answers 'no'.
    
    Introduce a retry_ask_yes_no() helper function that handles retry with
    small delay, asking the user, and restoring errno.
    
    mingw_unlink: include _wchmod in the retry loop (which may fail if the
    file is locked exclusively).
    
    mingw_rmdir: include special error handling in the retry loop.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3d9ebe3 View commit details
    Browse the repository at this point in the history
  175. Win32: change default of 'core.symlinks' to false

    Symlinks on Windows don't work the same way as on Unix systems. E.g. there
    are different types of symlinks for directories and files, creating
    symlinks requires administrative privileges etc.
    
    By default, disable symlink support on Windows. I.e. users explicitly have
    to enable it with 'git config [--system|--global] core.symlinks true'.
    
    The test suite ignores system / global config files. Allow testing *with*
    symlink support by checking if native symlinks are enabled in MSys2 (via
    'MSYS=winsymlinks:nativestrict').
    
    Reminder: This would need to be changed if / when we find a way to run the
    test suite in a non-MSys-based shell (e.g. dash).
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    65c4e23 View commit details
    Browse the repository at this point in the history
  176. Win32: add symlink-specific error codes

    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d8c0893 View commit details
    Browse the repository at this point in the history
  177. Win32: mingw_unlink: support symlinks to directories

    _wunlink() / DeleteFileW() refuses to delete symlinks to directories. If
    _wunlink() fails with ERROR_ACCESS_DENIED, try _wrmdir() as well.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3a56953 View commit details
    Browse the repository at this point in the history
  178. Win32: mingw_rename: support renaming symlinks

    MSVCRT's _wrename() cannot rename symlinks over existing files: it returns
    success without doing anything. Newer MSVCR*.dll versions probably do not
    have this problem: according to CRT sources, they just call MoveFileEx()
    with the MOVEFILE_COPY_ALLOWED flag.
    
    Get rid of _wrename() and call MoveFileEx() with proper error handling.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d27bed2 View commit details
    Browse the repository at this point in the history
  179. Win32: mingw_chdir: change to symlink-resolved directory

    If symlinks are enabled, resolve all symlinks when changing directories,
    as required by POSIX.
    
    Note: Git's real_path() function bases its link resolution algorithm on
    this property of chdir(). Unfortunately, the current directory on Windows
    is limited to only MAX_PATH (260) characters. Therefore using symlinks and
    long paths in combination may be problematic.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f99b857 View commit details
    Browse the repository at this point in the history
  180. Win32: implement readlink()

    Implement readlink() by reading NTFS reparse points. Works for symlinks
    and directory junctions. If symlinks are disabled, fail with ENOSYS.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    86420a1 View commit details
    Browse the repository at this point in the history
  181. mingw: lstat: compute correct size for symlinks

    This commit fixes mingw_lstat by computing the proper size for symlinks
    according to POSIX. POSIX specifies that upon successful return from
    lstat: "the value of the st_size member shall be set to the length of
    the pathname contained in the symbolic link not including any
    terminating null byte".
    
    Prior to this commit the mingw_lstat function returned a fixed size of
    4096. This caused problems in git repositories that were accessed by
    git for Cygwin or git for WSL. For example, doing `git reset --hard`
    using git for Windows would update the size of symlinks in the index
    to be 4096; at a later time git for Cygwin or git for WSL would find
    that symlinks have changed size during `git status`. Vice versa doing
    `git reset --hard` in git for Cygwin or git for WSL would update the
    size of symlinks in the index with the correct value, only for git for
    Windows to find incorrectly at a later time that the size had changed.
    
    Signed-off-by: Bill Zissimopoulos <billziss@navimatics.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    billziss-gh authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    dc59fce View commit details
    Browse the repository at this point in the history
  182. Win32: implement basic symlink() functionality (file symlinks only)

    Implement symlink() that always creates file symlinks. Fails with ENOSYS
    if symlinks are disabled or unsupported.
    
    Note: CreateSymbolicLinkW() was introduced with symlink support in Windows
    Vista. For compatibility with Windows XP, we need to load it dynamically
    and fail gracefully if it isnt's available.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a7ad2f4 View commit details
    Browse the repository at this point in the history
  183. Win32: symlink: add support for symlinks to directories

    Symlinks on Windows have a flag that indicates whether the target is a file
    or a directory. Symlinks of wrong type simply don't work. This even affects
    core Win32 APIs (e.g. DeleteFile() refuses to delete directory symlinks).
    
    However, CreateFile() with FILE_FLAG_BACKUP_SEMANTICS doesn't seem to care.
    Check the target type by first creating a tentative file symlink, opening
    it, and checking the type of the resulting handle. If it is a directory,
    recreate the symlink with the directory flag set.
    
    It is possible to create symlinks before the target exists (or in case of
    symlinks to symlinks: before the target type is known). If this happens,
    create a tentative file symlink and postpone the directory decision: keep
    a list of phantom symlinks to be processed whenever a new directory is
    created in mingw_mkdir().
    
    Limitations: This algorithm may fail if a link target changes from file to
    directory or vice versa, or if the target directory is created in another
    process.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    kblees authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    73e5a1e View commit details
    Browse the repository at this point in the history
  184. mingw: try to create symlinks without elevated permissions

    With Windows 10 Build 14972 in Developer Mode, a new flag is supported
    by CreateSymbolicLink() to create symbolic links even when running
    outside of an elevated session (which was previously required).
    
    This new flag is called SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE and
    has the numeric value 0x02.
    
    Previous Windows 10 versions will not understand that flag and return an
    ERROR_INVALID_PARAMETER, therefore we have to be careful to try passing
    that flag only when the build number indicates that it is supported.
    
    For more information about the new flag, see this blog post:
    https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/
    
    This patch is loosely based on the patch submitted by Samuel D. Leslie
    as git-for-windows#1184.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    248e7b4 View commit details
    Browse the repository at this point in the history
  185. mingw: introduce code to detect whether we're inside a Windows container

    This will come in handy in the next commit.
    
    Signed-off-by: JiSeop Moon <zcube@zcube.kr>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    ZCube authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    4d79b76 View commit details
    Browse the repository at this point in the history
  186. mingw: when running in a Windows container, try to rename() harder

    It is a known issue that a rename() can fail with an "Access denied"
    error at times, when copying followed by deleting the original file
    works. Let's just fall back to that behavior.
    
    Signed-off-by: JiSeop Moon <zcube@zcube.kr>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    ZCube authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f3ef296 View commit details
    Browse the repository at this point in the history
  187. mingw: emulate stat() a little more faithfully

    When creating directories via `safe_create_leading_directories()`, we
    might encounter an already-existing directory which is not
    readable by the current user. To handle that situation, Git's code calls
    `stat()` to determine whether we're looking at a directory.
    
    In such a case, `CreateFile()` will fail, though, no matter what, and
    consequently `mingw_stat()` will fail, too. But POSIX semantics seem to
    still allow `stat()` to go forward.
    
    So let's call `mingw_lstat()` for the rescue if we fail to get a file
    handle due to denied permission in `mingw_stat()`, and fill the stat
    info that way.
    
    We need to be careful to not allow this to go forward in case that we're
    looking at a symbolic link: to resolve the link, we would still have to
    create a file handle, and we just found out that we cannot. Therefore,
    `stat()` still needs to fail with `EACCES` in that case.
    
    This fixes git-for-windows#2531.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ebe8730 View commit details
    Browse the repository at this point in the history
  188. mingw: move the file_attr_to_st_mode() function definition

    In preparation for making this function a bit more complicated (to allow
    for special-casing the `ContainerMappedDirectories` in Windows
    containers, which look like a symbolic link, but are not), let's move it
    out of the header.
    
    Signed-off-by: JiSeop Moon <zcube@zcube.kr>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    ZCube authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    7d1cc97 View commit details
    Browse the repository at this point in the history
  189. mingw: special-case index entries for symlinks with buggy size

    In git-for-windows#2637, we fixed a bug
    where symbolic links' target path sizes were recorded incorrectly in the
    index. The downside of this fix was that every user with tracked
    symbolic links in their checkouts would see them as modified in `git
    status`, but not in `git diff`, and only a `git add <path>` (or `git add
    -u`) would "fix" this.
    
    Let's do better than that: we can detect that situation and simply
    pretend that a symbolic link with a known bad size (or a size that just
    happens to be that bad size, a _very_ unlikely scenario because it would
    overflow our buffers due to the trailing NUL byte) means that it needs
    to be re-checked as if we had just checked it out.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    132c94f View commit details
    Browse the repository at this point in the history
  190. mingw: Windows Docker volumes are *not* symbolic links

    ... even if they may look like them.
    
    As looking up the target of the "symbolic link" (just to see whether it
    starts with `/ContainerMappedDirectories/`) is pretty expensive, we
    do it when we can be *really* sure that there is a possibility that this
    might be the case.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    Signed-off-by: JiSeop Moon <zcube@zcube.kr>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    53cb5c1 View commit details
    Browse the repository at this point in the history
  191. Win32: symlink: move phantom symlink creation to a separate function

    Signed-off-by: Bert Belder <bertbelder@gmail.com>
    piscisaureus authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ec29d2b View commit details
    Browse the repository at this point in the history
  192. Introduce helper to create symlinks that knows about index_state

    On Windows, symbolic links actually have a type depending on the target:
    it can be a file or a directory.
    
    In certain circumstances, this poses problems, e.g. when a symbolic link
    is supposed to point into a submodule that is not checked out, so there
    is no way for Git to auto-detect the type.
    
    To help with that, we will add support over the course of the next
    commits to specify that symlink type via the Git attributes. This
    requires an index_state, though, something that Git for Windows'
    `symlink()` replacement cannot know about because the function signature
    is defined by the POSIX standard and not ours to change.
    
    So let's introduce a helper function to create symbolic links that
    *does* know about the index_state.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    c088807 View commit details
    Browse the repository at this point in the history
  193. mingw: allow to specify the symlink type in .gitattributes

    On Windows, symbolic links have a type: a "file symlink" must point at
    a file, and a "directory symlink" must point at a directory. If the
    type of symlink does not match its target, it doesn't work.
    
    Git does not record the type of symlink in the index or in a tree. On
    checkout it'll guess the type, which only works if the target exists
    at the time the symlink is created. This may often not be the case,
    for example when the link points at a directory inside a submodule.
    
    By specifying `symlink=file` or `symlink=dir` the user can specify what
    type of symlink Git should create, so Git doesn't have to rely on
    unreliable heuristics.
    
    Signed-off-by: Bert Belder <bertbelder@gmail.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    piscisaureus authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    5a30497 View commit details
    Browse the repository at this point in the history
  194. mingw: work around rename() failing on a read-only file

    At least on _some_ APFS network shares, Git fails to rename the object
    files because they are marked as read-only, because that has the effect
    of setting the uchg flag on APFS, which then means the file can't be
    renamed or deleted.
    
    To work around that, when a rename failed, and the read-only flag is
    set, try to turn it off and on again.
    
    This fixes git-for-windows#4482
    
    Signed-off-by: David Lomas <dl3@pale-eds.co.uk>
    Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
    dsl101 authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    fc96572 View commit details
    Browse the repository at this point in the history
  195. Win32: symlink: add test for symlink attribute

    To verify that the symlink is resolved correctly, we use the fact that
    `git.exe` is a native Win32 program, and that `git.exe config -f <path>`
    therefore uses the native symlink resolution.
    
    Signed-off-by: Bert Belder <bertbelder@gmail.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    piscisaureus authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a9ed70c View commit details
    Browse the repository at this point in the history
  196. mingw: explicitly specify with which cmd to prefix the cmdline

    The main idea of this patch is that even if we have to look up the
    absolute path of the script, if only the basename was specified as
    argv[0], then we should use that basename on the command line, too, not
    the absolute path.
    
    This patch will also help with the upcoming patch where we automatically
    substitute "sh ..." by "busybox sh ..." if "sh" is not in the PATH but
    "busybox" is: we will do that by substituting the actual executable, but
    still keep prepending "sh" to the command line.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    236b7b7 View commit details
    Browse the repository at this point in the history
  197. mingw: when path_lookup() failed, try BusyBox

    BusyBox comes with a ton of applets ("applet" being the identical
    concept to Git's "builtins"). And similar to Git's builtins, the applets
    can be called via `busybox <command>`, or the BusyBox executable can be
    copied/hard-linked to the command name.
    
    The similarities do not end here. Just as with Git's builtins, it is
    problematic that BusyBox' hard-linked applets cannot easily be put into
    a .zip file: .zip archives have no concept of hard-links and therefore
    would store identical copies (and also extract identical copies,
    "inflating" the archive unnecessarily).
    
    To counteract that issue, MinGit already ships without hard-linked
    copies of the builtins, and the plan is to do the same with BusyBox'
    applets: simply ship busybox.exe as single executable, without
    hard-linked applets.
    
    To accommodate that, Git is being taught by this commit a very special
    trick, exploiting the fact that it is possible to call an executable
    with a command-line whose argv[0] is different from the executable's
    name: when `sh` is to be spawned, and no `sh` is found in the PATH, but
    busybox.exe is, use that executable (with unchanged argv).
    
    Likewise, if any executable to be spawned is not on the PATH, but
    busybox.exe is found, parse the output of `busybox.exe --help` to find
    out what applets are included, and if the command matches an included
    applet name, use busybox.exe to execute it.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    9fd2e84 View commit details
    Browse the repository at this point in the history
  198. test-lib: avoid unnecessary Perl invocation

    It is a bit strange, and even undesirable, to require Perl just to run
    the test suite even when NO_PERL was set.
    
    This patch does not fix this problem by any stretch of imagination.
    However, it fixes *the* Perl invocation that *every single* test script
    has to run.
    
    While at it, it makes the source code also more grep'able, as the code
    that unsets some, but not all, GIT_* environment variables just became a
    *lot* more explicit. And all that while still reducing the total number
    of lines.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    cf2a479 View commit details
    Browse the repository at this point in the history
  199. test-tool: learn to act as a drop-in replacement for iconv

    It is convenient to assume that everybody who wants to build & test Git
    has access to a working `iconv` executable (after all, we already pretty
    much require libiconv).
    
    However, that limits esoteric test scenarios such as Git for Windows',
    where an end user installation has to ship with `iconv` for the sole
    purpose of being testable. That payload serves no other purpose.
    
    So let's just have a test helper (to be able to test Git, the test
    helpers have to be available, after all) to act as `iconv` replacement.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    00139f9 View commit details
    Browse the repository at this point in the history
  200. tests(mingw): if iconv is unavailable, use test-helper --iconv

    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    7774624 View commit details
    Browse the repository at this point in the history
  201. gitattributes: mark .png files as binary

    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    023f17b View commit details
    Browse the repository at this point in the history
  202. tests: move test PNGs into t/lib-diff/

    We already have a directory where we store files intended for use by
    multiple test scripts. The same directory is a better home for the
    test-binary-*.png files than t/.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    9ab6b6b View commit details
    Browse the repository at this point in the history
  203. tests: only override sort & find if there are usable ones in /usr/bin/

    The idea is to allow running the test suite on MinGit with BusyBox
    installed in /mingw64/bin/sh.exe. In that case, we will want to exclude
    sort & find (and other Unix utilities) from being bundled.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    95f6db9 View commit details
    Browse the repository at this point in the history
  204. tests: use the correct path separator with BusyBox

    BusyBox-w32 is a true Win32 application, i.e. it does not come with a
    POSIX emulation layer.
    
    That also means that it does *not* use the Unix convention of separating
    the entries in the PATH variable using colons, but semicolons.
    
    However, there are also BusyBox ports to Windows which use a POSIX
    emulation layer such as Cygwin's or MSYS2's runtime, i.e. using colons
    as PATH separators.
    
    As a tell-tale, let's use the presence of semicolons in the PATH
    variable: on Unix, it is highly unlikely that it contains semicolons,
    and on Windows (without POSIX emulation), it is virtually guaranteed, as
    everybody should have both $SYSTEMROOT and $SYSTEMROOT/system32 in their
    PATH.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1aab1ad View commit details
    Browse the repository at this point in the history
  205. mingw: only use Bash-ism builtin pwd -W when available

    Traditionally, Git for Windows' SDK uses Bash as its default shell.
    However, other Unix shells are available, too. Most notably, the Win32
    port of BusyBox comes with `ash` whose `pwd` command already prints
    Windows paths as Git for Windows wants them, while there is not even a
    `builtin` command.
    
    Therefore, let's be careful not to override `pwd` unless we know that
    the `builtin` command is available.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e033676 View commit details
    Browse the repository at this point in the history
  206. tests (mingw): remove Bash-specific pwd option

    The -W option is only understood by MSYS2 Bash's pwd command. We already
    make sure to override `pwd` by `builtin pwd -W` for MINGW, so let's not
    double the effort here.
    
    This will also help when switching the shell to another one (such as
    BusyBox' ash) whose pwd does *not* understand the -W option.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8e3b66b View commit details
    Browse the repository at this point in the history
  207. test-lib: add BUSYBOX prerequisite

    When running with BusyBox, we will want to avoid calling executables on
    the PATH that are implemented in BusyBox itself.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    7ae8372 View commit details
    Browse the repository at this point in the history
  208. t5003: use binary file from t/lib-diff/

    At some stage, t5003-archive-zip wants to add a file that is not ASCII.
    To that end, it uses /bin/sh. But that file may actually not exist (it
    is too easy to forget that not all the world is Unix/Linux...)! Besides,
    we already have perfectly fine binary files intended for use solely by
    the tests. So let's use one of them instead.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    bb1b278 View commit details
    Browse the repository at this point in the history
  209. t5532: workaround for BusyBox on Windows

    While it may seem super convenient to some old Unix hands to simpy
    require Perl to be available when running the test suite, this is a
    major hassle on Windows, where we want to verify that Perl is not,
    actually, required in a NO_PERL build.
    
    As a super ugly workaround, we "install" a script into /usr/bin/perl
    reading like this:
    
    	#!/bin/sh
    
    	# We'd much rather avoid requiring Perl altogether when testing
    	# an installed Git. Oh well, that's why we cannot have nice
    	# things.
    	exec c:/git-sdk-64/usr/bin/perl.exe "$@"
    
    The problem with that is that BusyBox assumes that the #! line in a
    script refers to an executable, not to a script. So when it encounters
    the line #!/usr/bin/perl in t5532's proxy-get-cmd, it barfs.
    
    Let's help this situation by simply executing the Perl script with the
    "interpreter" specified explicitly.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ffe9503 View commit details
    Browse the repository at this point in the history
  210. t5605: special-case hardlink test for BusyBox-w32

    When t5605 tries to verify that files are hardlinked (or that they are
    not), it uses the `-links` option of the `find` utility.
    
    BusyBox' implementation does not support that option, and BusyBox-w32's
    lstat() does not even report the number of hard links correctly (for
    performance reasons).
    
    So let's just switch to a different method that actually works on
    Windows.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    dbb27bd View commit details
    Browse the repository at this point in the history
  211. t5813: allow for $PWD to be a Windows path

    Git for Windows uses MSYS2's Bash to run the test suite, which comes
    with benefits but also at a heavy price: on the plus side, MSYS2's
    POSIX emulation layer allows us to continue pretending that we are on a
    Unix system, e.g. use Unix paths instead of Windows ones, yet this is
    bought at a rather noticeable performance penalty.
    
    There *are* some more native ports of Unix shells out there, though,
    most notably BusyBox-w32's ash. These native ports do not use any POSIX
    emulation layer (or at most a *very* thin one, choosing to avoid
    features such as fork() that are expensive to emulate on Windows), and
    they use native Windows paths (usually with forward slashes instead of
    backslashes, which is perfectly legal in almost all use cases).
    
    And here comes the problem: with a $PWD looking like, say,
    C:/git-sdk-64/usr/src/git/t/trash directory.t5813-proto-disable-ssh
    Git's test scripts get quite a bit confused, as their assumptions have
    been shattered. Not only does this path contain a colon (oh no!), it
    also does not start with a slash.
    
    This is a problem e.g. when constructing a URL as t5813 does it:
    ssh://remote$PWD. Not only is it impossible to separate the "host" from
    the path with a $PWD as above, even prefixing $PWD by a slash won't
    work, as /C:/git-sdk-64/... is not a valid path.
    
    As a workaround, detect when $PWD does not start with a slash on
    Windows, and simply strip the drive prefix, using an obscure feature of
    Windows paths: if an absolute Windows path starts with a slash, it is
    implicitly prefixed by the drive prefix of the current directory. As we
    are talking about the current directory here, anyway, that strategy
    works.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    5b36798 View commit details
    Browse the repository at this point in the history
  212. t9200: skip tests when $PWD contains a colon

    On Windows, the current working directory is pretty much guaranteed to
    contain a colon. If we feed that path to CVS, it mistakes it for a
    separator between host and port, though.
    
    This has not been a problem so far because Git for Windows uses MSYS2's
    Bash using a POSIX emulation layer that also pretends that the current
    directory is a Unix path (at least as long as we're in a shell script).
    
    However, that is rather limiting, as Git for Windows also explores other
    ports of other Unix shells. One of those is BusyBox-w32's ash, which is
    a native port (i.e. *not* using any POSIX emulation layer, and certainly
    not emulating Unix paths).
    
    So let's just detect if there is a colon in $PWD and punt in that case.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    918f4ab View commit details
    Browse the repository at this point in the history
  213. mingw: add a Makefile target to copy test artifacts

    The Makefile target `install-mingit-test-artifacts` simply copies stuff
    and things directly into a MinGit directory, including an init.bat
    script to set everything up so that the tests can be run in a cmd
    window.
    
    Sadly, Git's test suite still relies on a Perl interpreter even if
    compiled with NO_PERL=YesPlease. We punt for now, installing a small
    script into /usr/bin/perl that hands off to an existing Perl of a Git
    for Windows SDK.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b99fde4 View commit details
    Browse the repository at this point in the history
  214. mingw: optionally enable wsl compability file mode bits

    The Windows Subsystem for Linux (WSL) version 2 allows to use `chmod` on
    NTFS volumes provided that they are mounted with metadata enabled (see
    https://devblogs.microsoft.com/commandline/chmod-chown-wsl-improvements/
    for details), for example:
    
    	$ chmod 0755 /mnt/d/test/a.sh
    
    In order to facilitate better collaboration between the Windows
    version of Git and the WSL version of Git, we can make the Windows
    version of Git also support reading and writing NTFS file modes
    in a manner compatible with WSL.
    
    Since this slightly slows down operations where lots of files are
    created (such as an initial checkout), this feature is only enabled when
    `core.WSLCompat` is set to true. Note that you also have to set
    `core.fileMode=true` in repositories that have been initialized without
    enabling WSL compatibility.
    
    There are several ways to enable metadata loading for NTFS volumes
    in WSL, one of which is to modify `/etc/wsl.conf` by adding:
    
    ```
    [automount]
    enabled = true
    options = "metadata,umask=027,fmask=117"
    ```
    
    And reboot WSL.
    
    It can also be enabled temporarily by this incantation:
    
    	$ sudo umount /mnt/c &&
    	  sudo mount -t drvfs C: /mnt/c -o metadata,uid=1000,gid=1000,umask=22,fmask=111
    
    It's important to note that this modification is compatible with, but
    does not depend on WSL. The helper functions in this commit can operate
    independently and functions normally on devices where WSL is not
    installed or properly configured.
    
    Signed-off-by: xungeng li <xungeng@gmail.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    swigger authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    0f35e77 View commit details
    Browse the repository at this point in the history
  215. mingw: kill child processes in a gentler way

    The TerminateProcess() function does not actually leave the child
    processes any chance to perform any cleanup operations. This is bad
    insofar as Git itself expects its signal handlers to run.
    
    A symptom is e.g. a left-behind .lock file that would not be left behind
    if the same operation was run, say, on Linux.
    
    To remedy this situation, we use an obscure trick: we inject a thread
    into the process that needs to be killed and to let that thread run the
    ExitProcess() function with the desired exit status. Thanks J Wyman for
    describing this trick.
    
    The advantage is that the ExitProcess() function lets the atexit
    handlers run. While this is still different from what Git expects (i.e.
    running a signal handler), in practice Git sets up signal handlers and
    atexit handlers that call the same code to clean up after itself.
    
    In case that the gentle method to terminate the process failed, we still
    fall back to calling TerminateProcess(), but in that case we now also
    make sure that processes spawned by the spawned process are terminated;
    TerminateProcess() does not give the spawned process a chance to do so
    itself.
    
    Please note that this change only affects how Git for Windows tries to
    terminate processes spawned by Git's own executables. Third-party
    software that *calls* Git and wants to terminate it *still* need to make
    sure to imitate this gentle method, otherwise this patch will not have
    any effect.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    10bb07f View commit details
    Browse the repository at this point in the history
  216. mingw: really handle SIGINT

    Previously, we did not install any handler for Ctrl+C, but now we really
    want to because the MSYS2 runtime learned the trick to call the
    ConsoleCtrlHandler when Ctrl+C was pressed.
    
    With this, hitting Ctrl+C while `git log` is running will only terminate
    the Git process, but not the pager. This finally matches the behavior on
    Linux and on macOS.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    26539c4 View commit details
    Browse the repository at this point in the history
  217. mingw: do not call xutftowcs_path in mingw_mktemp

    The `xutftowcs_path` function canonicalizes absolute paths using GetFullPathNameW.
    This canonicalization may change the length of the string (e.g. getting rid of \.\),
    which breaks callers that pass the template string in a strbuf and expect the
    length of the string to remain the same.
    
    In my particular case, the tmp-objdir code is passing a strbuf to mkdtemp and is
    breaking since the strbuf.len is no longer synchronized with strlen(strbuf.buf).
    
    Signed-off-by: Neeraj K. Singh <neerajsi@microsoft.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    neerajsi-msft authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    63cc420 View commit details
    Browse the repository at this point in the history
  218. Partially un-revert "editor: save and reset terminal after calling ED…

    …ITOR"
    
    In e3f7e01 (Revert "editor: save and reset terminal after calling
    EDITOR", 2021-11-22), we reverted the commit wholesale where the
    terminal state would be saved and restored before/after calling an
    editor.
    
    The reverted commit was intended to fix a problem with Windows Terminal
    where simply calling `vi` would cause problems afterwards.
    
    To fix the problem addressed by the revert, but _still_ keep the problem
    with Windows Terminal fixed, let's revert the revert, with a twist: we
    restrict the save/restore _specifically_ to the case where `vi` (or
    `vim`) is called, and do not do the same for any other editor.
    
    This should still catch the majority of the cases, and will bridge the
    time until the original patch is re-done in a way that addresses all
    concerns.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8b8b123 View commit details
    Browse the repository at this point in the history
  219. Add a GitHub workflow to monitor component updates

    Rather than using private IFTTT Applets that send mails to this
    maintainer whenever a new version of a Git for Windows component was
    released, let's use the power of GitHub workflows to make this process
    publicly visible.
    
    This workflow monitors the Atom/RSS feeds, and opens a ticket whenever a
    new version was released.
    
    Note: Bash sometimes releases multiple patched versions within a few
    minutes of each other (i.e. 5.1p1 through 5.1p4, 5.0p15 and 5.0p16). The
    MSYS2 runtime also has a similar system. We can address those patches as
    a group, so we shouldn't get multiple issues about them.
    
    Note further: We're not acting on newlib releases, OpenSSL alphas, Perl
    release candidates or non-stable Perl releases. There's no need to open
    issues about them.
    
    Co-authored-by: Matthias Aßhauer <mha1993@live.de>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho and rimrul committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    2ce548c View commit details
    Browse the repository at this point in the history
  220. reset: reinstate support for the deprecated --stdin option

    The `--stdin` option was a well-established paradigm in other commands,
    therefore we implemented it in `git reset` for use by Visual Studio.
    
    Unfortunately, upstream Git decided that it is time to introduce
    `--pathspec-from-file` instead.
    
    To keep backwards-compatibility for some grace period, we therefore
    reinstate the `--stdin` option on top of the `--pathspec-from-file`
    option, but mark it firmly as deprecated.
    
    Helped-by: Victoria Dye <vdye@github.com>
    Helped-by: Matthew John Cheetham <mjcheetham@outlook.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    c3e1525 View commit details
    Browse the repository at this point in the history
  221. fsmonitor: reintroduce core.useBuiltinFSMonitor

    Reintroduce the 'core.useBuiltinFSMonitor' config setting (originally added
    in 0a756b2 (fsmonitor: config settings are repository-specific,
    2021-03-05)) after its removal from the upstream version of FSMonitor.
    
    Upstream, the 'core.useBuiltinFSMonitor' setting was rendered obsolete by
    "overloading" the 'core.fsmonitor' setting to take a boolean value. However,
    several applications (e.g., 'scalar') utilize the original config setting,
    so it should be preserved for a deprecation period before complete removal:
    
    * if 'core.fsmonitor' is a boolean, the user is correctly using the new
      config syntax; do not use 'core.useBuiltinFSMonitor'.
    * if 'core.fsmonitor' is unspecified, use 'core.useBuiltinFSMonitor'.
    * if 'core.fsmonitor' is a path, override and use the builtin FSMonitor if
      'core.useBuiltinFSMonitor' is 'true'; otherwise, use the FSMonitor hook
      indicated by the path.
    
    Additionally, for this deprecation period, advise users to switch to using
    'core.fsmonitor' to specify their use of the builtin FSMonitor.
    
    Signed-off-by: Victoria Dye <vdye@github.com>
    vdye authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b4fb482 View commit details
    Browse the repository at this point in the history
  222. Configuration menu
    Copy the full SHA
    7a06b7b View commit details
    Browse the repository at this point in the history
  223. Describe Git for Windows' architecture [no ci]

    The Git for Windows project has grown quite complex over the years,
    certainly much more complex than during the first years where the
    `msysgit.git` repository was abusing Git for package management purposes
    and the `git/git` fork was called `4msysgit.git`.
    
    Let's describe the status quo in a thorough way.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    2ce307a View commit details
    Browse the repository at this point in the history
  224. Modify the Code of Conduct for Git for Windows

    The Git project followed Git for Windows' lead and added their Code of
    Conduct, based on the Contributor Covenant v1.4, later updated to v2.0.
    
    We adapt it slightly to Git for Windows.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    26bccbe View commit details
    Browse the repository at this point in the history
  225. CONTRIBUTING.md: add guide for first-time contributors

    Getting started contributing to Git can be difficult on a Windows
    machine. CONTRIBUTING.md contains a guide to getting started, including
    detailed steps for setting up build tools, running tests, and
    submitting patches to upstream.
    
    [includes an example by Pratik Karki how to submit v2, v3, v4, etc.]
    
    Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
    derrickstolee authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    141b8ab View commit details
    Browse the repository at this point in the history
  226. README.md: Add a Windows-specific preamble

    Includes touch-ups by 마누엘, Philip Oakley and 孙卓识.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b134efe View commit details
    Browse the repository at this point in the history
  227. Add an issue template

    With improvements by Clive Chan, Adric Norris, Ben Bodenmiller and
    Philip Oakley.
    
    Helped-by: Clive Chan <cc@clive.io>
    Helped-by: Adric Norris <landstander668@gmail.com>
    Helped-by: Ben Bodenmiller <bbodenmiller@hotmail.com>
    Helped-by: Philip Oakley <philipoakley@iee.org>
    Signed-off-by: Brendan Forster <brendan@github.com>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    shiftkey authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    fa6fc63 View commit details
    Browse the repository at this point in the history
  228. Modify the GitHub Pull Request template (to reflect Git for Windows)

    Git for Windows accepts pull requests; Core Git does not. Therefore we
    need to adjust the template (because it only matches core Git's
    project management style, not ours).
    
    Also: direct Git for Windows enhancements to their contributions page,
    space out the text for easy reading, and clarify that the mailing list
    is plain text, not HTML.
    
    Signed-off-by: Philip Oakley <philipoakley@iee.org>
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    Philip Oakley authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    0ce6de6 View commit details
    Browse the repository at this point in the history
  229. .github: Add configuration for the Sentiment Bot

    The sentiment bot will help detect when things get too heated.
    Hopefully.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e587604 View commit details
    Browse the repository at this point in the history
  230. Document how $HOME is set on Windows

    Git documentation refers to $HOME and $XDG_CONFIG_HOME often, but does not specify how or where these values come from on Windows where neither is set by default. The new documentation reflects the behavior of setup_windows_environment() in compat/mingw.c.
    
    Signed-off-by: Alejandro Barreto <alejandro.barreto@ni.com>
    alejandro5042 authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    2ff87a4 View commit details
    Browse the repository at this point in the history
  231. SECURITY.md: document Git for Windows' policies

    This is the recommended way on GitHub to describe policies revolving around
    security issues and about supported versions.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    9468391 View commit details
    Browse the repository at this point in the history
  232. Merge branch 'gitk-and-git-gui-patches'

    These are Git for Windows' Git GUI and gitk patches. We will have to
    decide at some point what to do about them, but that's a little lower
    priority (as Git GUI seems to be unmaintained for the time being, and
    the gitk maintainer keeps a very low profile on the Git mailing list,
    too).
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    c2afb5b View commit details
    Browse the repository at this point in the history
  233. Merge branch 'long-paths'

    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    17cfdb4 View commit details
    Browse the repository at this point in the history
  234. Configuration menu
    Copy the full SHA
    cd09bf7 View commit details
    Browse the repository at this point in the history
  235. Merge pull request git-for-windows#3817 from mathstuf/name-too-long-a…

    …dvice
    
    clean: suggest using `core.longPaths` if paths are too long to remove
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    0f7ef80 View commit details
    Browse the repository at this point in the history
  236. Merge branch 'msys2'

    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    8af67a1 View commit details
    Browse the repository at this point in the history
  237. Merge branch 'kblees/kb/symlinks'

    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    00488c3 View commit details
    Browse the repository at this point in the history
  238. Merge 'docker-volumes-are-no-symlinks'

    This was pull request git-for-windows#1645 from ZCube/master
    
    Support windows container.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    139da25 View commit details
    Browse the repository at this point in the history
  239. mingw: try resetting the read-only bit if rename fails (git-for-windo…

    …ws#4527)
    
    With this patch, Git for Windows works as intended on mounted APFS
    volumes (where renaming read-only files would fail).
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    7dff356 View commit details
    Browse the repository at this point in the history
  240. Merge pull request git-for-windows#1897 from piscisaureus/symlink-attr

    Specify symlink type in .gitattributes
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    56a2b67 View commit details
    Browse the repository at this point in the history
  241. Merge branch 'busybox-w32'

    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3255bd8 View commit details
    Browse the repository at this point in the history
  242. Merge branch 'wsl-file-mode-bits'

    This patch introduces support to set special NTFS attributes that are
    interpreted by the Windows Subsystem for Linux as file mode bits, UID
    and GID.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    c232aa7 View commit details
    Browse the repository at this point in the history
  243. Merge pull request git-for-windows#1170 from dscho/mingw-kill-process

    Handle Ctrl+C in Git Bash nicely
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f28f927 View commit details
    Browse the repository at this point in the history
  244. Merge pull request git-for-windows#3492 from dscho/ns/batched-fsync

    Switch to batched fsync by default
    vdye authored and dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    bd3f271 View commit details
    Browse the repository at this point in the history
  245. Merge branch 'un-revert-editor-save-and-reset'

    A fix for calling `vim` in Windows Terminal caused a regression and was
    reverted. We partially un-revert this, to get the fix again.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    61490f2 View commit details
    Browse the repository at this point in the history
  246. Merge branch 'phase-out-reset-stdin'

    This topic branch re-adds the deprecated --stdin/-z options to `git
    reset`. Those patches were overridden by a different set of options in
    the upstream Git project before we could propose `--stdin`.
    
    We offered this in MinGit to applications that wanted a safer way to
    pass lots of pathspecs to Git, and these applications will need to be
    adjusted.
    
    Instead of `--stdin`, `--pathspec-from-file=-` should be used, and
    instead of `-z`, `--pathspec-file-nul`.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ebcb5b5 View commit details
    Browse the repository at this point in the history
  247. Merge branch 'deprecate-core.useBuiltinFSMonitor'

    Originally introduced as `core.useBuiltinFSMonitor` in Git for Windows
    and developed, improved and stabilized there, the built-in FSMonitor
    only made it into upstream Git (after unnecessarily long hemming and
    hawing and throwing overly perfectionist style review sticks into the
    spokes) as `core.fsmonitor = true`.
    
    In Git for Windows, with this topic branch, we re-introduce the
    now-obsolete config setting, with warnings suggesting to existing users
    how to switch to the new config setting, with the intention to
    ultimately drop the patch at some stage.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b608f0e View commit details
    Browse the repository at this point in the history
  248. Merge pull request git-for-windows#2837 from dscho/monitor-component-…

    …updates
    
    Start monitoring updates of Git for Windows' component in the open
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    02788b7 View commit details
    Browse the repository at this point in the history
  249. Merge 'readme' into HEAD

    Add a README.md for GitHub goodness.
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ed7f388 View commit details
    Browse the repository at this point in the history
  250. fixup! path-walk API: avoid adding a root tree more than once

    Ooops. Must not risk a segmentation fault in a partial clone missing
    trees...
    
    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    dscho committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d53e464 View commit details
    Browse the repository at this point in the history