-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rebase to v2.47.0 #5196
Rebase to v2.47.0 #5196
Commits on Oct 8, 2024
-
path-walk: introduce an object walk by path
In anticipation of a few planned applications, introduce the most basic form of a path-walk API. It currently assumes that there are no UNINTERESTING objects, and does not include any complicated filters. It calls a function pointer on groups of tree and blob objects as grouped by path. This only includes objects the first time they are discovered, so an object that appears at multiple paths will not be included in two batches. There are many future adaptations that could be made, but they are left for future updates when consumers are ready to take advantage of those features. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 685d130 - Browse repository at this point
Copy the full SHA 685d130View commit details -
t6601: add helper for testing path-walk API
Add some tests based on the current behavior, doing interesting checks for different sets of branches, ranges, and the --boundary option. This sets a baseline for the behavior and we can extend it as new options are introduced. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for f10a957 - Browse repository at this point
Copy the full SHA f10a957View commit details -
path-walk: allow consumer to specify object types
We add the ability to filter the object types in the path-walk API so the callback function is called fewer times. This adds the ability to ask for the commits in a list, as well. Future changes will add the ability to visit annotated tags. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 118b06d - Browse repository at this point
Copy the full SHA 118b06dView commit details -
path-walk: allow visiting tags
In anticipation of using the path-walk API to analyze tags or include them in a pack-file, add the ability to walk the tags that were included in the revision walk. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 731998c - Browse repository at this point
Copy the full SHA 731998cView commit details -
revision: create mark_trees_uninteresting_dense()
The sparse tree walk algorithm was created in d5d2e93 (revision: implement sparse algorithm, 2019-01-16) and involves using the mark_trees_uninteresting_sparse() method. This method takes a repository and an oidset of tree IDs, some of which have the UNINTERESTING flag and some of which do not. Create a method that has an equivalent set of preconditions but uses a "dense" walk (recursively visits all reachable trees, as long as they have not previously been marked UNINTERESTING). This is an important difference from mark_tree_uninteresting(), which short-circuits if the given tree has the UNINTERESTING flag. A use of this method will be added in a later change, with a condition set whether the sparse or dense approach should be used. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 9e7da31 - Browse repository at this point
Copy the full SHA 9e7da31View commit details -
path-walk: add prune_all_uninteresting option
This option causes the path-walk API to act like the sparse tree-walk algorithm implemented by mark_trees_uninteresting_sparse() in list-objects.c. Starting from the commits marked as UNINTERESTING, their root trees and all objects reachable from those trees are UNINTERSTING, at least as we walk path-by-path. When we reach a path where all objects associated with that path are marked UNINTERESTING, then do no continue walking the children of that path. We need to be careful to pass the UNINTERESTING flag in a deep way on the UNINTERESTING objects before we start the path-walk, or else the depth-first search for the path-walk API may accidentally report some objects as interesting. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for ec1a5d4 - Browse repository at this point
Copy the full SHA ec1a5d4View commit details -
pack-objects: extract should_attempt_deltas()
This will be helpful in a future change. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 4578489 - Browse repository at this point
Copy the full SHA 4578489View commit details -
pack-objects: add --path-walk option
In order to more easily compute delta bases among objects that appear at the exact same path, add a --path-walk option to 'git pack-objects'. This option will use the path-walk API instead of the object walk given by the revision machinery. Since objects will be provided in batches representing a common path, those objects can be tested for delta bases immediately instead of waiting for a sort of the full object list by name-hash. This has multiple benefits, including avoiding collisions by name-hash. The objects marked as UNINTERESTING are included in these batches, so we are guaranteeing some locality to find good delta bases. After the individual passes are done on a per-path basis, the default name-hash is used to find other opportunistic delta bases that did not match exactly by the full path name. RFC TODO: It is important to note that this option is inherently incompatible with using a bitmap index. This walk probably also does not work with other advanced features, such as delta islands. Getting ahead of myself, this option compares well with --full-name-hash when the packfile is large enough, but also performs at least as well as the default in all cases that I've seen. RFC TODO: this should probably be recording the batch locations to another list so they could be processed in a second phase using threads. RFC TODO: list some examples of how this outperforms previous pack-objects strategies. (This is coming in later commits that include performance test changes.) Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 454cfdd - Browse repository at this point
Copy the full SHA 454cfddView commit details -
pack-objects: introduce GIT_TEST_PACK_PATH_WALK
There are many tests that validate whether 'git pack-objects' works as expected. Instead of duplicating these tests, add a new test environment variable, GIT_TEST_PACK_PATH_WALK, that implies --path-walk by default when specified. This was useful in testing the implementation of the --path-walk implementation, especially in conjunction with test such as: - t0411-clone-from-partial.sh : One test fetches from a repo that does not have the boundary objects. This causes the path-based walk to fail. Disable the variable for this test. - t5306-pack-nobase.sh : Similar to t0411, one test fetches from a repo without a boundary object. - t5310-pack-bitmaps.sh : One test compares the case when packing with bitmaps to the case when packing without them. Since we disable the test variable when writing bitmaps, this causes a difference in the object list (the --path-walk option adds an extra object). Specify --no-path-walk in both processes for the comparison. Another test checks for a specific delta base, but when computing dynamically without using bitmaps, the base object it too small to be considered in the delta calculations so no base is used. - t5316-pack-delta-depth.sh : This script cares about certain delta choices and their chain lengths. The --path-walk option changes how these chains are selected, and thus changes the results of this test. - t5322-pack-objects-sparse.sh : This demonstrates the effectiveness of the --sparse option and how it combines with --path-walk. - t5332-multi-pack-reuse.sh : This test verifies that the preferred pack is used for delta reuse when possible. The --path-walk option is not currently aware of the preferred pack at all, so finds a different delta base. - t7406-submodule-update.sh : When using the variable, the --depth option collides with the --path-walk feature, resulting in a warning message. Disable the variable so this warning does not appear. I want to call out one specific test change that is only temporary: - t5530-upload-pack-error.sh : One test cares specifically about an "unable to read" error message. Since the current implementation performs delta calculations within the path-walk API callback, a different "unable to get size" error message appears. When this is changed in a future refactoring, this test change can be reverted. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 82abe40 - Browse repository at this point
Copy the full SHA 82abe40View commit details -
repack: add --path-walk option
Since 'git pack-objects' supports a --path-walk option, allow passing it through in 'git repack'. This presents interesting testing opportunities for comparing the different repacking strategies against each other. Add the --path-walk option to the performance tests in p5313. For the microsoft/fluentui repo [1] checked out at a specific commit [2], the results are very interesting: Test this tree ------------------------------------------------------------------ 5313.2: thin pack 0.40(0.47+0.04) 5313.3: thin pack size 1.2M 5313.4: thin pack with --full-name-hash 0.09(0.10+0.04) 5313.5: thin pack size with --full-name-hash 22.8K 5313.6: thin pack with --path-walk 0.08(0.06+0.02) 5313.7: thin pack size with --path-walk 20.8K 5313.8: big pack 2.16(8.43+0.23) 5313.9: big pack size 17.7M 5313.10: big pack with --full-name-hash 1.42(3.06+0.21) 5313.11: big pack size with --full-name-hash 18.0M 5313.12: big pack with --path-walk 2.21(8.39+0.24) 5313.13: big pack size with --path-walk 17.8M 5313.14: repack 98.05(662.37+2.64) 5313.15: repack size 449.1K 5313.16: repack with --full-name-hash 33.95(129.44+2.63) 5313.17: repack size with --full-name-hash 182.9K 5313.18: repack with --path-walk 106.21(121.58+0.82) 5313.19: repack size with --path-walk 159.6K [1] https://github.com/microsoft/fluentui [2] e70848ebac1cd720875bccaa3026f4a9ed700e08 This repo suffers from having a lot of paths that collide in the name hash, so examining them in groups by path leads to better deltas. Also, in this case, the single-threaded implementation is competitive with the full repack. This is saving time diffing files that have significant differences from each other. A similar, but private, repo has even more extremes in the thin packs: Test this tree -------------------------------------------------------------- 5313.2: thin pack 2.39(2.91+0.10) 5313.3: thin pack size 4.5M 5313.4: thin pack with --full-name-hash 0.29(0.47+0.12) 5313.5: thin pack size with --full-name-hash 15.5K 5313.6: thin pack with --path-walk 0.35(0.31+0.04) 5313.7: thin pack size with --path-walk 14.2K Notice, however, that while the --full-name-hash version is working quite well in these cases for the thin pack, it does poorly for some other standard cases, such as this test on the Linux kernel repository: Test this tree -------------------------------------------------------------- 5313.2: thin pack 0.01(0.00+0.00) 5313.3: thin pack size 310 5313.4: thin pack with --full-name-hash 0.00(0.00+0.00) 5313.5: thin pack size with --full-name-hash 1.4K 5313.6: thin pack with --path-walk 0.00(0.00+0.00) 5313.7: thin pack size with --path-walk 310 Here, the --full-name-hash option does much worse than the default name hash, but the path-walk option does exactly as well. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for f07075e - Browse repository at this point
Copy the full SHA f07075eView commit details -
pack-objects: enable --path-walk via config
Users may want to enable the --path-walk option for 'git pack-objects' by default, especially underneath commands like 'git push' or 'git repack'. This should be limited to client repositories, since the --path-walk option disables bitmap walks, so would be bad to include in Git servers when serving fetches and clones. There is potential that it may be helpful to consider when repacking the repository, to take advantage of improved deltas across historical versions of the same files. Much like how "pack.useSparse" was introduced and included in "feature.experimental" before being enabled by default, use the repository settings infrastructure to make the new "pack.usePathWalk" config enabled by "feature.experimental" and "feature.manyFiles". Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for bf51d09 - Browse repository at this point
Copy the full SHA bf51d09View commit details -
scalar: enable path-walk during push via config
Repositories registered with Scalar are expected to be client-only repositories that are rather large. This means that they are more likely to be good candidates for using the --path-walk option when running 'git pack-objects', especially under the hood of 'git push'. Enable this config in Scalar repositories. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 2d533c9 - Browse repository at this point
Copy the full SHA 2d533c9View commit details -
pack-objects: refactor path-walk delta phase
Previously, the --path-walk option to 'git pack-objects' would compute deltas inline with the path-walk logic. This would make the progress indicator look like it is taking a long time to enumerate objects, and then very quickly computed deltas. Instead of computing deltas on each region of objects organized by tree, store a list of regions corresponding to these groups. These can later be pulled from the list for delta compression before doing the "global" delta search. This presents a new progress indicator that can be used in tests to verify that this stage is happening. The current implementation is not integrated with threads, but could be done in a future update. Since we do not attempt to sort objects by size until after exploring all trees, we can remove the previous change to t5530 due to a different error message appearing first. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 2f07cd7 - Browse repository at this point
Copy the full SHA 2f07cd7View commit details -
pack-objects: thread the path-based compression
Adapting the implementation of ll_find_deltas(), create a threaded version of the --path-walk compression step in 'git pack-objects'. This involves adding a 'regions' member to the thread_params struct, allowing each thread to own a section of paths. We can simplify the way jobs are split because there is no value in extending the batch based on name-hash the way sections of the object entry array are attempted to be grouped. We re-use the 'list_size' and 'remaining' items for the purpose of borrowing work in progress from other "victim" threads when a thread has finished its batch of work more quickly. Using the Git repository as a test repo, the p5313 performance test shows that the resulting size of the repo is the same, but the threaded implementation gives gains of varying degrees depending on the number of objects being packed. (This was tested on a 16-core machine.) Test HEAD~1 HEAD ------------------------------------------------------------- 5313.6: thin pack with --path-walk 0.01 0.01 +0.0% 5313.7: thin pack size with --path-walk 475 475 +0.0% 5313.12: big pack with --path-walk 1.99 1.87 -6.0% 5313.13: big pack size with --path-walk 14.4M 14.3M -0.4% 5313.18: repack with --path-walk 98.14 41.46 -57.8% 5313.19: repack size with --path-walk 197.2M 197.3M +0.0% Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for cd6e3a6 - Browse repository at this point
Copy the full SHA cd6e3a6View commit details -
path-walk API: avoid adding a root tree more than once
When adding tree objects, we are very careful to avoid adding the same tree object more than once. There was one small gap in that logic, though: when adding a root tree object. Two refs can easily share the same root tree object, and we should still not add it more than once. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 8b02eeb - Browse repository at this point
Copy the full SHA 8b02eebView commit details -
backfill: add builtin boilerplate
In anticipation of implementing 'git backfill', populate the necessary files with the boilerplate of a new builtin. RFC TODO: When preparing this for a full implementation, make sure it is based on the newest standards introduced by [1]. [1] https://lore.kernel.org/git/xmqqjzfq2f0f.fsf@gitster.g/T/#m606036ea2e75a6d6819d6b5c90e729643b0ff7f7 [PATCH 1/3] builtin: add a repository parameter for builtin functions Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for a288333 - Browse repository at this point
Copy the full SHA a288333View commit details -
backfill: basic functionality and tests
The default behavior of 'git backfill' is to fetch all missing blobs that are reachable from HEAD. Document and test this behavior. The implementation is a very simple use of the path-walk API, initializing the revision walk at HEAD to start the path-walk from all commits reachable from HEAD. Ignore the object arrays that correspond to tree entries, assuming that they are all present already. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for e61870b - Browse repository at this point
Copy the full SHA e61870bView commit details -
backfill: add --batch-size=<n> option
Users may want to specify a minimum batch size for their needs. This is only a minimum: the path-walk API provides a list of OIDs that correspond to the same path, and thus it is optimal to allow delta compression across those objects in a single server request. We could consider limiting the request to have a maximum batch size in the future. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for c7df6b4 - Browse repository at this point
Copy the full SHA c7df6b4View commit details -
One way to significantly reduce the cost of a Git clone and later fetches is to use a blobless partial clone and combine that with a sparse-checkout that reduces the paths that need to be populated in the working directory. Not only does this reduce the cost of clones and fetches, the sparse-checkout reduces the number of objects needed to download from a promisor remote. However, history investigations can be expensie as computing blob diffs will trigger promisor remote requests for one object at a time. This can be avoided by downloading the blobs needed for the given sparse-checkout using 'git backfill' and its new '--sparse' mode, at a time that the user is willing to pay that extra cost. Note that this is distinctly different from the '--filter=sparse:<oid>' option, as this assumes that the partial clone has all reachable trees and we are using client-side logic to avoid downloading blobs outside of the sparse-checkout cone. This avoids the server-side cost of walking trees while also achieving a similar goal. It also downloads in batches based on similar path names, presenting a resumable download if things are interrupted. This augments the path-walk API to have a possibly-NULL 'pl' member that may point to a 'struct pattern_list'. This could be more general than the sparse-checkout definition at HEAD, but 'git backfill --sparse' is currently the only consumer. Be sure to test this in both cone mode and not cone mode. Cone mode has the benefit that the path-walk can skip certain paths once they would expand beyond the sparse-checkout. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 938bdf9 - Browse repository at this point
Copy the full SHA 938bdf9View commit details -
survey: stub in new experimental 'git-survey' command
Start work on a new 'git survey' command to scan the repository for monorepo performance and scaling problems. The goal is to measure the various known "dimensions of scale" and serve as a foundation for adding additional measurements as we learn more about Git monorepo scaling problems. The initial goal is to complement the scanning and analysis performed by the GO-based 'git-sizer' (https://github.com/github/git-sizer) tool. It is hoped that by creating a builtin command, we may be able to take advantage of internal Git data structures and code that is not accessible from GO to gain further insight into potential scaling problems. Co-authored-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Jeff Hostetler <git@jeffhostetler.com> Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 6d2fb8b - Browse repository at this point
Copy the full SHA 6d2fb8bView commit details -
backfill: assume --sparse when sparse-checkout is enabled
The previous change introduced the '--[no-]sparse' option for the 'git backfill' command, but did not assume it as enabled by default. However, this is likely the behavior that users will most often want to happen. Without this default, users with a small sparse-checkout may be confused when 'git backfill' downloads every version of every object in the full history. However, this is left as a separate change so this decision can be reviewed independently of the value of the '--[no-]sparse' option. Add a test of adding the '--sparse' option to a repo without sparse-checkout to make it clear that supplying it without a sparse-checkout is an error. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for b8cebf0 - Browse repository at this point
Copy the full SHA b8cebf0View commit details -
survey: add command line opts to select references
By default we will scan all references in "refs/heads/", "refs/tags/" and "refs/remotes/". Add command line opts let the use ask for all refs or a subset of them and to include a detached HEAD. Signed-off-by: Jeff Hostetler <git@jeffhostetler.com> Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 4e61f93 - Browse repository at this point
Copy the full SHA 4e61f93View commit details -
backfill: mark it as experimental
This is a highly useful command, and we want it to get some testing "in the wild". However, the patches have not yet been reviewed on the Git mailing list, and are therefore subject to change. By marking the command as experimental, users will be warned to pay attention to those changes. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 51b8ae8 - Browse repository at this point
Copy the full SHA 51b8ae8View commit details -
survey: start pretty printing data in table form
When 'git survey' provides information to the user, this will be presented in one of two formats: plaintext and JSON. The JSON implementation will be delayed until the functionality is complete for the plaintext format. The most important parts of the plaintext format are headers specifying the different sections of the report and tables providing concreted data. Create a custom table data structure that allows specifying a list of strings for the row values. When printing the table, check each column for the maximum width so we can create a table of the correct size from the start. The table structure is designed to be flexible to the different kinds of output that will be implemented in future changes. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 3c9a3dd - Browse repository at this point
Copy the full SHA 3c9a3ddView commit details -
survey: add object count summary
At the moment, nothing is obvious about the reason for the use of the path-walk API, but this will become more prevelant in future iterations. For now, use the path-walk API to sum up the counts of each kind of object. For example, this is the reachable object summary output for my local repo: REACHABLE OBJECT SUMMARY ======================== Object Type | Count ------------+------- Tags | 1343 Commits | 179344 Trees | 314350 Blobs | 184030 Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for dc4a5f7 - Browse repository at this point
Copy the full SHA dc4a5f7View commit details -
survey: summarize total sizes by object type
Now that we have explored objects by count, we can expand that a bit more to summarize the data for the on-disk and inflated size of those objects. This information is helpful for diagnosing both why disk space (and perhaps clone or fetch times) is growing but also why certain operations are slow because the inflated size of the abstract objects that must be processed is so large. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 61f5ff5 - Browse repository at this point
Copy the full SHA 61f5ff5View commit details -
survey: show progress during object walk
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 0714ece - Browse repository at this point
Copy the full SHA 0714eceView commit details -
survey: add ability to track prioritized lists
In future changes, we will make use of these methods. The intention is to keep track of the top contributors according to some metric. We don't want to store all of the entries and do a sort at the end, so track a constant-size table and remove rows that get pushed out depending on the chosen sorting algorithm. Co-authored-by: Jeff Hostetler <git@jeffhostetler.com> Signed-off-by; Jeff Hostetler <git@jeffhostetler.com> Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 76ae6b5 - Browse repository at this point
Copy the full SHA 76ae6b5View commit details -
survey: add report of "largest" paths
Since we are already walking our reachable objects using the path-walk API, let's now collect lists of the paths that contribute most to different metrics. Specifically, we care about * Number of versions. * Total size on disk. * Total inflated size (no delta or zlib compression). This information can be critical to discovering which parts of the repository are causing the most growth, especially on-disk size. Different packing strategies might help compress data more efficiently, but the toal inflated size is a representation of the raw size of all snapshots of those paths. Even when stored efficiently on disk, that size represents how much information must be processed to complete a command such as 'git blame'. Since the on-disk size is likely to be fragile, stop testing the exact output of 'git survey' and check that the correct set of headers is output. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 1eef810 - Browse repository at this point
Copy the full SHA 1eef810View commit details -
survey: add --top=<N> option and config
The 'git survey' builtin provides several detail tables, such as "top files by on-disk size". The size of these tables defaults to 100, currently. Allow the user to specify this number via a new --top=<N> option or the new survey.top config key. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 910d865 - Browse repository at this point
Copy the full SHA 910d865View commit details -
line-log: protect inner strbuf from free
The output_prefix() method in line-log.c may call a function pointer via the diff_options struct. This function pointer returns a strbuf struct and then its buffer is passed back. However, that implies that the consumer is responsible to free the string. This is especially true because the default behavior is to duplicate the empty string. The existing functions used in the output_prefix pointer include: 1. idiff_prefix_cb() in diff-lib.c. This returns the data pointer, so the value exists across multiple calls. 2. diff_output_prefix_callback() in graph.c. This uses a static strbuf struct, so it reuses buffers across calls. These should not be freed. 3. output_prefix_cb() in range-diff.c. This is similar to the diff-lib.c case. In each case, we should not be freeing this buffer. We can convert the output_prefix() function to return a const char pointer and stop freeing the result. This choice is essentially the opposite of what was done in 394affd (line-log: always allocate the output prefix, 2024-06-07). This was discovered via 'valgrind' while investigating a public report of a bug in 'git log --graph -L' [1]. [1] git-for-windows#5185 This issue would have been caught by the new test, when Git is compiled with ASan to catch these double frees. Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 1b6847f - Browse repository at this point
Copy the full SHA 1b6847fView commit details -
survey: clearly note the experimental nature in the output
While this command is definitely something we _want_, chances are that upstreaming this will require substantial changes. We still want to be able to experiment with this before that, to focus on what we need out of this command: To assist with diagnosing issues with large repositories, as well as to help monitoring the growth and the associated painpoints of such repositories. To that end, we are about to integrate this command into `microsoft/git`, to get the tool into the hands of users who need it most, with the idea to iterate in close collaboration between these users and the developers familar with Git's internals. However, we will definitely want to avoid letting anybody have the impression that this command, its exact inner workings, as well as its output format, are anywhere close to stable. To make that fact utterly clear (and thereby protect the freedom to iterate and innovate freely before upstreaming the command), let's mark its output as experimental in all-caps, as the first thing we do. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for a563945 - Browse repository at this point
Copy the full SHA a563945View commit details -
line-log: remove output_prefix()
Now that output_prefix() returns a const char * type, it matches the behavior of diff_line_prefix() and no longer needs to exist on its own. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for eff0ca0 - Browse repository at this point
Copy the full SHA eff0ca0View commit details -
diff: modify output_prefix function pointer
The uses of the output_prefix function pointer in the diff_options struct is currently difficult to work with by returning a pointer to a strbuf. There is only one use that cares about the length of the string, which appears to be the only justification of the return type. We already noticed confusing memory issues around this return type, so use a const char * return type to make it clear that the caller does not own this string buffer. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Derrick Stolee <stolee@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for db93c66 - Browse repository at this point
Copy the full SHA db93c66View commit details -
Merge branch 'safe-PATH-lookup-in-gitk-on-Windows'
This topic branch extends the protections introduced for Git GUI's CVE-2022-41953 to cover `gitk`, too. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for f77097a - Browse repository at this point
Copy the full SHA f77097aView commit details -
Merge 'remote-hg-prerequisites' into HEAD
These fixes were necessary for Sverre Rabbelier's remote-hg to work, but for some magic reason they are not necessary for the current remote-hg. Makes you wonder how that one gets away with it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for d3c7781 - Browse repository at this point
Copy the full SHA d3c7781View commit details -
This topic branch allows us to specify absolute paths without the drive prefix e.g. when cloning. Example: C:\Users\me> git clone https://github.com/git/git \upstream-git This will clone into a new directory C:\upstream-git, in line with how Windows interprets absolute paths. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 3259881 - Browse repository at this point
Copy the full SHA 3259881View commit details -
Merge branch 'dont-clean-junctions'
This topic branch teaches `git clean` to respect NTFS junctions and Unix bind mounts: it will now stop at those boundaries. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 4b5f1af - Browse repository at this point
Copy the full SHA 4b5f1afView commit details -
In MSYS2, we have two Python interpreters at our disposal, so we can include the Python stuff in the build. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 4731663 - Browse repository at this point
Copy the full SHA 4731663View commit details -
Merge branch 'mimalloc-v2.0.9'
This topic vendors in mimalloc v2.0.9, a fast allocator that allows Git for Windows to perform efficiently. Switch Git for Windows to using mimalloc instead of nedmalloc
Configuration menu - View commit details
-
Copy full SHA for f6ea2a7 - Browse repository at this point
Copy the full SHA f6ea2a7View commit details -
Merge pull request git-for-windows#2375 from assarbad/reintroduce-sid…
…eband-config Config option to disable side-band-64k for transport
Configuration menu - View commit details
-
Copy full SHA for 1351770 - Browse repository at this point
Copy the full SHA 1351770View commit details -
Merge pull request git-for-windows#2405 from dscho/mingw-setsockopt
Make sure `errno` is set when socket operations fail
Configuration menu - View commit details
-
Copy full SHA for 5630f19 - Browse repository at this point
Copy the full SHA 5630f19View commit details -
Merge pull request git-for-windows#2449 from dscho/mingw-getcwd-and-s…
…ymlinks Do resolve symlinks in `getcwd()`
Configuration menu - View commit details
-
Copy full SHA for ae5ba9c - Browse repository at this point
Copy the full SHA ae5ba9cView commit details -
Merge pull request git-for-windows#2488 from bmueller84/master
mingw: fix fatal error working on mapped network drives on Windows
Configuration menu - View commit details
-
Copy full SHA for 6600b1a - Browse repository at this point
Copy the full SHA 6600b1aView commit details -
Merge pull request git-for-windows#2501 from jeffhostetler/clink-debu…
…g-curl clink.pl: fix MSVC compile script to handle libcurl-d.lib
Configuration menu - View commit details
-
Copy full SHA for 3f7fa1d - Browse repository at this point
Copy the full SHA 3f7fa1dView commit details -
Merge pull request git-for-windows#2504 from dscho/access-repo-via-ju…
…nction Handle `git add <file>` where <file> traverses an NTFS junction
Configuration menu - View commit details
-
Copy full SHA for e99cdcd - Browse repository at this point
Copy the full SHA e99cdcdView commit details -
Merge pull request git-for-windows#2506 from dscho/issue-2283
Allow running Git directly from `C:\Program Files\Git\mingw64\bin\git.exe`
Configuration menu - View commit details
-
Copy full SHA for 1587bd1 - Browse repository at this point
Copy the full SHA 1587bd1View commit details -
Merge pull request git-for-windows#2535 from dscho/schannel-revoke-be…
…st-effort Introduce and use the new "best effort" strategy for Secure Channel revoke checking
Configuration menu - View commit details
-
Copy full SHA for 3059223 - Browse repository at this point
Copy the full SHA 3059223View commit details -
Merge pull request git-for-windows#2618 from dscho/avoid-d/f-conflict…
…-in-vs/master ci: avoid d/f conflict in vs/master
Configuration menu - View commit details
-
Copy full SHA for 270f7f4 - Browse repository at this point
Copy the full SHA 270f7f4View commit details -
This topic branch allows `add -p` and `add -i` with a large number of files. It is kind of a hack that was never really meant to be upstreamed. Let's see if we can do better in the built-in `add -p`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 876026a - Browse repository at this point
Copy the full SHA 876026aView commit details -
Merge pull request git-for-windows#2730 from dscho/crlf-aware-git-add-i
git add -i: handle CR/LF line endings in the interactive input
Configuration menu - View commit details
-
Copy full SHA for 54386d6 - Browse repository at this point
Copy the full SHA 54386d6View commit details -
Merge pull request git-for-windows#2714 from lbonanomi/crlf-scissors
Rationalize line endings for scissors-cleanup
Configuration menu - View commit details
-
Copy full SHA for 6a320e0 - Browse repository at this point
Copy the full SHA 6a320e0View commit details -
Merge pull request git-for-windows#2655 from jglathe/jg/t0014_trace_e…
…xtra_info t/t0014: fix: eliminate additional lines from trace
Configuration menu - View commit details
-
Copy full SHA for 7ff3d47 - Browse repository at this point
Copy the full SHA 7ff3d47View commit details -
Merge 'git-gui/js/intent-to-add'
This merges the current version of the patch that tries to address Git GUI's problems with intent-to-add files. This patch will likely be improved substantially before it is merged into Git GUI's main branch, but we want to have _something_ resembling a fix already in Git for Windows v2.29.0. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 6f512cd - Browse repository at this point
Copy the full SHA 6f512cdView commit details -
Merge pull request git-for-windows#2974 from derrickstolee/maintenanc…
…e-and-headless Include Windows-specific maintenance and headless-git
Configuration menu - View commit details
-
Copy full SHA for f0e44dd - Browse repository at this point
Copy the full SHA f0e44ddView commit details -
cmake: install headless-git. (git-for-windows#4338)
Even if CMake is not the canonical way to build Git for Windows, but CMake support merely exists in Git to support building Git for Windows using Visual Studio, we should include `headless-git` in such a scenario when installing the binaries to a given location.
Configuration menu - View commit details
-
Copy full SHA for 4877c51 - Browse repository at this point
Copy the full SHA 4877c51View commit details -
Merge pull request git-for-windows#2351 from PhilipOakley/vcpkg-tip
Vcpkg Install: detect lack of working Git, and note possible vcpkg time outs
Configuration menu - View commit details
-
Copy full SHA for 54be4aa - Browse repository at this point
Copy the full SHA 54be4aaView commit details -
Merge pull request git-for-windows#2915 from dennisameling/windows-ar…
…m64-support Windows arm64 support
Configuration menu - View commit details
-
Copy full SHA for 1984428 - Browse repository at this point
Copy the full SHA 1984428View commit details -
Merge pull request git-for-windows#3327 from dennisameling/fix-host-cpu
cmake(): allow setting HOST_CPU for cross-compilation
Configuration menu - View commit details
-
Copy full SHA for bb450f4 - Browse repository at this point
Copy the full SHA bb450f4View commit details -
Merge pull request git-for-windows#3165 from dscho/increase-allowed-l…
…ength-of-interpreter-path mingw: allow for longer paths in `parse_interpreter()`
Configuration menu - View commit details
-
Copy full SHA for 24e816a - Browse repository at this point
Copy the full SHA 24e816aView commit details -
Merge pull request git-for-windows#3220 from dscho/there-is-no-vs/mas…
…ter-anymore Let the documentation reflect that there is no vs/master anymore
Configuration menu - View commit details
-
Copy full SHA for 82f250d - Browse repository at this point
Copy the full SHA 82f250dView commit details -
Merge pull request git-for-windows#3293 from pascalmuller/http-suppor…
…t-automatically-sending-client-certificate http: Add support for enabling automatic sending of SSL client certificate
Configuration menu - View commit details
-
Copy full SHA for 88864c0 - Browse repository at this point
Copy the full SHA 88864c0View commit details -
Merge pull request git-for-windows#3349 from vdye/feature/ci-subtree-…
…tests Add `contrib/subtree` test execution to CI builds
Configuration menu - View commit details
-
Copy full SHA for 85472a9 - Browse repository at this point
Copy the full SHA 85472a9View commit details -
Merge pull request git-for-windows#3306 from PhilipOakley/vs-sln
Make Git for Windows start builds in modern Visual Studio
Configuration menu - View commit details
-
Copy full SHA for e7ee159 - Browse repository at this point
Copy the full SHA e7ee159View commit details -
Merge pull request git-for-windows#3417 from dscho/initialize-core.sy…
…mlinks-earlier init: respect core.symlinks before copying the templates
Configuration menu - View commit details
-
Copy full SHA for 2f8fcbb - Browse repository at this point
Copy the full SHA 2f8fcbbView commit details -
Merge pull request git-for-windows#3533 from PhilipOakley/hashliteral_t
Begin `unsigned long`->`size_t` conversion to support large files on Windows
Configuration menu - View commit details
-
Copy full SHA for 9805f8a - Browse repository at this point
Copy the full SHA 9805f8aView commit details -
Merge pull request git-for-windows#3791: Various fixes around `safe.d…
…irectory` The first three commits are rebased versions of those in gitgitgadget#1215. These allow the following: 1. Fix `git config --global foo.bar <path>` from allowing the `<path>`. As a bonus, users with a config value starting with `/` will not get a warning about "old-style" paths needing a "`%(prefix)/`". 2. When in WSL, the path starts with `/` so it needs to be interpolated properly. Update the warning to include `%(prefix)/` to get the right value for WSL users. (This is specifically for using Git for Windows from Git Bash, but in a WSL directory.) 3. When using WSL, the ownership check fails and reports an error message. This is noisy, and happens even if the user has marked the path with `safe.directory`. Remove that error message.
Configuration menu - View commit details
-
Copy full SHA for 1908d17 - Browse repository at this point
Copy the full SHA 1908d17View commit details -
Merge pull request git-for-windows#3751 from rkitover/native-term
mingw: set $env:TERM=xterm-256color for newer OSes
Configuration menu - View commit details
-
Copy full SHA for 3968416 - Browse repository at this point
Copy the full SHA 3968416View commit details -
Merge pull request git-for-windows#3875 from 1480c1/wine/detect_msys_tty
winansi: check result before using Name for pty
Configuration menu - View commit details
-
Copy full SHA for d5a7a36 - Browse repository at this point
Copy the full SHA d5a7a36View commit details -
Merge branch 'optionally-dont-append-atomically-on-windows'
Fix append failure issue under remote directories git-for-windows#2753 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 887a326 - Browse repository at this point
Copy the full SHA 887a326View commit details -
Merge branch 'fsync-object-files-always'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 43392d7 - Browse repository at this point
Copy the full SHA 43392d7View commit details -
Backport a couple fixes to make the CI build run again (so much for reproducible builds...). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 047f7b4 - Browse repository at this point
Copy the full SHA 047f7b4View commit details -
Merge pull request git-for-windows#3942 from rimrul/mingw-tsaware
MinGW: link as terminal server aware
Configuration menu - View commit details
-
Copy full SHA for 8505d76 - Browse repository at this point
Copy the full SHA 8505d76View commit details -
Fix Windows version resources (git-for-windows#4092)
Add `FileVersion`, which is a required string ([Microsoft documentation](https://learn.microsoft.com/en-us/windows/win32/menurc/versioninfo-resource)) in the `StringFileInfo` block. As not all required strings were present in the block, none were being included. Fixes git-for-windows#4090 After including the `FileVersion` string, all other defined strings are now being included on executables. File version information for `git.exe` has changed from: ``` PS C:\Program Files\Git\bin> [System.Diagnostics.FileVersionInfo]::GetVersionInfo("C:\Data\git-sdk-64\usr\src\git\git.exe") | Select-Object * FileVersionRaw : 2.38.1.1 ProductVersionRaw : 2.38.1.1 Comments : CompanyName : FileBuildPart : 1 FileDescription : FileMajorPart : 2 FileMinorPart : 38 FileName : C:\Data\git-sdk-64\usr\src\git\git.exe FilePrivatePart : 1 FileVersion : InternalName : IsDebug : False IsPatched : False IsPrivateBuild : False IsPreRelease : False IsSpecialBuild : False Language : English (United States) LegalCopyright : LegalTrademarks : OriginalFilename : PrivateBuild : ProductBuildPart : 1 ProductMajorPart : 2 ProductMinorPart : 38 ProductName : ProductPrivatePart : 1 ProductVersion : SpecialBuild : ``` To the following: ``` PS C:\Program Files\Git\bin> [System.Diagnostics.FileVersionInfo]::GetVersionInfo("C:\Data\git-sdk-64\usr\src\git\git.exe") | Select-Object * FileVersionRaw : 2.38.1.1 ProductVersionRaw : 2.38.1.1 Comments : CompanyName : The Git Development Community FileBuildPart : 1 FileDescription : Git for Windows FileMajorPart : 2 FileMinorPart : 38 FileName : C:\Data\git-sdk-64\usr\src\git\git.exe FilePrivatePart : 1 FileVersion : 2.38.1.windows.1.10.g6ed65a6fab InternalName : git IsDebug : False IsPatched : False IsPrivateBuild : False IsPreRelease : False IsSpecialBuild : False Language : English (United States) LegalCopyright : LegalTrademarks : OriginalFilename : git.exe PrivateBuild : ProductBuildPart : 1 ProductMajorPart : 2 ProductMinorPart : 38 ProductName : Git ProductPrivatePart : 1 ProductVersion : 2.38.1.windows.1.10.g6ed65a6fab SpecialBuild : ``` I wasn't really expecting `GIT_VERSION` to contain the Git commit, I was hoping for just `2.38.1` or `2.38.1.1`, at least for the `FileVersion` string. Anybody know if it's possible to concatenate the `MAJOR`, `MINOR`, `MICRO`, and `PATCHLEVEL` fields with dots, or if there's another variable that can be used (with or without `PATCHLEVEL`)? Alternatively, use the complete `GIT_VERSION` for both `FileVersion` and `ProductVersion`.
Configuration menu - View commit details
-
Copy full SHA for 9af5c1d - Browse repository at this point
Copy the full SHA 9af5c1dView commit details -
Merge branch 'builtin-swap-functions'
Do prefer GCC's built-in bit-swap functions. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for ff8b985 - Browse repository at this point
Copy the full SHA ff8b985View commit details -
Add support for CLANGARM64 target (git-for-windows#3916)
**This requires an ARM64-machine with Windows 11 installed (which supports x64 emulation for MSYS2)** ### The main idea - Use the main MSYS2/git-sdk-64 setup, which works on Windows 11 on ARM thanks to x64-emulation - Configure the official `clangarm64` MSYS2 repo - Install `mingw-w64-clang-aarch64-toolchain` which contains the ARM64-native Clang compiler
Configuration menu - View commit details
-
Copy full SHA for e8c362b - Browse repository at this point
Copy the full SHA e8c362bView commit details -
Fix global repository field not being cleared (git-for-windows#4083)
It is checked for w.r.t. global repository struct down in the callstack in compatibility layer for MinGW before being assigned in the function that `free()`'d it.
Configuration menu - View commit details
-
Copy full SHA for 56b7a4a - Browse repository at this point
Copy the full SHA 56b7a4aView commit details -
Skip linking the "dashed"
git-<command>
s for built-ins (git-for-win……dows#4252) It is merely a historical wart that, say, `git-commit` exists in the `libexec/git-core/` directory, a tribute to the original idea to let Git be essentially a bunch of Unix shell scripts revolving around very few "plumbing" (AKA low-level) commands. Git has evolved a lot from there. These days, most of Git's functionality is contained within the `git` executable, in the form of "built-in" commands. To accommodate for scripts that use the "dashed" form of Git commands, even today, Git provides hard-links that make the `git` executable available as, say, `git-commit`, just in case that an old script has not been updated to invoke `git commit`. Those hard-links do not come cheap: they take about half a minute for every build of Git on Windows, they are mistaken for taking up huge amounts of space by some Windows Explorer versions that do not understand hard-links, and therefore many a "bug" report had to be addressed. The "dashed form" has been officially deprecated in Git version 1.5.4, which was released on February 2nd, 2008, i.e. a very long time ago. This deprecation was never finalized by skipping these hard-links, but we can start the process now, in Git for Windows. This addresses the concern raised in git-for-windows#4185 (comment)
Configuration menu - View commit details
-
Copy full SHA for d49561f - Browse repository at this point
Copy the full SHA d49561fView commit details -
Git GUI: fix Repository>Explore Working Copy (git-for-windows#4357)
This is a companion PR of prati0100/git-gui#95 Since Git v2.39.1, we are a bit more stringent in searching the PATH. In particular, we specifically require the `.exe` suffix. However, the `Repository>Explore Working Copy` command asks for `explorer.exe` to be found on the `PATH`, which _already_ has that suffix. Let's unstartle the PATH-finding logic about this scenario. This fixes git-for-windows#4356
Configuration menu - View commit details
-
Copy full SHA for ebb4d3f - Browse repository at this point
Copy the full SHA ebb4d3fView commit details -
Lazy load libcurl, allowing for an SSL/TLS backend-specific libcurl (g…
…it-for-windows#4410) As per git-for-windows#4350 (comment), the major block for upgrading Git for Windows' OpenSSL from v1.1 to v3 is the tricky part where such an upgrade would break `git fetch`/`git clone` and `git push` because the libcurl depends on the OpenSSL DLL, and the major version bump will _change_ the file name of said DLL. To overcome that, the plan is to build libcurl flavors for each supported SSL/TLS backend, aligning with the way MSYS2 builds libcurl, then switch Git for Windows' SDK to the Secure Channel-flavored libcurl, and teach Git to look for the specific flavor of libcurl corresponding to the `http.sslBackend` setting (if that was configured). Here is the PR to teach Git that trick.
Configuration menu - View commit details
-
Copy full SHA for 1bdf694 - Browse repository at this point
Copy the full SHA 1bdf694View commit details -
This patch adds a GitHub workflow (to be triggered manually) to allow for conveniently verifying that Git and Scalar still work as intended in Windows Nano Server (a relatively small container base image that is frequently used where a "small Windows" is needed, e.g. in automation ;-)) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 47a1180 - Browse repository at this point
Copy the full SHA 47a1180View commit details -
Additional error checks for issuing the windows.appendAtomically warn…
…ing (git-for-windows#4528) Another (hopefully clean) PR for showing the error warning about atomic append on windows after failure on APFS, which returns EBADF not EINVAL. Signed-off-by: David Lomas <dl3@pale-eds.co.uk> Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for df5f41d - Browse repository at this point
Copy the full SHA df5f41dView commit details -
win32: use native ANSI sequence processing, if possible (git-for-wind…
…ows#4700) Windows 10 version 1511 (also known as Anniversary Update), according to https://learn.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences introduced native support for ANSI sequence processing. This allows using colors from the entire 24-bit color range. All we need to do is test whether the console's "virtual processing support" can be enabled. If it can, we do not even need to start the `console_thread` to handle ANSI sequences. Incidentally, this addresses git-for-windows#3712.
Configuration menu - View commit details
-
Copy full SHA for 4da4c9f - Browse repository at this point
Copy the full SHA 4da4c9fView commit details -
ARM64: Embed manifest properly (git-for-windows#4718)
Teach our ARM64 based builds to embed the manifest file correctly. This fixes git-for-windows#4707
Configuration menu - View commit details
-
Copy full SHA for 8b72a3c - Browse repository at this point
Copy the full SHA 8b72a3cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 744024e - Browse repository at this point
Copy the full SHA 744024eView commit details -
Merge branch 'run-t5601-and-t7406-with-symlinks-on-windows-10'
This topic branch contains a patch that made it into Git for Windows v2.45.1 but not into Git v2.45.1 (because the latter does not come with symlink support on Windows). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 519ce46 - Browse repository at this point
Copy the full SHA 519ce46View commit details -
Merge branch 'Fix-i686-build-with-GCC-v14'
This fixes a long-time compile warning turned error by GCC v14. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for f3a25a8 - Browse repository at this point
Copy the full SHA f3a25a8View commit details -
Merge branch 'Fallback-to-AppData-if-XDG-CONFIG-HOME-is-unset'
This topic branch adds support for a more Windows-native user-wide config file than `XDG_CONFIG_HOME` (or `~/.config/`) will ever be. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for b32ad2e - Browse repository at this point
Copy the full SHA b32ad2eView commit details -
Merge branch 'run-command-be-helpful-when-Git-LFS-fails-on-Windows-7'
Since Git LFS v3.5.x implicitly dropped Windows 7 support, we now want users to be advised _what_ is going wrong on that Windows version. This topic branch goes out of its way to provide users with such guidance. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 15972c5 - Browse repository at this point
Copy the full SHA 15972c5View commit details -
pack-objects: create new name-hash algorithm (git-for-windows#5157)
This is an updated version of gitgitgadget#1785, intended for early consumption into Git for Windows. The idea here is to add a new `--full-name-hash` option to `git pack-objects` and `git repack`. This adjusts the name-hash value used for finding delta bases in such a way that uses the full path name with a lower likelihood of collisions than the default name-hash algorithm. In many repositories with name-hash collisions and many versions of those paths, this can significantly reduce the size of a full repack. It can also help in certain cases of `git push`, but only if the pack is already artificially inflated by name-hash collisions; cases that find "sibling" deltas as better choices become worse with `--full-name-hash`. Thus, this option is currently recommended for full repacks of large repos, and on client machines without reachability bitmaps. Some care is taken to ignore this option when using bitmaps, either writing bitmaps or using a bitmap walk during reads. The bitmap file format contains name-hash values, but no way to indicate which function is used, so compatibility is a concern for bitmaps. Future work could explore this idea. After this PR is merged, then the more-involved `--path-walk` option may be considered.
Configuration menu - View commit details
-
Copy full SHA for 1f75682 - Browse repository at this point
Copy the full SHA 1f75682View commit details -
Add path walk API and its use in 'git pack-objects' (git-for-windows#…
…5171) This is a follow up to git-for-windows#5157 as well as motivated by the RFC in gitgitgadget#1786. We have ways of walking all objects, but it is focused on visiting a single commit and then expanding the new trees and blobs reachable from that commit that have not been visited yet. This means that objects arrive without any locality based on their path. Add a new "path walk API" that focuses on walking objects in batches according to their type and path. This will walk all annotated tags, all commits, all root trees, and then start a depth-first search among all paths in the repo to collect trees and blobs in batches. The most important application for this is being fast-tracked to Git for Windows: `git pack-objects --path-walk`. This application of the path walk API discovers the objects to pack via this batched walk, and automatically groups objects that appear at a common path so they can be checked for delta comparisons. This use completely avoids any name-hash collisions (even the collisions that sometimes occur with the new `--full-name-hash` option) and can be much faster to compute since the first pass of delta calculations does not waste time on objects that are unlikely to be diffable. Some statistics are available in the commit messages.
Configuration menu - View commit details
-
Copy full SHA for 68d135b - Browse repository at this point
Copy the full SHA 68d135bView commit details -
path-walk API: avoid adding a root tree more than once (git-for-windo…
…ws#5195) When adding tree objects, we are very careful to avoid adding the same tree object more than once. There was one small gap in that logic, though: when adding a root tree object. Two refs can easily share the same root tree object, and we should still not add it more than once.
Configuration menu - View commit details
-
Copy full SHA for a0b4e54 - Browse repository at this point
Copy the full SHA a0b4e54View commit details -
Introduce 'git backfill' to get missing blobs in a partial clone (git…
…-for-windows#5172) This change introduces the `git backfill` command which uses the path walk API to download missing blobs in a blobless partial clone. By downloading blobs that correspond to the same file path at the same time, we hope to maximize the potential benefits of delta compression against multiple versions. These downloads occur in a configurable batch size, presenting a mechanism to perform "resumable" clones: `git clone --filter=blob:none` gets the commits and trees, then `git backfill` will download all missing blobs. If `git backfill` is interrupted partway through, it can be restarted and will redownload only the missing objects. When combining blobless partial clones with sparse-checkout, `git backfill` will assume its `--sparse` option and download only the blobs within the sparse-checkout. Users may want to do this as the repo size will still be smaller than the full repo size, but commands like `git blame` or `git log -L` will not suffer from many one-by-one blob downloads. Future directions should consider adding a pathspec or file prefix to further focus which paths are being downloaded in a batch.
Configuration menu - View commit details
-
Copy full SHA for 8e758c4 - Browse repository at this point
Copy the full SHA 8e758c4View commit details -
Add experimental 'git survey' builtin (git-for-windows#5174)
This introduces `git survey` to Git for Windows ahead of upstream for the express purpose of getting the path-based analysis in the hands of more folks. The inspiration of this builtin is [`git-sizer`](https://github.com/github/git-sizer), but since that command relies on `git cat-file --batch` to get the contents of objects, it has limits to how much information it can provide. This is mostly a rewrite of the `git survey` builtin that was introduced into the `microsoft/git` fork in microsoft#667. That version had a lot more bells and whistles, including an analysis much closer to what `git-sizer` provides. The biggest difference in this version is that this one is focused on using the path-walk API in order to visit batches of objects based on a common path. This allows identifying, for instance, the path that is contributing the most to the on-disk size across all versions at that path. For example, here are the top ten paths contributing to my local Git repository (which includes `microsoft/git` and `gitster/git`): ``` TOP FILES BY DISK SIZE ============================================================================ Path | Count | Disk Size | Inflated Size -----------------------------------------+-------+-----------+-------------- whats-cooking.txt | 1373 | 11637459 | 37226854 t/helper/test-gvfs-protocol | 2 | 6847105 | 17233072 git-rebase--helper | 1 | 6027849 | 15269664 compat/mingw.c | 6111 | 5194453 | 463466970 t/helper/test-parse-options | 1 | 3420385 | 8807968 t/helper/test-pkt-line | 1 | 3408661 | 8778960 t/helper/test-dump-untracked-cache | 1 | 3408645 | 8780816 t/helper/test-dump-fsmonitor | 1 | 3406639 | 8776656 po/vi.po | 104 | 1376337 | 51441603 po/de.po | 210 | 1360112 | 71198603 ``` This kind of analysis has been helpful in identifying the reasons for growth in a few internal monorepos. Those findings motivated the changes in git-for-windows#5157 and git-for-windows#5171. With this early version in Git for Windows, we can expand the reach of the experimental tool in advance of it being contributed to the upstream project. Unfortunately, this will mean that in the next `microsoft/git` rebase, @jeffhostetler's version will need to be pulled out since there are enough conflicts. These conflicts include how tables are stored and generated, as the version in this PR is slightly more general to allow for different kinds of data.
Configuration menu - View commit details
-
Copy full SHA for b5869d3 - Browse repository at this point
Copy the full SHA b5869d3View commit details -
Fix
git log --graph -u
hangs (git-for-windows#5193)This fixes git-for-windows#5185 by backporting gitgitgadget#1806 (which, sadly, seems not to have made it into Git v2.47.0).
Configuration menu - View commit details
-
Copy full SHA for a8e465c - Browse repository at this point
Copy the full SHA a8e465cView commit details -
Win32: make FILETIME conversion functions public
We will use them in the upcoming "FSCache" patches (to accelerate sequential lstat() calls). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 2e6b871 - Browse repository at this point
Copy the full SHA 2e6b871View commit details -
Merge branch 'ready-for-upstream'
This is the branch thicket of patches in Git for Windows that are considered ready for upstream. To keep them in a ready-to-submit shape, they are kept as close to the beginning of the branch thicket as possible.
Configuration menu - View commit details
-
Copy full SHA for 0d79092 - Browse repository at this point
Copy the full SHA 0d79092View commit details -
Win32: dirent.c: Move opendir down
Move opendir down in preparation for the next patch. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for dcfad57 - Browse repository at this point
Copy the full SHA dcfad57View commit details -
mingw: make the dirent implementation pluggable
Emulating the POSIX `dirent` API on Windows via `FindFirstFile()`/`FindNextFile()` is pretty staightforward, however, most of the information provided in the `WIN32_FIND_DATA` structure is thrown away in the process. A more sophisticated implementation may cache this data, e.g. for later reuse in calls to `lstat()`. Make the `dirent` implementation pluggable so that it can be switched at runtime, e.g. based on a config option. Define a base DIR structure with pointers to `readdir()`/`closedir()` that match the `opendir()` implementation (similar to vtable pointers in Object-Oriented Programming). Define `readdir()`/`closedir()` so that they call the function pointers in the `DIR` structure. This allows to choose the `opendir()` implementation on a call-by-call basis. Make the fixed-size `dirent.d_name` buffer a flex array, as `d_name` may be implementation specific (e.g. a caching implementation may allocate a `struct dirent` with _just_ the size needed to hold the `d_name` in question). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 0077575 - Browse repository at this point
Copy the full SHA 0077575View commit details -
Win32: make the lstat implementation pluggable
Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite slow. Windows operating system APIs seem to be much better at scanning the status of entire directories than checking single files. A caching implementation may improve performance by bulk-reading entire directories or reusing data obtained via opendir / readdir. Make the lstat implementation pluggable so that it can be switched at runtime, e.g. based on a config option. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 95bccd8 - Browse repository at this point
Copy the full SHA 95bccd8View commit details -
mingw: add infrastructure for read-only file system level caches
Add a macro to mark code sections that only read from the file system, along with a config option and documentation. This facilitates implementation of relatively simple file system level caches without the need to synchronize with the file system. Enable read-only sections for 'git status' and preload_index. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for 98e220a - Browse repository at this point
Copy the full SHA 98e220aView commit details -
mingw: add a cache below mingw's lstat and dirent implementations
Checking the work tree status is quite slow on Windows, due to slow `lstat()` emulation (git calls `lstat()` once for each file in the index). Windows operating system APIs seem to be much better at scanning the status of entire directories than checking single files. Add an `lstat()` implementation that uses a cache for lstat data. Cache misses read the entire parent directory and add it to the cache. Subsequent `lstat()` calls for the same directory are served directly from the cache. Also implement `opendir()`/`readdir()`/`closedir()` so that they create and use directory listings in the cache. The cache doesn't track file system changes and doesn't plug into any modifying file APIs, so it has to be explicitly enabled for git functions that don't modify the working copy. Note: in an earlier version of this patch, the cache was always active and tracked file system changes via ReadDirectoryChangesW. However, this was much more complex and had negative impact on the performance of modifying git commands such as 'git checkout'. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 028efda - Browse repository at this point
Copy the full SHA 028efdaView commit details -
fscache: load directories only once
If multiple threads access a directory that is not yet in the cache, the directory will be loaded by each thread. Only one of the results is added to the cache, all others are leaked. This wastes performance and memory. On cache miss, add a future object to the cache to indicate that the directory is currently being loaded. Subsequent threads register themselves with the future object and wait. When the first thread has loaded the directory, it replaces the future object with the result and notifies waiting threads. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for d4f0444 - Browse repository at this point
Copy the full SHA d4f0444View commit details -
fscache: add key for GIT_TRACE_FSCACHE
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for e45f8ad - Browse repository at this point
Copy the full SHA e45f8adView commit details -
fscache: remember not-found directories
Teach FSCACHE to remember "not found" directories. This is a performance optimization. FSCACHE is a performance optimization available for Windows. It intercepts Posix-style lstat() calls into an in-memory directory using FindFirst/FindNext. It improves performance on Windows by catching the first lstat() call in a directory, using FindFirst/ FindNext to read the list of files (and attribute data) for the entire directory into the cache, and short-cut subsequent lstat() calls in the same directory. This gives a major performance boost on Windows. However, it does not remember "not found" directories. When STATUS runs and there are missing directories, the lstat() interception fails to find the parent directory and simply return ENOENT for the file -- it does not remember that the FindFirst on the directory failed. Thus subsequent lstat() calls in the same directory, each re-attempt the FindFirst. This completely defeats any performance gains. This can be seen by doing a sparse-checkout on a large repo and then doing a read-tree to reset the skip-worktree bits and then running status. This change reduced status times for my very large repo by 60%. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for d00800e - Browse repository at this point
Copy the full SHA d00800eView commit details -
dir.c: make add_excludes aware of fscache during status
Teach read_directory_recursive() and add_excludes() to be aware of optional fscache and avoid trying to open() and fstat() non-existant ".gitignore" files in every directory in the worktree. The current code in add_excludes() calls open() and then fstat() for a ".gitignore" file in each directory present in the worktree. Change that when fscache is enabled to call lstat() first and if present, call open(). This seems backwards because both lstat needs to do more work than fstat. But when fscache is enabled, fscache will already know if the .gitignore file exists and can completely avoid the IO calls. This works because of the lstat diversion to mingw_lstat when fscache is enabled. This reduced status times on a 350K file enlistment of the Windows repo on a NVMe SSD by 0.25 seconds. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for fda16c8 - Browse repository at this point
Copy the full SHA fda16c8View commit details -
fscache: add a test for the dir-not-found optimization
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for ade3ba7 - Browse repository at this point
Copy the full SHA ade3ba7View commit details -
add: use preload-index and fscache for performance
Teach "add" to use preload-index and fscache features to improve performance on very large repositories. During an "add", a call is made to run_diff_files() which calls check_remove() for each index-entry. This calls lstat(). On Windows, the fscache code intercepts the lstat() calls and builds a private cache using the FindFirst/FindNext routines, which are much faster. Somewhat independent of this, is the preload-index code which distributes some of the start-up costs across multiple threads. We need to keep the call to read_cache() before parsing the pathspecs (and hence cannot use the pathspecs to limit any preload) because parse_pathspec() is using the index to determine whether a pathspec is, in fact, in a submodule. If we would not read the index first, parse_pathspec() would not error out on a path that is inside a submodule, and t7400-submodule-basic.sh would fail with not ok 47 - do not add files from a submodule We still want the nice preload performance boost, though, so we simply call read_cache_preload(&pathspecs) after parsing the pathspecs. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 53de33f - Browse repository at this point
Copy the full SHA 53de33fView commit details -
fscache: make fscache_enabled() public
Make fscache_enabled() function public rather than static. Remove unneeded fscache_is_enabled() function. Change is_fscache_enabled() macro to call fscache_enabled(). is_fscache_enabled() now takes a pathname so that the answer is more precise and mean "is fscache enabled for this pathname", since fscache only stores repo-relative paths and not absolute paths, we can avoid attempting lookups for absolute paths. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 3826f0d - Browse repository at this point
Copy the full SHA 3826f0dView commit details -
dir.c: regression fix for add_excludes with fscache
Fix regression described in: git-for-windows#1392 which was introduced in: git-for-windows@b235337 Problem Symptoms ================ When the user has a .gitignore file that is a symlink, the fscache optimization introduced above caused the stat-data from the symlink, rather that of the target file, to be returned. Later when the ignore file was read, the buffer length did not match the stat.st_size field and we called die("cannot use <path> as an exclude file") Optimization Rationale ====================== The above optimization calls lstat() before open() primarily to ask fscache if the file exists. It gets the current stat-data as a side effect essentially for free (since we already have it in memory). If the file does not exist, it does not need to call open(). And since very few directories have .gitignore files, we can greatly reduce time spent in the filesystem. Discussion of Fix ================= The above optimization calls lstat() rather than stat() because the fscache only intercepts lstat() calls. Calls to stat() stay directed to the mingw_stat() completly bypassing fscache. Furthermore, calls to mingw_stat() always call {open, fstat, close} so that symlinks are properly dereferenced, which adds *additional* open/close calls on top of what the original code in dir.c is doing. Since the problem only manifests for symlinks, we add code to overwrite the stat-data when the path is a symlink. This preserves the effect of the performance gains provided by the fscache in the normal case. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for eeead24 - Browse repository at this point
Copy the full SHA eeead24View commit details -
fetch-pack.c: enable fscache for stats under .git/objects
When I do git fetch, git call file stats under .git/objects for each refs. This takes time when there are many refs. By enabling fscache, git takes file stats by directory traversing and that improved the speed of fetch-pack for repository having large number of refs. In my windows workstation, this improves the time of `git fetch` for chromium repository like below. I took stats 3 times. * With this patch TotalSeconds: 9.9825165 TotalSeconds: 9.1862075 TotalSeconds: 10.1956256 Avg: 9.78811653333333 * Without this patch TotalSeconds: 15.8406702 TotalSeconds: 15.6248053 TotalSeconds: 15.2085938 Avg: 15.5580231 Signed-off-by: Takuto Ikuta <tikuta@chromium.org> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7973ab2 - Browse repository at this point
Copy the full SHA 7973ab2View commit details -
checkout.c: enable fscache for checkout again
This is retry of git-for-windows#1419. I added flush_fscache macro to flush cached stats after disk writing with tests for regression reported in git-for-windows#1438 and git-for-windows#1442. git checkout checks each file path in sorted order, so cache flushing does not make performance worse unless we have large number of modified files in a directory containing many files. Using chromium repository, I tested `git checkout .` performance when I delete 10 files in different directories. With this patch: TotalSeconds: 4.307272 TotalSeconds: 4.4863595 TotalSeconds: 4.2975562 Avg: 4.36372923333333 Without this patch: TotalSeconds: 20.9705431 TotalSeconds: 22.4867685 TotalSeconds: 18.8968292 Avg: 20.7847136 I confirmed this patch passed all tests in t/ with core_fscache=1. Signed-off-by: Takuto Ikuta <tikuta@chromium.org>
Configuration menu - View commit details
-
Copy full SHA for bb923bb - Browse repository at this point
Copy the full SHA bb923bbView commit details -
Enable the filesystem cache (fscache) in refresh_index().
On file systems that support it, this can dramatically speed up operations like add, commit, describe, rebase, reset, rm that would otherwise have to lstat() every file to "re-match" the stat information in the index to that of the file system. On a synthetic repo with 1M files, "git reset" dropped from 52.02 seconds to 14.42 seconds for a savings of 72%. Signed-off-by: Ben Peart <benpeart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 6f5ebad - Browse repository at this point
Copy the full SHA 6f5ebadView commit details -
fscache: use FindFirstFileExW to avoid retrieving the short name
Use FindFirstFileExW with FindExInfoBasic to avoid forcing NTFS to look up the short name. Also switch to a larger (64K vs 4K) buffer using FIND_FIRST_EX_LARGE_FETCH to minimize round trips to the kernel. In a repo with ~200K files, this drops warm cache status times from 3.19 seconds to 2.67 seconds for a 16% savings. Signed-off-by: Ben Peart <benpeart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 5517a9f - Browse repository at this point
Copy the full SHA 5517a9fView commit details -
status: disable and free fscache at the end of the status command
At the end of the status command, disable and free the fscache so that we don't leak the memory and so that we can dump the fscache statistics. Signed-off-by: Ben Peart <benpeart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 2a5d448 - Browse repository at this point
Copy the full SHA 2a5d448View commit details -
fscache: add GIT_TEST_FSCACHE support
Add support to fscache to enable running the entire test suite with the fscache enabled. Signed-off-by: Ben Peart <benpeart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for e931185 - Browse repository at this point
Copy the full SHA e931185View commit details -
mem_pool: add GIT_TRACE_MEMPOOL support
Add tracing around initializing and discarding mempools. In discard report on the amount of memory unused in the current block to help tune setting the initial_size. Signed-off-by: Ben Peart <benpeart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 3257dfe - Browse repository at this point
Copy the full SHA 3257dfeView commit details -
fscache: fscache takes an initial size
Update enable_fscache() to take an optional initial size parameter which is used to initialize the hashmap so that it can avoid having to rehash as additional entries are added. Add a separate disable_fscache() macro to make the code clearer and easier to read. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 9e1e228 - Browse repository at this point
Copy the full SHA 9e1e228View commit details -
fscache: update fscache to be thread specific instead of global
The threading model for fscache has been to have a single, global cache. This puts requirements on it to be thread safe so that callers like preload-index can call it from multiple threads. This was implemented with a single mutex and completion events which introduces contention between the calling threads. Simplify the threading model by making fscache thread specific. This allows us to remove the global mutex and synchronization events entirely and instead associate a fscache with every thread that requests one. This works well with the current multi-threading which divides the cache entries into blocks with a separate thread processing each block. At the end of each worker thread, if there is a fscache on the primary thread, merge the cached results from the worker into the primary thread cache. This enables us to reuse the cache later especially when scanning for untracked files. In testing, this reduced the time spent in preload_index() by about 25% and also reduced the CPU utilization significantly. On a repo with ~200K files, it reduced overall status times by ~12%. Signed-off-by: Ben Peart <benpeart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for daf7e83 - Browse repository at this point
Copy the full SHA daf7e83View commit details -
fscache: add fscache hit statistics
Track fscache hits and misses for lstat and opendir requests. Reporting of statistics is done when the cache is disabled for the last time and freed and is only reported if GIT_TRACE_FSCACHE is set. Sample output is: 11:33:11.836428 compat/win32/fscache.c:433 fscache: lstat 3775, opendir 263, total requests/misses 4052/269 Signed-off-by: Ben Peart <benpeart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 94dd824 - Browse repository at this point
Copy the full SHA 94dd824View commit details -
fscache: teach fscache to use mempool
Now that the fscache is single threaded, take advantage of the mem_pool as the allocator to significantly reduce the cost of allocations and frees. With the reduced cost of free, in future patches, we can start freeing the fscache at the end of commands instead of just leaking it. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for b8f8cdc - Browse repository at this point
Copy the full SHA b8f8cdcView commit details -
fscache: remember the reparse tag for each entry
We will use this in the next commit to implement an FSCache-aware version of is_mount_point(). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 8cf3d40 - Browse repository at this point
Copy the full SHA 8cf3d40View commit details -
fscache: make fscache_enable() thread safe
The recent change to make fscache thread specific relied on fscache_enable() being called first from the primary thread before being called in parallel from worker threads. Make that more robust and protect it with a critical section to avoid any issues. Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Ben Peart <benpeart@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for f34dae8 - Browse repository at this point
Copy the full SHA f34dae8View commit details -
fscache: teach fscache to use NtQueryDirectoryFile
Using FindFirstFileExW() requires the OS to allocate a 64K buffer for each directory and then free it when we call FindClose(). Update fscache to call the underlying kernel API NtQueryDirectoryFile so that we can do the buffer management ourselves. That allows us to allocate a single buffer for the lifetime of the cache and reuse it for each directory. This change improves performance of 'git status' by 18% in a repo with ~200K files and 30k folders. Documentation for NtQueryDirectoryFile can be found at: https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntifs/nf-ntifs-ntquerydirectoryfile https://docs.microsoft.com/en-us/windows/desktop/FileIO/file-attribute-constants https://docs.microsoft.com/en-us/windows/desktop/fileio/reparse-point-tags To determine if the specified directory is a symbolic link, inspect the FileAttributes member to see if the FILE_ATTRIBUTE_REPARSE_POINT flag is set. If so, EaSize will contain the reparse tag (this is a so far undocumented feature, but confirmed by the NTFS developers). To determine if the reparse point is a symbolic link (and not some other form of reparse point), test whether the tag value equals the value IO_REPARSE_TAG_SYMLINK. The NtQueryDirectoryFile() call works best (and on Windows 8.1 and earlier, it works *only*) with buffer sizes up to 64kB. Which is 32k wide characters, so let's use that as our buffer size. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 6330375 - Browse repository at this point
Copy the full SHA 6330375View commit details -
unpack-trees: enable fscache for sparse-checkout
When updating the skip-worktree bits in the index to align with new values in a sparse-checkout file, Git scans the entire working directory with lstat() calls. In a sparse-checkout, many of these lstat() calls are for paths that do not exist. Enable the fscache feature during this scan. Since enable_fscache() calls nest, the disable_fscache() method decrements a counter and would only clear the cache if that counter reaches zero. In a local test of a repo with ~2.2 million paths, updating the index with git read-tree -m -u HEAD with a sparse-checkout file containing only /.gitattributes improved from 2-3 minutes to ~6 seconds. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 3629591 - Browse repository at this point
Copy the full SHA 3629591View commit details -
git-gui: provide question helper for retry fallback on Windows
Make use of the new environment variable GIT_ASK_YESNO to support the recently implemented fallback in case unlink, rename or rmdir fail for files in use on Windows. The added dialog will present a yes/no question to the the user which will currently be used by the windows compat layer to let the user retry a failed file operation. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Configuration menu - View commit details
-
Copy full SHA for 8b37641 - Browse repository at this point
Copy the full SHA 8b37641View commit details -
fscache: implement an FSCache-aware is_mount_point()
When FSCache is active, we can cache the reparse tag and use it directly to determine whether a path refers to an NTFS junction, without any additional, costly I/O. Note: this change only makes a difference with the next commit, which will make use of the FSCache in `git clean` (contingent on `core.fscache` set, of course). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for d5c7454 - Browse repository at this point
Copy the full SHA d5c7454View commit details -
git gui: set GIT_ASKPASS=git-gui--askpass if not set yet
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 3158c66 - Browse repository at this point
Copy the full SHA 3158c66View commit details -
The `git clean` command needs to enumerate plenty of files and directories, and can therefore benefit from the FSCache. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 75ce329 - Browse repository at this point
Copy the full SHA 75ce329View commit details -
gitk: Unicode file name support
Assumes file names in git tree objects are UTF-8 encoded. On most unix systems, the system encoding (and thus the TCL system encoding) will be UTF-8, so file names will be displayed correctly. On Windows, it is impossible to set the system encoding to UTF-8. Changing the TCL system encoding (via 'encoding system ...', e.g. in the startup code) is explicitly discouraged by the TCL docs. Change gitk functions dealing with file names to always convert from and to UTF-8. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 76d79d9 - Browse repository at this point
Copy the full SHA 76d79d9View commit details -
git-gui--askyesno: fix funny text wrapping
The text wrapping seems to be aligned to the right side of the Yes button, leaving an awful lot of empty space. Let's try to counter this by using pixel units. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 4ab6eca - Browse repository at this point
Copy the full SHA 4ab6ecaView commit details -
gitk: Use an external icon file on Windows
Git for Windows now ships with the new Git icon from git-scm.com. Use that icon file if it exists instead of the old procedurally drawn one. This patch was sent upstream but so far no decision on its inclusion was made, so commit it to our fork. Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 0c82cc9 - Browse repository at this point
Copy the full SHA 0c82cc9View commit details -
git-gui--askyesno: allow overriding the window title
"Question?" is maybe not the most informative thing to ask. In the absence of better information, it is the best we can do, of course. However, Git for Windows' auto updater just learned the trick to use git-gui--askyesno to ask the user whether to update now or not. And in this scripted scenario, we can easily pass a command-line option to change the window title. So let's support that with the new `--title <title>` option. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for dde11f4 - Browse repository at this point
Copy the full SHA dde11f4View commit details -
gitk: fix arrow keys in input fields with Tcl/Tk >= 8.6
Tcl/Tk 8.6 introduced new events for the cursor left/right keys and apparently changed the behavior of the previous event. Let's work around that by using the new events when we are running with Tcl/Tk 8.6 or later. This fixes git-for-windows#495 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 28d40b9 - Browse repository at this point
Copy the full SHA 28d40b9View commit details -
git-gui--askyesno (mingw): use Git for Windows' icon, if available
For additional GUI goodness. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for e6058c8 - Browse repository at this point
Copy the full SHA e6058c8View commit details -
Configuration menu - View commit details
-
Copy full SHA for a99ea8c - Browse repository at this point
Copy the full SHA a99ea8cView commit details -
gitk: make the "list references" default window width wider
When using remotes (with git-flow especially), the remote reference names are almost always wordwrapped in the "list references" window because it's somewhat narrow by default. It's possible to resize it with a mouse, but it's annoying to have to do this every time, especially on Windows 10, where the window border seems to be only one (1) pixel wide, thus making the grabbing of the window border tricky. Signed-off-by: James J. Raden <james.raden@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 24be74a - Browse repository at this point
Copy the full SHA 24be74aView commit details -
Merge branch 'git-gui-askyesno'
These changes are necessary to support better Git for Windows' new auto-update feature. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for b0cdda0 - Browse repository at this point
Copy the full SHA b0cdda0View commit details -
Configuration menu - View commit details
-
Copy full SHA for f7d8568 - Browse repository at this point
Copy the full SHA f7d8568View commit details -
Configuration menu - View commit details
-
Copy full SHA for a96aa60 - Browse repository at this point
Copy the full SHA a96aa60View commit details -
Merge pull request git-for-windows#994 from jeffhostetler/jeffhostetl…
…er/fscache_nfd fscache: add not-found directory cache to fscache
Configuration menu - View commit details
-
Copy full SHA for fa1bafd - Browse repository at this point
Copy the full SHA fa1bafdView commit details -
Merge pull request git-for-windows#971 from jeffhostetler/jeffhostetl…
…er/add_preload_fscache add: use preload-index and fscache for performance
Configuration menu - View commit details
-
Copy full SHA for 95e4f9f - Browse repository at this point
Copy the full SHA 95e4f9fView commit details -
Merge pull request git-for-windows#1344 from jeffhostetler/perf_add_e…
…xcludes_with_fscache dir.c: make add_excludes aware of fscache during status
Configuration menu - View commit details
-
Copy full SHA for a70f212 - Browse repository at this point
Copy the full SHA a70f212View commit details -
Merge pull request git-for-windows#1426 from atetubou/fetch_pack
fetch-pack.c: enable fscache for stats under .git/objects
Configuration menu - View commit details
-
Copy full SHA for 6ca479f - Browse repository at this point
Copy the full SHA 6ca479fView commit details -
Merge pull request git-for-windows#1468 from atetubou/fscache_checkou…
…t_flush checkout.c: enable fscache for checkout again Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 8b5ce62 - Browse repository at this point
Copy the full SHA 8b5ce62View commit details -
Merge pull request git-for-windows#1827 from benpeart/fscache_refresh…
…_index Enable the filesystem cache (fscache) in refresh_index().
Configuration menu - View commit details
-
Copy full SHA for 61c18ed - Browse repository at this point
Copy the full SHA 61c18edView commit details -
Merge pull request git-for-windows#1908 from benpeart/FindFirstFileEx…
…-gfw fscache: use FindFirstFileExW to avoid retrieving the short name
Configuration menu - View commit details
-
Copy full SHA for de71e31 - Browse repository at this point
Copy the full SHA de71e31View commit details -
Merge pull request git-for-windows#1909 from benpeart/free-fscache-af…
…ter-status-gfw status: disable and free fscache at the end of the status command
Configuration menu - View commit details
-
Copy full SHA for 00f3380 - Browse repository at this point
Copy the full SHA 00f3380View commit details -
Merge pull request git-for-windows#1911 from benpeart/git_test_fscach…
…e-gfw fscache: add GIT_TEST_FSCACHE support
Configuration menu - View commit details
-
Copy full SHA for 3aac229 - Browse repository at this point
Copy the full SHA 3aac229View commit details -
Merge pull request git-for-windows#1910 from benpeart/fscache_statist…
…ics-gfw fscache: add fscache hit statistics
Configuration menu - View commit details
-
Copy full SHA for 466ba3b - Browse repository at this point
Copy the full SHA 466ba3bView commit details -
Merge remote-tracking branch 'benpeart/fscache-per-thread-gfw'
This brings substantial wins in performance because the FSCache is now per-thread, being merged to the primary thread only at the end, so we do not have to lock (except while merging). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 77b1db5 - Browse repository at this point
Copy the full SHA 77b1db5View commit details -
Merge pull request git-for-windows#1934 from benpeart/fscache-thread-…
…safe-enable-gfw fscache: make fscache_enable() thread safe
Configuration menu - View commit details
-
Copy full SHA for f2bd2da - Browse repository at this point
Copy the full SHA f2bd2daView commit details -
Merge pull request git-for-windows#1937 from benpeart/fscache-NtQuery…
…DirectoryFile-gfw fscache: teach fscache to use NtQueryDirectoryFile
Configuration menu - View commit details
-
Copy full SHA for 51cf6a9 - Browse repository at this point
Copy the full SHA 51cf6a9View commit details -
Merge branch 'fscache-and-sparse-checkout'
When updating the skip-worktree bits in the index to align with new values in a sparse-checkout file, Git scans the entire working directory with lstat() calls. In a sparse-checkout, many of these lstat() calls are for paths that do not exist. Enable the fscache feature during this scan. In a local test of a repo with ~2.2 million paths, updating the index with `git read-tree -m -u HEAD` with a sparse-checkout file containing only `/.gitattributes` improved from 2-3 minutes to 15-20 seconds. More work could be done to stop running lstat() calls when recursing into directories that are known to not exist.
Configuration menu - View commit details
-
Copy full SHA for b0add8c - Browse repository at this point
Copy the full SHA b0add8cView commit details -
pack-objects (mingw): demonstrate a segmentation fault with large deltas
There is a problem in the way 9ac3f0e (pack-objects: fix performance issues on packing large deltas, 2018-07-22) initializes that mutex in the `packing_data` struct. The problem manifests in a segmentation fault on Windows, when a mutex (AKA critical section) is accessed without being initialized. (With pthreads, you apparently do not really have to initialize them?) This was reported in git-for-windows#1839. Signed-off-by: Doug Kelly <dougk.ff7@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for bf28c7f - Browse repository at this point
Copy the full SHA bf28c7fView commit details -
Windows paths are typically limited to MAX_PATH = 260 characters, even though the underlying NTFS file system supports paths up to 32,767 chars. This limitation is also evident in Windows Explorer, cmd.exe and many other applications (including IDEs). Particularly annoying is that most Windows APIs return bogus error codes if a relative path only barely exceeds MAX_PATH in conjunction with the current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG. Many Windows wide char APIs support longer than MAX_PATH paths through the file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path. Notable exceptions include functions dealing with executables and the current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...). Introduce a handle_long_path function to check the length of a specified path properly (and fail with ENAMETOOLONG), and to optionally expand long paths using the '\\?\' file namespace prefix. Short paths will not be modified, so we don't need to worry about device names (NUL, CON, AUX). Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be limited to MAX_PATH (at least not on Win7), so we can use it to do the heavy lifting of the conversion (translate '/' to '\', eliminate '.' and '..', and make an absolute path). Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH limit. Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs that support long paths. While improved error checking is always active, long paths support must be explicitly enabled via 'core.longpaths' option. This is to prevent end users to shoot themselves in the foot by checking out files that Windows Explorer, cmd/bash or their favorite IDE cannot handle. Test suite: Test the case is when the full pathname length of a dir is close to 260 (MAX_PATH). Bug report and an original reproducer by Andrey Rogozhnikov: msysgit#122 (comment) [jes: adjusted test number to avoid conflicts, added support for chdir(), etc] Thanks-to: Martin W. Kirst <maki@bitkings.de> Thanks-to: Doug Kelly <dougk.ff7@gmail.com> Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com> Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 0b30bc0 - Browse repository at this point
Copy the full SHA 0b30bc0View commit details -
Merge branch 'dont-clean-junctions-fscache'
We already avoid traversing NTFS junction points in `git clean -dfx`. With this topic branch, we do that when the FSCache is enabled, too. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for e3535ad - Browse repository at this point
Copy the full SHA e3535adView commit details -
Win32: fix 'lstat("dir/")' with long paths
Use a suffciently large buffer to strip the trailing slash. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for 0a9dc09 - Browse repository at this point
Copy the full SHA 0a9dc09View commit details -
win32(long path support): leave drive-less absolute paths intact
When trying to ensure that long paths are handled correctly, we first normalize absolute paths as we encounter them. However, if the path is a so-called "drive-less" absolute path, i.e. if it is relative to the current drive but _does_ start with a directory separator, we would want the normalized path to be such a drive-less absolute path, too. Let's do that, being careful to still include the drive prefix when we need to go through the `\\?\` dance (because there, the drive prefix is absolutely required). This fixes git-for-windows#4586. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 5ff1ebe - Browse repository at this point
Copy the full SHA 5ff1ebeView commit details -
compat/fsmonitor/fsm-*-win32: support long paths
Update wchar_t buffers to use MAX_LONG_PATH instead of MAX_PATH and call xutftowcs_long_path() in the Win32 backend source files. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for e4527a8 - Browse repository at this point
Copy the full SHA e4527a8View commit details -
mingw: Support
git_terminal_prompt
with more terminalsThe `git_terminal_prompt()` function expects the terminal window to be attached to a Win32 Console. However, this is not the case with terminal windows other than `cmd.exe`'s, e.g. with MSys2's own `mintty`. Non-cmd terminals such as `mintty` still have to have a Win32 Console to be proper console programs, but have to hide the Win32 Console to be able to provide more flexibility (such as being resizeable not only vertically but also horizontally). By writing to that Win32 Console, `git_terminal_prompt()` manages only to send the prompt to nowhere and to wait for input from a Console to which the user has no access. This commit introduces a function specifically to support `mintty` -- or other terminals that are compatible with MSys2's `/dev/tty` emulation. We use the `TERM` environment variable as an indicator for that: if the value starts with "xterm" (such as `mintty`'s "xterm_256color"), we prefer to let `xterm_prompt()` handle the user interaction. The most prominent user of `git_terminal_prompt()` is certainly `git-remote-https.exe`. It is an interesting use case because both `stdin` and `stdout` are redirected when Git calls said executable, yet it still wants to access the terminal. When running inside a `mintty`, the terminal is not accessible to the `git-remote-https.exe` program, though, because it is a MinGW program and the `mintty` terminal is not backed by a Win32 console. To solve that problem, we simply call out to the shell -- which is an *MSys2* program and can therefore access `/dev/tty`. Helped-by: nalla <nalla@hamal.uberspace.de> Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 67337c8 - Browse repository at this point
Copy the full SHA 67337c8View commit details -
compat/terminal.c: only use the Windows console if bash 'read -r' fails
Accessing the Windows console through the special CONIN$ / CONOUT$ devices doesn't work properly for non-ASCII usernames an passwords. It also doesn't work for terminal emulators that hide the native console window (such as mintty), and 'TERM=xterm*' is not necessarily a reliable indicator for such terminals. The new shell_prompt() function, on the other hand, works fine for both MSys1 and MSys2, in native console windows as well as mintty, and properly supports Unicode. It just needs bash on the path (for 'read -s', which is bash-specific). On Windows, try to use the shell to read from the terminal. If that fails with ENOENT (i.e. bash was not found), use CONIN/OUT as fallback. Note: To test this, create a UTF-8 credential file with non-ASCII chars, e.g. in git-bash: 'echo url=http://täst.com > cred.txt'. Then in git-cmd, 'git credential fill <cred.txt' works (shell version), while calling git without the git-wrapper (i.e. 'mingw64\bin\git credential fill <cred.txt') mangles non-ASCII chars in both console output and input. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for b5dc26a - Browse repository at this point
Copy the full SHA b5dc26aView commit details -
strbuf_readlink: don't call readlink twice if hint is the exact link …
…size strbuf_readlink() calls readlink() twice if the hint argument specifies the exact size of the link target (e.g. by passing stat.st_size as returned by lstat()). This is necessary because 'readlink(..., hint) == hint' could mean that the buffer was too small. Use hint + 1 as buffer size to prevent this. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for 2c87e7b - Browse repository at this point
Copy the full SHA 2c87e7bView commit details -
clean: suggest using
core.longPaths
if paths are too long to removeOn Windows, git repositories may have extra files which need cleaned (e.g., a build directory) that may be arbitrarily deep. Suggest using `core.longPaths` if such situations are encountered. Fixes: git-for-windows#2715 Signed-off-by: Ben Boeckel <mathstuf@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for a36b695 - Browse repository at this point
Copy the full SHA a36b695View commit details -
mingw (git_terminal_prompt): do fall back to CONIN$/CONOUT$ method
To support Git Bash running in a MinTTY, we use a dirty trick to access the MSYS2 pseudo terminal: we execute a Bash snippet that accesses /dev/tty. The idea was to fall back to writing to/reading from CONOUT$/CONIN$ if that Bash call failed because Bash was not found. However, we should fall back even in other error conditions, because we have not successfully read the user input. Let's make it so. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 1f391dd - Browse repository at this point
Copy the full SHA 1f391ddView commit details -
strbuf_readlink: support link targets that exceed PATH_MAX
strbuf_readlink() refuses to read link targets that exceed PATH_MAX (even if a sufficient size was specified by the caller). As some platforms support longer paths, remove this restriction (similar to strbuf_getcwd()). Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for 5bea09f - Browse repository at this point
Copy the full SHA 5bea09fView commit details -
lockfile.c: use is_dir_sep() instead of hardcoded '/' checks
Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for 512490f - Browse repository at this point
Copy the full SHA 512490fView commit details -
Win32: don't call GetFileAttributes twice in mingw_lstat()
GetFileAttributes cannot handle paths with trailing dir separator. The current [l]stat implementation calls GetFileAttributes twice if the path has trailing slashes (first with the original path passed to [l]stat, and and a second time with a path copy with trailing '/' removed). With Unicode conversion, we get the length of the path for free and also have a (wide char) buffer that can be modified. Remove trailing directory separators before calling the Win32 API. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for bd1d088 - Browse repository at this point
Copy the full SHA bd1d088View commit details -
Win32: implement stat() with symlink support
With respect to symlinks, the current stat() implementation is almost the same as lstat(): except for the file type (st_mode & S_IFMT), it returns information about the link rather than the target. Implement stat by opening the file with as little permissions as possible and calling GetFileInformationByHandle on it. This way, all link resoltion is handled by the Windows file system layer. If symlinks are disabled, use lstat() as before, but fail with ELOOP if a symlink would have to be resolved. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 6d21967 - Browse repository at this point
Copy the full SHA 6d21967View commit details -
Win32: remove separate do_lstat() function
With the new mingw_stat() implementation, do_lstat() is only called from mingw_lstat() (with follow == 0). Remove the extra function and the old mingw_stat()-specific (follow == 1) logic. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for a32c452 - Browse repository at this point
Copy the full SHA a32c452View commit details -
Win32: let mingw_lstat() error early upon problems with reparse points
When obtaining lstat information for reparse points, we need to call FindFirstFile() in addition to GetFileInformationEx() to obtain the type of the reparse point (symlink, mount point etc.). However, currently there is no error handling whatsoever if FindFirstFile() fails. Call FindFirstFile() before modifying the stat *buf output parameter and error out if the call fails. Note: The FindFirstFile() return value includes all the data that we get from GetFileAttributesEx(), so we could replace GetFileAttributesEx() with FindFirstFile(). We don't do that because GetFileAttributesEx() is about twice as fast for single files. I.e. we only pay the extra cost of calling FindFirstFile() in the rare case that we encounter a reparse point. Note: The indentation of the remaining reparse point code will be fixed in the next patch. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for 04ac44d - Browse repository at this point
Copy the full SHA 04ac44dView commit details -
mingw: teach fscache and dirent about symlinks
Move S_IFLNK detection to file_attr_to_st_mode() and reuse it in fscache. Implement DT_LNK detection in dirent.c and the fscache readdir version. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 797b60f - Browse repository at this point
Copy the full SHA 797b60fView commit details -
Win32: lstat(): return adequate stat.st_size for symlinks
Git typically doesn't trust the stat.st_size member of symlinks (e.g. see strbuf_readlink()). However, some functions take shortcuts if st_size is 0 (e.g. diff_populate_filespec()). In mingw_lstat() and fscache_lstat(), make sure to return an adequate size. The extra overhead of opening and reading the reparse point to calculate the exact size is not necessary, as git doesn't rely on the value anyway. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 147a648 - Browse repository at this point
Copy the full SHA 147a648View commit details -
The retry pattern is duplicated in three places. It also seems to be too hard to use: mingw_unlink() and mingw_rmdir() duplicate the code to retry, and both of them do so incompletely. They also do not restore errno if the user answers 'no'. Introduce a retry_ask_yes_no() helper function that handles retry with small delay, asking the user, and restoring errno. mingw_unlink: include _wchmod in the retry loop (which may fail if the file is locked exclusively). mingw_rmdir: include special error handling in the retry loop. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for 3d9ebe3 - Browse repository at this point
Copy the full SHA 3d9ebe3View commit details -
Win32: change default of 'core.symlinks' to false
Symlinks on Windows don't work the same way as on Unix systems. E.g. there are different types of symlinks for directories and files, creating symlinks requires administrative privileges etc. By default, disable symlink support on Windows. I.e. users explicitly have to enable it with 'git config [--system|--global] core.symlinks true'. The test suite ignores system / global config files. Allow testing *with* symlink support by checking if native symlinks are enabled in MSys2 (via 'MSYS=winsymlinks:nativestrict'). Reminder: This would need to be changed if / when we find a way to run the test suite in a non-MSys-based shell (e.g. dash). Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for 65c4e23 - Browse repository at this point
Copy the full SHA 65c4e23View commit details -
Win32: add symlink-specific error codes
Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for d8c0893 - Browse repository at this point
Copy the full SHA d8c0893View commit details -
Win32: mingw_unlink: support symlinks to directories
_wunlink() / DeleteFileW() refuses to delete symlinks to directories. If _wunlink() fails with ERROR_ACCESS_DENIED, try _wrmdir() as well. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for 3a56953 - Browse repository at this point
Copy the full SHA 3a56953View commit details -
Win32: mingw_rename: support renaming symlinks
MSVCRT's _wrename() cannot rename symlinks over existing files: it returns success without doing anything. Newer MSVCR*.dll versions probably do not have this problem: according to CRT sources, they just call MoveFileEx() with the MOVEFILE_COPY_ALLOWED flag. Get rid of _wrename() and call MoveFileEx() with proper error handling. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for d27bed2 - Browse repository at this point
Copy the full SHA d27bed2View commit details -
Win32: mingw_chdir: change to symlink-resolved directory
If symlinks are enabled, resolve all symlinks when changing directories, as required by POSIX. Note: Git's real_path() function bases its link resolution algorithm on this property of chdir(). Unfortunately, the current directory on Windows is limited to only MAX_PATH (260) characters. Therefore using symlinks and long paths in combination may be problematic. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for f99b857 - Browse repository at this point
Copy the full SHA f99b857View commit details -
Implement readlink() by reading NTFS reparse points. Works for symlinks and directory junctions. If symlinks are disabled, fail with ENOSYS. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for 86420a1 - Browse repository at this point
Copy the full SHA 86420a1View commit details -
mingw: lstat: compute correct size for symlinks
This commit fixes mingw_lstat by computing the proper size for symlinks according to POSIX. POSIX specifies that upon successful return from lstat: "the value of the st_size member shall be set to the length of the pathname contained in the symbolic link not including any terminating null byte". Prior to this commit the mingw_lstat function returned a fixed size of 4096. This caused problems in git repositories that were accessed by git for Cygwin or git for WSL. For example, doing `git reset --hard` using git for Windows would update the size of symlinks in the index to be 4096; at a later time git for Cygwin or git for WSL would find that symlinks have changed size during `git status`. Vice versa doing `git reset --hard` in git for Cygwin or git for WSL would update the size of symlinks in the index with the correct value, only for git for Windows to find incorrectly at a later time that the size had changed. Signed-off-by: Bill Zissimopoulos <billziss@navimatics.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for dc59fce - Browse repository at this point
Copy the full SHA dc59fceView commit details -
Win32: implement basic symlink() functionality (file symlinks only)
Implement symlink() that always creates file symlinks. Fails with ENOSYS if symlinks are disabled or unsupported. Note: CreateSymbolicLinkW() was introduced with symlink support in Windows Vista. For compatibility with Windows XP, we need to load it dynamically and fail gracefully if it isnt's available. Signed-off-by: Karsten Blees <blees@dcon.de>
Configuration menu - View commit details
-
Copy full SHA for a7ad2f4 - Browse repository at this point
Copy the full SHA a7ad2f4View commit details -
Win32: symlink: add support for symlinks to directories
Symlinks on Windows have a flag that indicates whether the target is a file or a directory. Symlinks of wrong type simply don't work. This even affects core Win32 APIs (e.g. DeleteFile() refuses to delete directory symlinks). However, CreateFile() with FILE_FLAG_BACKUP_SEMANTICS doesn't seem to care. Check the target type by first creating a tentative file symlink, opening it, and checking the type of the resulting handle. If it is a directory, recreate the symlink with the directory flag set. It is possible to create symlinks before the target exists (or in case of symlinks to symlinks: before the target type is known). If this happens, create a tentative file symlink and postpone the directory decision: keep a list of phantom symlinks to be processed whenever a new directory is created in mingw_mkdir(). Limitations: This algorithm may fail if a link target changes from file to directory or vice versa, or if the target directory is created in another process. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 73e5a1e - Browse repository at this point
Copy the full SHA 73e5a1eView commit details -
mingw: try to create symlinks without elevated permissions
With Windows 10 Build 14972 in Developer Mode, a new flag is supported by CreateSymbolicLink() to create symbolic links even when running outside of an elevated session (which was previously required). This new flag is called SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE and has the numeric value 0x02. Previous Windows 10 versions will not understand that flag and return an ERROR_INVALID_PARAMETER, therefore we have to be careful to try passing that flag only when the build number indicates that it is supported. For more information about the new flag, see this blog post: https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/ This patch is loosely based on the patch submitted by Samuel D. Leslie as git-for-windows#1184. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 248e7b4 - Browse repository at this point
Copy the full SHA 248e7b4View commit details -
mingw: introduce code to detect whether we're inside a Windows container
This will come in handy in the next commit. Signed-off-by: JiSeop Moon <zcube@zcube.kr> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 4d79b76 - Browse repository at this point
Copy the full SHA 4d79b76View commit details -
mingw: when running in a Windows container, try to rename() harder
It is a known issue that a rename() can fail with an "Access denied" error at times, when copying followed by deleting the original file works. Let's just fall back to that behavior. Signed-off-by: JiSeop Moon <zcube@zcube.kr> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for f3ef296 - Browse repository at this point
Copy the full SHA f3ef296View commit details -
mingw: emulate stat() a little more faithfully
When creating directories via `safe_create_leading_directories()`, we might encounter an already-existing directory which is not readable by the current user. To handle that situation, Git's code calls `stat()` to determine whether we're looking at a directory. In such a case, `CreateFile()` will fail, though, no matter what, and consequently `mingw_stat()` will fail, too. But POSIX semantics seem to still allow `stat()` to go forward. So let's call `mingw_lstat()` for the rescue if we fail to get a file handle due to denied permission in `mingw_stat()`, and fill the stat info that way. We need to be careful to not allow this to go forward in case that we're looking at a symbolic link: to resolve the link, we would still have to create a file handle, and we just found out that we cannot. Therefore, `stat()` still needs to fail with `EACCES` in that case. This fixes git-for-windows#2531. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for ebe8730 - Browse repository at this point
Copy the full SHA ebe8730View commit details -
mingw: move the file_attr_to_st_mode() function definition
In preparation for making this function a bit more complicated (to allow for special-casing the `ContainerMappedDirectories` in Windows containers, which look like a symbolic link, but are not), let's move it out of the header. Signed-off-by: JiSeop Moon <zcube@zcube.kr> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7d1cc97 - Browse repository at this point
Copy the full SHA 7d1cc97View commit details -
mingw: special-case index entries for symlinks with buggy size
In git-for-windows#2637, we fixed a bug where symbolic links' target path sizes were recorded incorrectly in the index. The downside of this fix was that every user with tracked symbolic links in their checkouts would see them as modified in `git status`, but not in `git diff`, and only a `git add <path>` (or `git add -u`) would "fix" this. Let's do better than that: we can detect that situation and simply pretend that a symbolic link with a known bad size (or a size that just happens to be that bad size, a _very_ unlikely scenario because it would overflow our buffers due to the trailing NUL byte) means that it needs to be re-checked as if we had just checked it out. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 132c94f - Browse repository at this point
Copy the full SHA 132c94fView commit details -
mingw: Windows Docker volumes are *not* symbolic links
... even if they may look like them. As looking up the target of the "symbolic link" (just to see whether it starts with `/ContainerMappedDirectories/`) is pretty expensive, we do it when we can be *really* sure that there is a possibility that this might be the case. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: JiSeop Moon <zcube@zcube.kr>
Configuration menu - View commit details
-
Copy full SHA for 53cb5c1 - Browse repository at this point
Copy the full SHA 53cb5c1View commit details -
Win32: symlink: move phantom symlink creation to a separate function
Signed-off-by: Bert Belder <bertbelder@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for ec29d2b - Browse repository at this point
Copy the full SHA ec29d2bView commit details -
Introduce helper to create symlinks that knows about index_state
On Windows, symbolic links actually have a type depending on the target: it can be a file or a directory. In certain circumstances, this poses problems, e.g. when a symbolic link is supposed to point into a submodule that is not checked out, so there is no way for Git to auto-detect the type. To help with that, we will add support over the course of the next commits to specify that symlink type via the Git attributes. This requires an index_state, though, something that Git for Windows' `symlink()` replacement cannot know about because the function signature is defined by the POSIX standard and not ours to change. So let's introduce a helper function to create symbolic links that *does* know about the index_state. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for c088807 - Browse repository at this point
Copy the full SHA c088807View commit details -
mingw: allow to specify the symlink type in .gitattributes
On Windows, symbolic links have a type: a "file symlink" must point at a file, and a "directory symlink" must point at a directory. If the type of symlink does not match its target, it doesn't work. Git does not record the type of symlink in the index or in a tree. On checkout it'll guess the type, which only works if the target exists at the time the symlink is created. This may often not be the case, for example when the link points at a directory inside a submodule. By specifying `symlink=file` or `symlink=dir` the user can specify what type of symlink Git should create, so Git doesn't have to rely on unreliable heuristics. Signed-off-by: Bert Belder <bertbelder@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 5a30497 - Browse repository at this point
Copy the full SHA 5a30497View commit details -
mingw: work around rename() failing on a read-only file
At least on _some_ APFS network shares, Git fails to rename the object files because they are marked as read-only, because that has the effect of setting the uchg flag on APFS, which then means the file can't be renamed or deleted. To work around that, when a rename failed, and the read-only flag is set, try to turn it off and on again. This fixes git-for-windows#4482 Signed-off-by: David Lomas <dl3@pale-eds.co.uk> Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for fc96572 - Browse repository at this point
Copy the full SHA fc96572View commit details -
Win32: symlink: add test for
symlink
attributeTo verify that the symlink is resolved correctly, we use the fact that `git.exe` is a native Win32 program, and that `git.exe config -f <path>` therefore uses the native symlink resolution. Signed-off-by: Bert Belder <bertbelder@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for a9ed70c - Browse repository at this point
Copy the full SHA a9ed70cView commit details -
mingw: explicitly specify with which cmd to prefix the cmdline
The main idea of this patch is that even if we have to look up the absolute path of the script, if only the basename was specified as argv[0], then we should use that basename on the command line, too, not the absolute path. This patch will also help with the upcoming patch where we automatically substitute "sh ..." by "busybox sh ..." if "sh" is not in the PATH but "busybox" is: we will do that by substituting the actual executable, but still keep prepending "sh" to the command line. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 236b7b7 - Browse repository at this point
Copy the full SHA 236b7b7View commit details -
mingw: when path_lookup() failed, try BusyBox
BusyBox comes with a ton of applets ("applet" being the identical concept to Git's "builtins"). And similar to Git's builtins, the applets can be called via `busybox <command>`, or the BusyBox executable can be copied/hard-linked to the command name. The similarities do not end here. Just as with Git's builtins, it is problematic that BusyBox' hard-linked applets cannot easily be put into a .zip file: .zip archives have no concept of hard-links and therefore would store identical copies (and also extract identical copies, "inflating" the archive unnecessarily). To counteract that issue, MinGit already ships without hard-linked copies of the builtins, and the plan is to do the same with BusyBox' applets: simply ship busybox.exe as single executable, without hard-linked applets. To accommodate that, Git is being taught by this commit a very special trick, exploiting the fact that it is possible to call an executable with a command-line whose argv[0] is different from the executable's name: when `sh` is to be spawned, and no `sh` is found in the PATH, but busybox.exe is, use that executable (with unchanged argv). Likewise, if any executable to be spawned is not on the PATH, but busybox.exe is found, parse the output of `busybox.exe --help` to find out what applets are included, and if the command matches an included applet name, use busybox.exe to execute it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 9fd2e84 - Browse repository at this point
Copy the full SHA 9fd2e84View commit details -
test-lib: avoid unnecessary Perl invocation
It is a bit strange, and even undesirable, to require Perl just to run the test suite even when NO_PERL was set. This patch does not fix this problem by any stretch of imagination. However, it fixes *the* Perl invocation that *every single* test script has to run. While at it, it makes the source code also more grep'able, as the code that unsets some, but not all, GIT_* environment variables just became a *lot* more explicit. And all that while still reducing the total number of lines. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for cf2a479 - Browse repository at this point
Copy the full SHA cf2a479View commit details -
test-tool: learn to act as a drop-in replacement for
iconv
It is convenient to assume that everybody who wants to build & test Git has access to a working `iconv` executable (after all, we already pretty much require libiconv). However, that limits esoteric test scenarios such as Git for Windows', where an end user installation has to ship with `iconv` for the sole purpose of being testable. That payload serves no other purpose. So let's just have a test helper (to be able to test Git, the test helpers have to be available, after all) to act as `iconv` replacement. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 00139f9 - Browse repository at this point
Copy the full SHA 00139f9View commit details -
tests(mingw): if
iconv
is unavailable, usetest-helper --iconv
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7774624 - Browse repository at this point
Copy the full SHA 7774624View commit details -
gitattributes: mark .png files as binary
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 023f17b - Browse repository at this point
Copy the full SHA 023f17bView commit details -
tests: move test PNGs into t/lib-diff/
We already have a directory where we store files intended for use by multiple test scripts. The same directory is a better home for the test-binary-*.png files than t/. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 9ab6b6b - Browse repository at this point
Copy the full SHA 9ab6b6bView commit details -
tests: only override sort & find if there are usable ones in /usr/bin/
The idea is to allow running the test suite on MinGit with BusyBox installed in /mingw64/bin/sh.exe. In that case, we will want to exclude sort & find (and other Unix utilities) from being bundled. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 95f6db9 - Browse repository at this point
Copy the full SHA 95f6db9View commit details -
tests: use the correct path separator with BusyBox
BusyBox-w32 is a true Win32 application, i.e. it does not come with a POSIX emulation layer. That also means that it does *not* use the Unix convention of separating the entries in the PATH variable using colons, but semicolons. However, there are also BusyBox ports to Windows which use a POSIX emulation layer such as Cygwin's or MSYS2's runtime, i.e. using colons as PATH separators. As a tell-tale, let's use the presence of semicolons in the PATH variable: on Unix, it is highly unlikely that it contains semicolons, and on Windows (without POSIX emulation), it is virtually guaranteed, as everybody should have both $SYSTEMROOT and $SYSTEMROOT/system32 in their PATH. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 1aab1ad - Browse repository at this point
Copy the full SHA 1aab1adView commit details -
mingw: only use Bash-ism
builtin pwd -W
when availableTraditionally, Git for Windows' SDK uses Bash as its default shell. However, other Unix shells are available, too. Most notably, the Win32 port of BusyBox comes with `ash` whose `pwd` command already prints Windows paths as Git for Windows wants them, while there is not even a `builtin` command. Therefore, let's be careful not to override `pwd` unless we know that the `builtin` command is available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for e033676 - Browse repository at this point
Copy the full SHA e033676View commit details -
tests (mingw): remove Bash-specific pwd option
The -W option is only understood by MSYS2 Bash's pwd command. We already make sure to override `pwd` by `builtin pwd -W` for MINGW, so let's not double the effort here. This will also help when switching the shell to another one (such as BusyBox' ash) whose pwd does *not* understand the -W option. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 8e3b66b - Browse repository at this point
Copy the full SHA 8e3b66bView commit details -
test-lib: add BUSYBOX prerequisite
When running with BusyBox, we will want to avoid calling executables on the PATH that are implemented in BusyBox itself. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7ae8372 - Browse repository at this point
Copy the full SHA 7ae8372View commit details -
t5003: use binary file from t/lib-diff/
At some stage, t5003-archive-zip wants to add a file that is not ASCII. To that end, it uses /bin/sh. But that file may actually not exist (it is too easy to forget that not all the world is Unix/Linux...)! Besides, we already have perfectly fine binary files intended for use solely by the tests. So let's use one of them instead. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for bb1b278 - Browse repository at this point
Copy the full SHA bb1b278View commit details -
t5532: workaround for BusyBox on Windows
While it may seem super convenient to some old Unix hands to simpy require Perl to be available when running the test suite, this is a major hassle on Windows, where we want to verify that Perl is not, actually, required in a NO_PERL build. As a super ugly workaround, we "install" a script into /usr/bin/perl reading like this: #!/bin/sh # We'd much rather avoid requiring Perl altogether when testing # an installed Git. Oh well, that's why we cannot have nice # things. exec c:/git-sdk-64/usr/bin/perl.exe "$@" The problem with that is that BusyBox assumes that the #! line in a script refers to an executable, not to a script. So when it encounters the line #!/usr/bin/perl in t5532's proxy-get-cmd, it barfs. Let's help this situation by simply executing the Perl script with the "interpreter" specified explicitly. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for ffe9503 - Browse repository at this point
Copy the full SHA ffe9503View commit details -
t5605: special-case hardlink test for BusyBox-w32
When t5605 tries to verify that files are hardlinked (or that they are not), it uses the `-links` option of the `find` utility. BusyBox' implementation does not support that option, and BusyBox-w32's lstat() does not even report the number of hard links correctly (for performance reasons). So let's just switch to a different method that actually works on Windows. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for dbb27bd - Browse repository at this point
Copy the full SHA dbb27bdView commit details -
t5813: allow for $PWD to be a Windows path
Git for Windows uses MSYS2's Bash to run the test suite, which comes with benefits but also at a heavy price: on the plus side, MSYS2's POSIX emulation layer allows us to continue pretending that we are on a Unix system, e.g. use Unix paths instead of Windows ones, yet this is bought at a rather noticeable performance penalty. There *are* some more native ports of Unix shells out there, though, most notably BusyBox-w32's ash. These native ports do not use any POSIX emulation layer (or at most a *very* thin one, choosing to avoid features such as fork() that are expensive to emulate on Windows), and they use native Windows paths (usually with forward slashes instead of backslashes, which is perfectly legal in almost all use cases). And here comes the problem: with a $PWD looking like, say, C:/git-sdk-64/usr/src/git/t/trash directory.t5813-proto-disable-ssh Git's test scripts get quite a bit confused, as their assumptions have been shattered. Not only does this path contain a colon (oh no!), it also does not start with a slash. This is a problem e.g. when constructing a URL as t5813 does it: ssh://remote$PWD. Not only is it impossible to separate the "host" from the path with a $PWD as above, even prefixing $PWD by a slash won't work, as /C:/git-sdk-64/... is not a valid path. As a workaround, detect when $PWD does not start with a slash on Windows, and simply strip the drive prefix, using an obscure feature of Windows paths: if an absolute Windows path starts with a slash, it is implicitly prefixed by the drive prefix of the current directory. As we are talking about the current directory here, anyway, that strategy works. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 5b36798 - Browse repository at this point
Copy the full SHA 5b36798View commit details -
t9200: skip tests when $PWD contains a colon
On Windows, the current working directory is pretty much guaranteed to contain a colon. If we feed that path to CVS, it mistakes it for a separator between host and port, though. This has not been a problem so far because Git for Windows uses MSYS2's Bash using a POSIX emulation layer that also pretends that the current directory is a Unix path (at least as long as we're in a shell script). However, that is rather limiting, as Git for Windows also explores other ports of other Unix shells. One of those is BusyBox-w32's ash, which is a native port (i.e. *not* using any POSIX emulation layer, and certainly not emulating Unix paths). So let's just detect if there is a colon in $PWD and punt in that case. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 918f4ab - Browse repository at this point
Copy the full SHA 918f4abView commit details -
mingw: add a Makefile target to copy test artifacts
The Makefile target `install-mingit-test-artifacts` simply copies stuff and things directly into a MinGit directory, including an init.bat script to set everything up so that the tests can be run in a cmd window. Sadly, Git's test suite still relies on a Perl interpreter even if compiled with NO_PERL=YesPlease. We punt for now, installing a small script into /usr/bin/perl that hands off to an existing Perl of a Git for Windows SDK. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for b99fde4 - Browse repository at this point
Copy the full SHA b99fde4View commit details -
mingw: optionally enable wsl compability file mode bits
The Windows Subsystem for Linux (WSL) version 2 allows to use `chmod` on NTFS volumes provided that they are mounted with metadata enabled (see https://devblogs.microsoft.com/commandline/chmod-chown-wsl-improvements/ for details), for example: $ chmod 0755 /mnt/d/test/a.sh In order to facilitate better collaboration between the Windows version of Git and the WSL version of Git, we can make the Windows version of Git also support reading and writing NTFS file modes in a manner compatible with WSL. Since this slightly slows down operations where lots of files are created (such as an initial checkout), this feature is only enabled when `core.WSLCompat` is set to true. Note that you also have to set `core.fileMode=true` in repositories that have been initialized without enabling WSL compatibility. There are several ways to enable metadata loading for NTFS volumes in WSL, one of which is to modify `/etc/wsl.conf` by adding: ``` [automount] enabled = true options = "metadata,umask=027,fmask=117" ``` And reboot WSL. It can also be enabled temporarily by this incantation: $ sudo umount /mnt/c && sudo mount -t drvfs C: /mnt/c -o metadata,uid=1000,gid=1000,umask=22,fmask=111 It's important to note that this modification is compatible with, but does not depend on WSL. The helper functions in this commit can operate independently and functions normally on devices where WSL is not installed or properly configured. Signed-off-by: xungeng li <xungeng@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 0f35e77 - Browse repository at this point
Copy the full SHA 0f35e77View commit details -
mingw: kill child processes in a gentler way
The TerminateProcess() function does not actually leave the child processes any chance to perform any cleanup operations. This is bad insofar as Git itself expects its signal handlers to run. A symptom is e.g. a left-behind .lock file that would not be left behind if the same operation was run, say, on Linux. To remedy this situation, we use an obscure trick: we inject a thread into the process that needs to be killed and to let that thread run the ExitProcess() function with the desired exit status. Thanks J Wyman for describing this trick. The advantage is that the ExitProcess() function lets the atexit handlers run. While this is still different from what Git expects (i.e. running a signal handler), in practice Git sets up signal handlers and atexit handlers that call the same code to clean up after itself. In case that the gentle method to terminate the process failed, we still fall back to calling TerminateProcess(), but in that case we now also make sure that processes spawned by the spawned process are terminated; TerminateProcess() does not give the spawned process a chance to do so itself. Please note that this change only affects how Git for Windows tries to terminate processes spawned by Git's own executables. Third-party software that *calls* Git and wants to terminate it *still* need to make sure to imitate this gentle method, otherwise this patch will not have any effect. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 10bb07f - Browse repository at this point
Copy the full SHA 10bb07fView commit details -
Previously, we did not install any handler for Ctrl+C, but now we really want to because the MSYS2 runtime learned the trick to call the ConsoleCtrlHandler when Ctrl+C was pressed. With this, hitting Ctrl+C while `git log` is running will only terminate the Git process, but not the pager. This finally matches the behavior on Linux and on macOS. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 26539c4 - Browse repository at this point
Copy the full SHA 26539c4View commit details -
mingw: do not call xutftowcs_path in mingw_mktemp
The `xutftowcs_path` function canonicalizes absolute paths using GetFullPathNameW. This canonicalization may change the length of the string (e.g. getting rid of \.\), which breaks callers that pass the template string in a strbuf and expect the length of the string to remain the same. In my particular case, the tmp-objdir code is passing a strbuf to mkdtemp and is breaking since the strbuf.len is no longer synchronized with strlen(strbuf.buf). Signed-off-by: Neeraj K. Singh <neerajsi@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 63cc420 - Browse repository at this point
Copy the full SHA 63cc420View commit details -
Partially un-revert "editor: save and reset terminal after calling ED…
…ITOR" In e3f7e01 (Revert "editor: save and reset terminal after calling EDITOR", 2021-11-22), we reverted the commit wholesale where the terminal state would be saved and restored before/after calling an editor. The reverted commit was intended to fix a problem with Windows Terminal where simply calling `vi` would cause problems afterwards. To fix the problem addressed by the revert, but _still_ keep the problem with Windows Terminal fixed, let's revert the revert, with a twist: we restrict the save/restore _specifically_ to the case where `vi` (or `vim`) is called, and do not do the same for any other editor. This should still catch the majority of the cases, and will bridge the time until the original patch is re-done in a way that addresses all concerns. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 8b8b123 - Browse repository at this point
Copy the full SHA 8b8b123View commit details -
Add a GitHub workflow to monitor component updates
Rather than using private IFTTT Applets that send mails to this maintainer whenever a new version of a Git for Windows component was released, let's use the power of GitHub workflows to make this process publicly visible. This workflow monitors the Atom/RSS feeds, and opens a ticket whenever a new version was released. Note: Bash sometimes releases multiple patched versions within a few minutes of each other (i.e. 5.1p1 through 5.1p4, 5.0p15 and 5.0p16). The MSYS2 runtime also has a similar system. We can address those patches as a group, so we shouldn't get multiple issues about them. Note further: We're not acting on newlib releases, OpenSSL alphas, Perl release candidates or non-stable Perl releases. There's no need to open issues about them. Co-authored-by: Matthias Aßhauer <mha1993@live.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 2ce548c - Browse repository at this point
Copy the full SHA 2ce548cView commit details -
reset: reinstate support for the deprecated --stdin option
The `--stdin` option was a well-established paradigm in other commands, therefore we implemented it in `git reset` for use by Visual Studio. Unfortunately, upstream Git decided that it is time to introduce `--pathspec-from-file` instead. To keep backwards-compatibility for some grace period, we therefore reinstate the `--stdin` option on top of the `--pathspec-from-file` option, but mark it firmly as deprecated. Helped-by: Victoria Dye <vdye@github.com> Helped-by: Matthew John Cheetham <mjcheetham@outlook.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for c3e1525 - Browse repository at this point
Copy the full SHA c3e1525View commit details -
fsmonitor: reintroduce core.useBuiltinFSMonitor
Reintroduce the 'core.useBuiltinFSMonitor' config setting (originally added in 0a756b2 (fsmonitor: config settings are repository-specific, 2021-03-05)) after its removal from the upstream version of FSMonitor. Upstream, the 'core.useBuiltinFSMonitor' setting was rendered obsolete by "overloading" the 'core.fsmonitor' setting to take a boolean value. However, several applications (e.g., 'scalar') utilize the original config setting, so it should be preserved for a deprecation period before complete removal: * if 'core.fsmonitor' is a boolean, the user is correctly using the new config syntax; do not use 'core.useBuiltinFSMonitor'. * if 'core.fsmonitor' is unspecified, use 'core.useBuiltinFSMonitor'. * if 'core.fsmonitor' is a path, override and use the builtin FSMonitor if 'core.useBuiltinFSMonitor' is 'true'; otherwise, use the FSMonitor hook indicated by the path. Additionally, for this deprecation period, advise users to switch to using 'core.fsmonitor' to specify their use of the builtin FSMonitor. Signed-off-by: Victoria Dye <vdye@github.com>
Configuration menu - View commit details
-
Copy full SHA for b4fb482 - Browse repository at this point
Copy the full SHA b4fb482View commit details -
dependabot: help keeping GitHub Actions versions up to date
See https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot#enabling-dependabot-version-updates-for-actions for details. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7a06b7b - Browse repository at this point
Copy the full SHA 7a06b7bView commit details -
Describe Git for Windows' architecture [no ci]
The Git for Windows project has grown quite complex over the years, certainly much more complex than during the first years where the `msysgit.git` repository was abusing Git for package management purposes and the `git/git` fork was called `4msysgit.git`. Let's describe the status quo in a thorough way. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 2ce307a - Browse repository at this point
Copy the full SHA 2ce307aView commit details -
Modify the Code of Conduct for Git for Windows
The Git project followed Git for Windows' lead and added their Code of Conduct, based on the Contributor Covenant v1.4, later updated to v2.0. We adapt it slightly to Git for Windows. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 26bccbe - Browse repository at this point
Copy the full SHA 26bccbeView commit details -
CONTRIBUTING.md: add guide for first-time contributors
Getting started contributing to Git can be difficult on a Windows machine. CONTRIBUTING.md contains a guide to getting started, including detailed steps for setting up build tools, running tests, and submitting patches to upstream. [includes an example by Pratik Karki how to submit v2, v3, v4, etc.] Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Configuration menu - View commit details
-
Copy full SHA for 141b8ab - Browse repository at this point
Copy the full SHA 141b8abView commit details -
README.md: Add a Windows-specific preamble
Includes touch-ups by 마누엘, Philip Oakley and 孙卓识. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for b134efe - Browse repository at this point
Copy the full SHA b134efeView commit details -
With improvements by Clive Chan, Adric Norris, Ben Bodenmiller and Philip Oakley. Helped-by: Clive Chan <cc@clive.io> Helped-by: Adric Norris <landstander668@gmail.com> Helped-by: Ben Bodenmiller <bbodenmiller@hotmail.com> Helped-by: Philip Oakley <philipoakley@iee.org> Signed-off-by: Brendan Forster <brendan@github.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for fa6fc63 - Browse repository at this point
Copy the full SHA fa6fc63View commit details -
Modify the GitHub Pull Request template (to reflect Git for Windows)
Git for Windows accepts pull requests; Core Git does not. Therefore we need to adjust the template (because it only matches core Git's project management style, not ours). Also: direct Git for Windows enhancements to their contributions page, space out the text for easy reading, and clarify that the mailing list is plain text, not HTML. Signed-off-by: Philip Oakley <philipoakley@iee.org> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 0ce6de6 - Browse repository at this point
Copy the full SHA 0ce6de6View commit details -
.github: Add configuration for the Sentiment Bot
The sentiment bot will help detect when things get too heated. Hopefully. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for e587604 - Browse repository at this point
Copy the full SHA e587604View commit details -
Document how $HOME is set on Windows
Git documentation refers to $HOME and $XDG_CONFIG_HOME often, but does not specify how or where these values come from on Windows where neither is set by default. The new documentation reflects the behavior of setup_windows_environment() in compat/mingw.c. Signed-off-by: Alejandro Barreto <alejandro.barreto@ni.com>
Configuration menu - View commit details
-
Copy full SHA for 2ff87a4 - Browse repository at this point
Copy the full SHA 2ff87a4View commit details -
SECURITY.md: document Git for Windows' policies
This is the recommended way on GitHub to describe policies revolving around security issues and about supported versions. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 9468391 - Browse repository at this point
Copy the full SHA 9468391View commit details -
Merge branch 'gitk-and-git-gui-patches'
These are Git for Windows' Git GUI and gitk patches. We will have to decide at some point what to do about them, but that's a little lower priority (as Git GUI seems to be unmaintained for the time being, and the gitk maintainer keeps a very low profile on the Git mailing list, too). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for c2afb5b - Browse repository at this point
Copy the full SHA c2afb5bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 17cfdb4 - Browse repository at this point
Copy the full SHA 17cfdb4View commit details -
Configuration menu - View commit details
-
Copy full SHA for cd09bf7 - Browse repository at this point
Copy the full SHA cd09bf7View commit details -
Merge pull request git-for-windows#3817 from mathstuf/name-too-long-a…
…dvice clean: suggest using `core.longPaths` if paths are too long to remove
Configuration menu - View commit details
-
Copy full SHA for 0f7ef80 - Browse repository at this point
Copy the full SHA 0f7ef80View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8af67a1 - Browse repository at this point
Copy the full SHA 8af67a1View commit details -
Merge branch 'kblees/kb/symlinks'
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 00488c3 - Browse repository at this point
Copy the full SHA 00488c3View commit details -
Merge 'docker-volumes-are-no-symlinks'
This was pull request git-for-windows#1645 from ZCube/master Support windows container. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 139da25 - Browse repository at this point
Copy the full SHA 139da25View commit details -
mingw: try resetting the read-only bit if rename fails (git-for-windo…
…ws#4527) With this patch, Git for Windows works as intended on mounted APFS volumes (where renaming read-only files would fail). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7dff356 - Browse repository at this point
Copy the full SHA 7dff356View commit details -
Merge pull request git-for-windows#1897 from piscisaureus/symlink-attr
Specify symlink type in .gitattributes
Configuration menu - View commit details
-
Copy full SHA for 56a2b67 - Browse repository at this point
Copy the full SHA 56a2b67View commit details -
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 3255bd8 - Browse repository at this point
Copy the full SHA 3255bd8View commit details -
Merge branch 'wsl-file-mode-bits'
This patch introduces support to set special NTFS attributes that are interpreted by the Windows Subsystem for Linux as file mode bits, UID and GID. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for c232aa7 - Browse repository at this point
Copy the full SHA c232aa7View commit details -
Merge pull request git-for-windows#1170 from dscho/mingw-kill-process
Handle Ctrl+C in Git Bash nicely Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for f28f927 - Browse repository at this point
Copy the full SHA f28f927View commit details -
Merge pull request git-for-windows#3492 from dscho/ns/batched-fsync
Switch to batched fsync by default
Configuration menu - View commit details
-
Copy full SHA for bd3f271 - Browse repository at this point
Copy the full SHA bd3f271View commit details -
Merge branch 'un-revert-editor-save-and-reset'
A fix for calling `vim` in Windows Terminal caused a regression and was reverted. We partially un-revert this, to get the fix again. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 61490f2 - Browse repository at this point
Copy the full SHA 61490f2View commit details -
Merge branch 'phase-out-reset-stdin'
This topic branch re-adds the deprecated --stdin/-z options to `git reset`. Those patches were overridden by a different set of options in the upstream Git project before we could propose `--stdin`. We offered this in MinGit to applications that wanted a safer way to pass lots of pathspecs to Git, and these applications will need to be adjusted. Instead of `--stdin`, `--pathspec-from-file=-` should be used, and instead of `-z`, `--pathspec-file-nul`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for ebcb5b5 - Browse repository at this point
Copy the full SHA ebcb5b5View commit details -
Merge branch 'deprecate-core.useBuiltinFSMonitor'
Originally introduced as `core.useBuiltinFSMonitor` in Git for Windows and developed, improved and stabilized there, the built-in FSMonitor only made it into upstream Git (after unnecessarily long hemming and hawing and throwing overly perfectionist style review sticks into the spokes) as `core.fsmonitor = true`. In Git for Windows, with this topic branch, we re-introduce the now-obsolete config setting, with warnings suggesting to existing users how to switch to the new config setting, with the intention to ultimately drop the patch at some stage. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for b608f0e - Browse repository at this point
Copy the full SHA b608f0eView commit details -
Merge pull request git-for-windows#2837 from dscho/monitor-component-…
…updates Start monitoring updates of Git for Windows' component in the open
Configuration menu - View commit details
-
Copy full SHA for 02788b7 - Browse repository at this point
Copy the full SHA 02788b7View commit details -
Add a README.md for GitHub goodness. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for ed7f388 - Browse repository at this point
Copy the full SHA ed7f388View commit details -
fixup! path-walk API: avoid adding a root tree more than once
Ooops. Must not risk a segmentation fault in a partial clone missing trees... Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for d53e464 - Browse repository at this point
Copy the full SHA d53e464View commit details