From df67d73ca3268eec5c924d6fe9d2c050ce23f3b1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Thalheim?= Date: Sun, 17 May 2026 15:21:11 +0200 Subject: [PATCH 01/47] config: retry acquiring config.lock, configurable via core.configLockTimeout MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Concurrent config writers race for the ".lock" file, which is taken with open(O_EXCL) and no retry, so the losers fail right away with "could not lock config file". This shows up with parallel "git worktree add -b" against the same repository: each one writes a couple of branch.* keys and the losers fail at random. Worse, "git worktree add" doesn't propagate that failure to its exit code, so the tracking config is silently dropped. (The swallowed error is a separate bug.) Retry instead of giving up on the first EEXIST. The lock is only held while rewriting a small file, so the loser only has to wait out the other writers. Same approach as 4ff0f01cb7 (refs: retry acquiring reference locks for 100ms, 2017-08-21). On the semantics: the on-disk config is read only after the lock is taken, so writers touching different keys can't lose each other's change. Writers touching the same key still get last-writer-wins, but that is already the case today and would need a compare-and-swap config API to fix. The retry only turns hard failures into successes. Default to 1000ms, like core.packedRefsTimeout: same shape of problem, one shared file everyone serializes through. A larger timeout only costs anything when a stale lock is left behind by a crash, which is rare; a smaller one fails spuriously on slow filesystems (NTFS has been seen needing more than 100ms). Make it configurable as core.configLockTimeout. There is no chicken-and-egg problem: we read the config before we lock it. microsoft/git carries a similar patch (core.configWriteLockTimeoutMS, default off) for Scalar's tests. Defaulting to non-zero here because the worktree case fails silently. Helped-by: Patrick Steinhardt Helped-by: Johannes Schindelin Signed-off-by: Jörg Thalheim Signed-off-by: Junio C Hamano --- Documentation/config/core.adoc | 8 ++++++++ config.c | 24 ++++++++++++++++++++++-- t/t1300-config.sh | 17 +++++++++++++++++ t/t3200-branch.sh | 6 ++++-- t/t5505-remote.sh | 3 ++- 5 files changed, 53 insertions(+), 5 deletions(-) diff --git a/Documentation/config/core.adoc b/Documentation/config/core.adoc index a0ebf03e2eb050..340329edc38143 100644 --- a/Documentation/config/core.adoc +++ b/Documentation/config/core.adoc @@ -589,6 +589,14 @@ core.packedRefsTimeout:: all; -1 means to try indefinitely. Default is 1000 (i.e., retry for 1 second). +core.configLockTimeout:: + The length of time, in milliseconds, to retry when trying to + lock a configuration file for writing. Value 0 means not to + retry at all; -1 means to try indefinitely. Default is 1000 + (i.e., retry for 1 second). This is read from the configuration + that is already on disk before the lock is taken, so it can be + set persistently like any other option. + core.pager:: Text viewer for use by Git commands (e.g., 'less'). The value is meant to be interpreted by the shell. The order of preference diff --git a/config.c b/config.c index 156f2a24fa0027..ac9563781bfb21 100644 --- a/config.c +++ b/config.c @@ -2903,6 +2903,24 @@ char *git_config_prepare_comment_string(const char *comment) return prepared; } +/* + * How long to retry acquiring config.lock when another process holds + * it. Default matches core.packedRefsTimeout; override via + * core.configLockTimeout. + */ +static long config_lock_timeout_ms(struct repository *r) +{ + static int configured; + static int timeout_ms = 1000; + + if (!configured) { + repo_config_get_int(r, "core.configlocktimeout", &timeout_ms); + configured = 1; + } + + return timeout_ms; +} + static void validate_comment_string(const char *comment) { size_t leading_blanks; @@ -2986,7 +3004,8 @@ int repo_config_set_multivar_in_file_gently(struct repository *r, * The lock serves a purpose in addition to locking: the new * contents of .git/config will be written into it. */ - fd = hold_lock_file_for_update(&lock, config_filename, 0); + fd = hold_lock_file_for_update_timeout(&lock, config_filename, 0, + config_lock_timeout_ms(r)); if (fd < 0) { error_errno(_("could not lock config file %s"), config_filename); ret = CONFIG_NO_LOCK; @@ -3331,7 +3350,8 @@ static int repo_config_copy_or_rename_section_in_file( if (!config_filename) config_filename = filename_buf = repo_git_path(r, "config"); - out_fd = hold_lock_file_for_update(&lock, config_filename, 0); + out_fd = hold_lock_file_for_update_timeout(&lock, config_filename, 0, + config_lock_timeout_ms(r)); if (out_fd < 0) { ret = error(_("could not lock config file %s"), config_filename); goto out; diff --git a/t/t1300-config.sh b/t/t1300-config.sh index 128971ee12fa6c..12a1cb85809b8b 100755 --- a/t/t1300-config.sh +++ b/t/t1300-config.sh @@ -2939,4 +2939,21 @@ test_expect_success 'writing value with trailing CR not stripped on read' ' test_cmp expect actual ' +test_expect_success 'writing config fails immediately with core.configLockTimeout=0' ' + test_when_finished "rm -f .git/config.lock" && + >.git/config.lock && + test_must_fail git -c core.configLockTimeout=0 config foo.bar baz 2>err && + test_grep "could not lock config file" err +' + +test_expect_success 'writing config retries until lock is released' ' + test_when_finished "rm -f .git/config.lock" && + >.git/config.lock && + { + ( sleep 1 && rm -f .git/config.lock ) & + } && + git -c core.configLockTimeout=5000 config retried.key value && + test "$(git config retried.key)" = value +' + test_done diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh index e7829c2c4bfdc3..5f8a31c21d0eaf 100755 --- a/t/t3200-branch.sh +++ b/t/t3200-branch.sh @@ -1037,7 +1037,8 @@ test_expect_success '--set-upstream-to fails on locked config' ' test_when_finished "rm -f .git/config.lock" && >.git/config.lock && git branch locked && - test_must_fail git branch --set-upstream-to locked 2>err && + test_must_fail git -c core.configLockTimeout=0 \ + branch --set-upstream-to locked 2>err && test_grep "could not lock config file .git/config" err ' @@ -1068,7 +1069,8 @@ test_expect_success '--unset-upstream should fail if config is locked' ' test_when_finished "rm -f .git/config.lock" && git branch --set-upstream-to locked && >.git/config.lock && - test_must_fail git branch --unset-upstream 2>err && + test_must_fail git -c core.configLockTimeout=0 \ + branch --unset-upstream 2>err && test_grep "could not lock config file .git/config" err ' diff --git a/t/t5505-remote.sh b/t/t5505-remote.sh index e592c0bcde91e9..aea9222649f7bb 100755 --- a/t/t5505-remote.sh +++ b/t/t5505-remote.sh @@ -1327,7 +1327,8 @@ test_expect_success 'remote set-url with locked config' ' test_when_finished "rm -f .git/config.lock" && git config --get-all remote.someremote.url >expect && >.git/config.lock && - test_must_fail git remote set-url someremote baz && + test_must_fail git -c core.configLockTimeout=0 \ + remote set-url someremote baz && git config --get-all remote.someremote.url >actual && cmp expect actual ' From 0201d2783e6317c7b76e9c511cbfcfa73fdc17b1 Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Tue, 14 Jul 2026 11:25:16 +0800 Subject: [PATCH 02/47] repository: introduce repo_config_values_clear() As part of the ongoing libification effort, dynamically allocated global configuration variables are being moved into 'struct repo_config_values'. To prevent memory leaks, we need a destructor to free these heap-allocated variables when a repository instance is torn down. Introduce 'repo_config_values_clear()' in environment.c and invoke it from 'repo_clear()' in repository.c. As a starting point, update this new function to handle the cleanup of 'attributes_file'. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- environment.c | 5 +++++ environment.h | 9 +++++++++ repository.c | 1 + 3 files changed, 15 insertions(+) diff --git a/environment.c b/environment.c index ba2c60103ff51c..ae05f16d041858 100644 --- a/environment.c +++ b/environment.c @@ -726,3 +726,8 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->sparse_expect_files_outside_of_patterns = 0; cfg->warn_on_object_refname_ambiguity = 1; } + +void repo_config_values_clear(struct repo_config_values *cfg) +{ + FREE_AND_NULL(cfg->attributes_file); +} diff --git a/environment.h b/environment.h index 6f182869558395..9169d7f62d9941 100644 --- a/environment.h +++ b/environment.h @@ -135,6 +135,15 @@ int git_default_core_config(const char *var, const char *value, void repo_config_values_init(struct repo_config_values *cfg); +/* + * Frees memory allocated for dynamically loaded configuration values + * inside `repo_config_values`. + * + * As dynamically allocated variables are migrated into this struct, + * their FREE_AND_NULL() calls should be appended here. + */ +void repo_config_values_clear(struct repo_config_values *cfg); + /* * TODO: All the below state either explicitly or implicitly relies on * `the_repository`. We should eventually get rid of these and make the diff --git a/repository.c b/repository.c index 187dd471c4e607..669e2d12004d77 100644 --- a/repository.c +++ b/repository.c @@ -388,6 +388,7 @@ void repo_clear(struct repository *repo) FREE_AND_NULL(repo->parsed_objects); repo_settings_clear(repo); + repo_config_values_clear(&repo->config_values_private_); if (repo->config) { git_configset_clear(repo->config); From 7488b9d1fd505ce1194b98893508fa5b194cf0cd Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Tue, 14 Jul 2026 11:25:17 +0800 Subject: [PATCH 03/47] environment: move excludes_file into repo_config_values The global variable 'excludes_file' is used to track the path to the global ignore file. If this variable is NULL, 'setup_standard_excludes()' in 'dir.c' forcefully evaluates and assigns the XDG default path to it. Continue the libification effort by encapsulating this lazy-loading fallback logic into a proper getter and moving the variable into 'struct repo_config_values'. Since 'excludes_file' is a dynamically allocated string, it requires proper heap memory management. It is safely freed using the newly introduced 'repo_config_values_clear()' function when the repository is torn down. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- dir.c | 4 ++-- environment.c | 17 ++++++++++++++--- environment.h | 4 +++- 3 files changed, 19 insertions(+), 6 deletions(-) diff --git a/dir.c b/dir.c index 32430090dcdf26..baaf899d9c47c4 100644 --- a/dir.c +++ b/dir.c @@ -3481,11 +3481,11 @@ static GIT_PATH_FUNC(git_path_info_exclude, "info/exclude") void setup_standard_excludes(struct dir_struct *dir) { + const char *excludes_file = repo_excludes_file(the_repository); + dir->exclude_per_dir = ".gitignore"; /* core.excludesfile defaulting to $XDG_CONFIG_HOME/git/ignore */ - if (!excludes_file) - excludes_file = xdg_config_home("ignore"); if (excludes_file && !access_or_warn(excludes_file, R_OK, 0)) add_patterns_from_file_1(dir, excludes_file, dir->untracked ? &dir->internal.ss_excludes_file : NULL); diff --git a/environment.c b/environment.c index ae05f16d041858..275931c213fb5c 100644 --- a/environment.c +++ b/environment.c @@ -57,7 +57,6 @@ enum fsync_method fsync_method = FSYNC_METHOD_DEFAULT; enum fsync_component fsync_components = FSYNC_COMPONENTS_DEFAULT; char *editor_program; char *askpass_program; -char *excludes_file; enum auto_crlf auto_crlf = AUTO_CRLF_FALSE; enum eol core_eol = EOL_UNSET; int global_conv_flags_eol = CONV_EOL_RNDTRP_WARN; @@ -134,6 +133,16 @@ int is_bare_repository(void) return is_bare_repository_cfg && !repo_get_work_tree(the_repository); } +const char *repo_excludes_file(struct repository *repo) +{ + struct repo_config_values *cfg = repo_config_values(repo); + + if (!cfg->excludes_file) + cfg->excludes_file = xdg_config_home("ignore"); + + return cfg->excludes_file; +} + int have_git_dir(void) { return startup_info->have_repository @@ -461,8 +470,8 @@ int git_default_core_config(const char *var, const char *value, } if (!strcmp(var, "core.excludesfile")) { - FREE_AND_NULL(excludes_file); - return git_config_pathname(&excludes_file, var, value); + FREE_AND_NULL(cfg->excludes_file); + return git_config_pathname(&cfg->excludes_file, var, value); } if (!strcmp(var, "core.whitespace")) { @@ -715,6 +724,7 @@ int git_default_config(const char *var, const char *value, void repo_config_values_init(struct repo_config_values *cfg) { cfg->attributes_file = NULL; + cfg->excludes_file = NULL; cfg->apply_sparse_checkout = 0; cfg->branch_track = BRANCH_TRACK_REMOTE; cfg->trust_ctime = 1; @@ -730,4 +740,5 @@ void repo_config_values_init(struct repo_config_values *cfg) void repo_config_values_clear(struct repo_config_values *cfg) { FREE_AND_NULL(cfg->attributes_file); + FREE_AND_NULL(cfg->excludes_file); } diff --git a/environment.h b/environment.h index 9169d7f62d9941..4776ccc6579051 100644 --- a/environment.h +++ b/environment.h @@ -90,6 +90,7 @@ struct repository; struct repo_config_values { /* section "core" config values */ char *attributes_file; + char *excludes_file; int apply_sparse_checkout; int trust_ctime; int check_stat; @@ -133,6 +134,8 @@ int git_default_config(const char *, const char *, int git_default_core_config(const char *var, const char *value, const struct config_context *ctx, void *cb); +const char *repo_excludes_file(struct repository *repo); + void repo_config_values_init(struct repo_config_values *cfg); /* @@ -217,7 +220,6 @@ extern char *git_log_output_encoding; extern char *editor_program; extern char *askpass_program; -extern char *excludes_file; /* * The character that begins a commented line in user-editable file From 469e95c2573e938c0963ba52e8d06dd6aa7ba262 Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Tue, 14 Jul 2026 11:25:18 +0800 Subject: [PATCH 04/47] environment: move editor_program into repo_config_values The global variable 'editor_program' holds the path to the user's preferred editor. Move 'editor_program' into 'struct repo_config_values' to continue the libification effort. There have been discussions on whether external programs like editors truly need to be configured on a per-repository basis within the same process. While a single process might rarely invoke different editors, this migration is necessary for two reasons: 1. Developers frequently use different toolchains for different projects. Per-repo configuration respects this. 2. Moving this string into 'repo_config_values' eliminates mutable global state. As the codebase moves toward becoming a long-running processes, managing multiple repositories concurrently must not overwrite each other's program configurations. No standalone getter function is introduced. Callers directly access the field via 'repo_config_values()'. Heap memory is safely reclaimed in 'repo_config_values_clear()'. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- editor.c | 4 ++-- environment.c | 7 ++++--- environment.h | 2 +- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/editor.c b/editor.c index fd174e6a034f1c..0d1cb8768db1a0 100644 --- a/editor.c +++ b/editor.c @@ -29,8 +29,8 @@ const char *git_editor(void) const char *editor = getenv("GIT_EDITOR"); int terminal_is_dumb = is_terminal_dumb(); - if (!editor && editor_program) - editor = editor_program; + if (!editor) + editor = repo_config_values(the_repository)->editor_program; if (!editor && !terminal_is_dumb) editor = getenv("VISUAL"); if (!editor) diff --git a/environment.c b/environment.c index 275931c213fb5c..a65d575af4a80d 100644 --- a/environment.c +++ b/environment.c @@ -55,7 +55,6 @@ int fsync_object_files = -1; int use_fsync = -1; enum fsync_method fsync_method = FSYNC_METHOD_DEFAULT; enum fsync_component fsync_components = FSYNC_COMPONENTS_DEFAULT; -char *editor_program; char *askpass_program; enum auto_crlf auto_crlf = AUTO_CRLF_FALSE; enum eol core_eol = EOL_UNSET; @@ -437,8 +436,8 @@ int git_default_core_config(const char *var, const char *value, } if (!strcmp(var, "core.editor")) { - FREE_AND_NULL(editor_program); - return git_config_string(&editor_program, var, value); + FREE_AND_NULL(cfg->editor_program); + return git_config_string(&cfg->editor_program, var, value); } if (!strcmp(var, "core.commentchar") || @@ -725,6 +724,7 @@ void repo_config_values_init(struct repo_config_values *cfg) { cfg->attributes_file = NULL; cfg->excludes_file = NULL; + cfg->editor_program = NULL; cfg->apply_sparse_checkout = 0; cfg->branch_track = BRANCH_TRACK_REMOTE; cfg->trust_ctime = 1; @@ -741,4 +741,5 @@ void repo_config_values_clear(struct repo_config_values *cfg) { FREE_AND_NULL(cfg->attributes_file); FREE_AND_NULL(cfg->excludes_file); + FREE_AND_NULL(cfg->editor_program); } diff --git a/environment.h b/environment.h index 4776ccc6579051..8178ebab76bba4 100644 --- a/environment.h +++ b/environment.h @@ -91,6 +91,7 @@ struct repo_config_values { /* section "core" config values */ char *attributes_file; char *excludes_file; + char *editor_program; int apply_sparse_checkout; int trust_ctime; int check_stat; @@ -218,7 +219,6 @@ const char *get_commit_output_encoding(void); extern char *git_commit_encoding; extern char *git_log_output_encoding; -extern char *editor_program; extern char *askpass_program; /* From e57125d804f29b8a6c70a47a03037cf31ef153d7 Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Tue, 14 Jul 2026 11:25:19 +0800 Subject: [PATCH 05/47] environment: move pager_program into repo_config_values The 'pager_program' variable is currently defined as a file-scoped static string in pager.c. Move it into 'struct repo_config_values'. The configuration parsing logic remains strictly within pager.c to respect subsystem boundaries. The read/write operations are simply redirected to the repository-specific structure using 'repo_config_values()'. All current callers indeed pass 'the_repository', so this new enforcement does not harm them. Similar to the recent editor_program migration, no standalone getter is introduced to keep the code minimal. The dynamically allocated memory is now managed by 'repo_config_values_clear()'. On top of that, fix memory leaks in pager.c while we are at it. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- environment.c | 2 ++ environment.h | 1 + pager.c | 32 +++++++++++++++++++++++--------- 3 files changed, 26 insertions(+), 9 deletions(-) diff --git a/environment.c b/environment.c index a65d575af4a80d..975c9cb9ebdb85 100644 --- a/environment.c +++ b/environment.c @@ -725,6 +725,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->attributes_file = NULL; cfg->excludes_file = NULL; cfg->editor_program = NULL; + cfg->pager_program = NULL; cfg->apply_sparse_checkout = 0; cfg->branch_track = BRANCH_TRACK_REMOTE; cfg->trust_ctime = 1; @@ -742,4 +743,5 @@ void repo_config_values_clear(struct repo_config_values *cfg) FREE_AND_NULL(cfg->attributes_file); FREE_AND_NULL(cfg->excludes_file); FREE_AND_NULL(cfg->editor_program); + FREE_AND_NULL(cfg->pager_program); } diff --git a/environment.h b/environment.h index 8178ebab76bba4..39b6691b478544 100644 --- a/environment.h +++ b/environment.h @@ -92,6 +92,7 @@ struct repo_config_values { char *attributes_file; char *excludes_file; char *editor_program; + char *pager_program; int apply_sparse_checkout; int trust_ctime; int check_stat; diff --git a/pager.c b/pager.c index 35b210e0484f90..543ef129366a19 100644 --- a/pager.c +++ b/pager.c @@ -5,6 +5,8 @@ #include "run-command.h" #include "sigchain.h" #include "alias.h" +#include "repository.h" +#include "environment.h" int pager_use_color = 1; @@ -13,7 +15,6 @@ int pager_use_color = 1; #endif static struct child_process pager_process; -static char *pager_program; static int old_fd1 = -1, old_fd2 = -1; /* Is the value coming back from term_columns() just a guess? */ @@ -75,10 +76,17 @@ static void wait_for_pager_signal(int signo) static int core_pager_config(const char *var, const char *value, const struct config_context *ctx UNUSED, - void *data UNUSED) + void *data) { - if (!strcmp(var, "core.pager")) - return git_config_string(&pager_program, var, value); + struct repository *r = data; + + if (!strcmp(var, "core.pager")) { + struct repo_config_values *cfg = repo_config_values(r); + + FREE_AND_NULL(cfg->pager_program); + return git_config_string(&cfg->pager_program, var, value); + } + return 0; } @@ -91,10 +99,12 @@ const char *git_pager(struct repository *r, int stdout_is_tty) pager = getenv("GIT_PAGER"); if (!pager) { - if (!pager_program) + struct repo_config_values *cfg = repo_config_values(r); + + if (!cfg->pager_program) read_early_config(r, - core_pager_config, NULL); - pager = pager_program; + core_pager_config, r); + pager = cfg->pager_program; } if (!pager) pager = getenv("PAGER"); @@ -302,7 +312,11 @@ int check_pager_config(struct repository *r, const char *cmd) read_early_config(r, pager_command_config, &data); - if (data.value) - pager_program = data.value; + if (data.value) { + struct repo_config_values *cfg = repo_config_values(r); + + free(cfg->pager_program); + cfg->pager_program = data.value; + } return data.want; } From 48cbe400794bd055d259143a01024da71da5fe1a Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Tue, 14 Jul 2026 11:25:20 +0800 Subject: [PATCH 06/47] environment: move askpass_program into repo_config_values The global variable 'askpass_program' stores the path to the program used to prompt the user for credentials. Move it into repo_config_values to continue the libification effort. While it is uncommon for a single process to require different askpass programs for different repositories, maintaining this value as a mutable global string is a blocker for libification. Global heap-allocated strings introduce thread-safety issues in a multi-repo environment. Move 'askpass_program' into 'struct repo_config_values' to eliminate this global state. The memory is now safely managed and freed via 'repo_config_values_clear()'. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- environment.c | 7 ++++--- environment.h | 3 +-- prompt.c | 3 ++- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/environment.c b/environment.c index 975c9cb9ebdb85..3857818da3f96c 100644 --- a/environment.c +++ b/environment.c @@ -55,7 +55,6 @@ int fsync_object_files = -1; int use_fsync = -1; enum fsync_method fsync_method = FSYNC_METHOD_DEFAULT; enum fsync_component fsync_components = FSYNC_COMPONENTS_DEFAULT; -char *askpass_program; enum auto_crlf auto_crlf = AUTO_CRLF_FALSE; enum eol core_eol = EOL_UNSET; int global_conv_flags_eol = CONV_EOL_RNDTRP_WARN; @@ -464,8 +463,8 @@ int git_default_core_config(const char *var, const char *value, } if (!strcmp(var, "core.askpass")) { - FREE_AND_NULL(askpass_program); - return git_config_string(&askpass_program, var, value); + FREE_AND_NULL(cfg->askpass_program); + return git_config_string(&cfg->askpass_program, var, value); } if (!strcmp(var, "core.excludesfile")) { @@ -726,6 +725,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->excludes_file = NULL; cfg->editor_program = NULL; cfg->pager_program = NULL; + cfg->askpass_program = NULL; cfg->apply_sparse_checkout = 0; cfg->branch_track = BRANCH_TRACK_REMOTE; cfg->trust_ctime = 1; @@ -744,4 +744,5 @@ void repo_config_values_clear(struct repo_config_values *cfg) FREE_AND_NULL(cfg->excludes_file); FREE_AND_NULL(cfg->editor_program); FREE_AND_NULL(cfg->pager_program); + FREE_AND_NULL(cfg->askpass_program); } diff --git a/environment.h b/environment.h index 39b6691b478544..856dc70cc47170 100644 --- a/environment.h +++ b/environment.h @@ -93,6 +93,7 @@ struct repo_config_values { char *excludes_file; char *editor_program; char *pager_program; + char *askpass_program; int apply_sparse_checkout; int trust_ctime; int check_stat; @@ -220,8 +221,6 @@ const char *get_commit_output_encoding(void); extern char *git_commit_encoding; extern char *git_log_output_encoding; -extern char *askpass_program; - /* * The character that begins a commented line in user-editable file * that is subject to stripspace. diff --git a/prompt.c b/prompt.c index 706fba2a5030c7..d8d74c7e379dbe 100644 --- a/prompt.c +++ b/prompt.c @@ -3,6 +3,7 @@ #include "git-compat-util.h" #include "parse.h" #include "environment.h" +#include "repository.h" #include "run-command.h" #include "strbuf.h" #include "prompt.h" @@ -51,7 +52,7 @@ char *git_prompt(const char *prompt, int flags) askpass = getenv("GIT_ASKPASS"); if (!askpass) - askpass = askpass_program; + askpass = repo_config_values(the_repository)->askpass_program; if (!askpass) askpass = getenv("SSH_ASKPASS"); if (askpass && *askpass) From a9f5e90fb9068fd1a1a9a7c9a60f005e7ed392dd Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Tue, 14 Jul 2026 11:25:21 +0800 Subject: [PATCH 07/47] environment: migrate apply_default_whitespace and apply_default_ignorewhitespace The global variables 'apply_default_whitespace' and 'apply_default_ignorewhitespace' are used to store the default whitespace configuration for 'git apply'. Move these variables into 'struct repo_config_values' to continue the libification effort. Dynamically allocated strings fetched via 'repo_config_get_string()' are now tracked per-repository and safely freed in 'repo_config_values_clear()'. As part of this transition, update 'git_apply_config()' to accept a 'struct repository *' argument rather than relying on the 'the_repository' global. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- apply.c | 28 ++++++++++++++++++++-------- environment.c | 6 ++++-- environment.h | 4 ++-- 3 files changed, 26 insertions(+), 12 deletions(-) diff --git a/apply.c b/apply.c index 5e54453f799ea2..7e5b12323ea79f 100644 --- a/apply.c +++ b/apply.c @@ -47,11 +47,17 @@ struct gitdiff_data { int p_value; }; -static void git_apply_config(void) +static void git_apply_config(struct repository *repo) { - repo_config_get_string(the_repository, "apply.whitespace", &apply_default_whitespace); - repo_config_get_string(the_repository, "apply.ignorewhitespace", &apply_default_ignorewhitespace); - repo_config(the_repository, git_xmerge_config, NULL); + struct repo_config_values *cfg = repo_config_values(repo); + + FREE_AND_NULL(cfg->apply_default_whitespace); + repo_config_get_string(repo, "apply.whitespace", + &cfg->apply_default_whitespace); + FREE_AND_NULL(cfg->apply_default_ignorewhitespace); + repo_config_get_string(repo, "apply.ignorewhitespace", + &cfg->apply_default_ignorewhitespace); + repo_config(repo, git_xmerge_config, NULL); } static int parse_whitespace_option(struct apply_state *state, const char *option) @@ -109,6 +115,8 @@ int init_apply_state(struct apply_state *state, struct repository *repo, const char *prefix) { + struct repo_config_values *cfg = repo_config_values(repo); + memset(state, 0, sizeof(*state)); state->prefix = prefix; state->repo = repo; @@ -126,10 +134,13 @@ int init_apply_state(struct apply_state *state, strset_init(&state->kept_symlinks); strbuf_init(&state->root, 0); - git_apply_config(); - if (apply_default_whitespace && parse_whitespace_option(state, apply_default_whitespace)) + git_apply_config(repo); + + if (cfg->apply_default_whitespace && + parse_whitespace_option(state, cfg->apply_default_whitespace)) return -1; - if (apply_default_ignorewhitespace && parse_ignorewhitespace_option(state, apply_default_ignorewhitespace)) + if (cfg->apply_default_ignorewhitespace && + parse_ignorewhitespace_option(state, cfg->apply_default_ignorewhitespace)) return -1; return 0; } @@ -192,7 +203,8 @@ int check_apply_state(struct apply_state *state, int force_apply) static void set_default_whitespace_mode(struct apply_state *state) { - if (!state->whitespace_option && !apply_default_whitespace) + if (!state->whitespace_option && + !repo_config_values(state->repo)->apply_default_whitespace) state->ws_error_action = (state->apply ? warn_on_ws_error : nowarn_ws_error); } diff --git a/environment.c b/environment.c index 3857818da3f96c..20500658a20d67 100644 --- a/environment.c +++ b/environment.c @@ -49,8 +49,6 @@ int assume_unchanged; int is_bare_repository_cfg = -1; /* unspecified */ char *git_commit_encoding; char *git_log_output_encoding; -char *apply_default_whitespace; -char *apply_default_ignorewhitespace; int fsync_object_files = -1; int use_fsync = -1; enum fsync_method fsync_method = FSYNC_METHOD_DEFAULT; @@ -726,6 +724,8 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->editor_program = NULL; cfg->pager_program = NULL; cfg->askpass_program = NULL; + cfg->apply_default_whitespace = NULL; + cfg->apply_default_ignorewhitespace = NULL; cfg->apply_sparse_checkout = 0; cfg->branch_track = BRANCH_TRACK_REMOTE; cfg->trust_ctime = 1; @@ -745,4 +745,6 @@ void repo_config_values_clear(struct repo_config_values *cfg) FREE_AND_NULL(cfg->editor_program); FREE_AND_NULL(cfg->pager_program); FREE_AND_NULL(cfg->askpass_program); + FREE_AND_NULL(cfg->apply_default_whitespace); + FREE_AND_NULL(cfg->apply_default_ignorewhitespace); } diff --git a/environment.h b/environment.h index 856dc70cc47170..f450242ac03bad 100644 --- a/environment.h +++ b/environment.h @@ -94,6 +94,8 @@ struct repo_config_values { char *editor_program; char *pager_program; char *askpass_program; + char *apply_default_whitespace; + char *apply_default_ignorewhitespace; int apply_sparse_checkout; int trust_ctime; int check_stat; @@ -182,8 +184,6 @@ extern int has_symlinks; extern int minimum_abbrev, default_abbrev; extern int ignore_case; extern int assume_unchanged; -extern char *apply_default_whitespace; -extern char *apply_default_ignorewhitespace; extern unsigned long pack_size_limit_cfg; extern int protect_hfs; From 1a6c84e98dffe06fbe1acdf03817dffa10e180ef Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Tue, 14 Jul 2026 11:25:22 +0800 Subject: [PATCH 08/47] environment: move push_default into repo_config_values The global variable 'push_default' specifies the default behavior of 'git push' when no explicit refspec is provided. Move 'push_default' into 'struct repo_config_values' to continue the libification effort. While 'enum push_default_type' ideally belongs in 'remote.h', moving it there introduces a circular dependency chain: remote.h -> hash.h -> repository.h -> environment.h. Therefore, the enum definition is kept in 'environment.h' just above 'struct repo_config_values' with a NEEDSWORK comment for future cleanup. Modify the configuration parsing in environment.c to update the per-repository structure directly, and update caller across the codebase to access the value via 'repo_config_values()'. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- builtin/push.c | 10 ++++++---- environment.c | 16 +++++++++------- environment.h | 26 ++++++++++++++++---------- remote.c | 2 +- 4 files changed, 32 insertions(+), 22 deletions(-) diff --git a/builtin/push.c b/builtin/push.c index 6021b71d668455..7578ff38c42ef3 100644 --- a/builtin/push.c +++ b/builtin/push.c @@ -73,6 +73,7 @@ static void refspec_append_mapped(struct refspec *refspec, const char *ref, struct remote *remote, struct ref *matched) { const char *branch_name; + struct repo_config_values *cfg = repo_config_values(the_repository); if (remote->push.nr) { struct refspec_item query = { @@ -88,7 +89,7 @@ static void refspec_append_mapped(struct refspec *refspec, const char *ref, } } - if (push_default == PUSH_DEFAULT_UPSTREAM && + if (cfg->push_default == PUSH_DEFAULT_UPSTREAM && skip_prefix(matched->name, "refs/heads/", &branch_name)) { struct branch *branch = branch_get(branch_name); if (branch->merge_nr == 1 && branch->merge[0]->src) { @@ -160,7 +161,7 @@ static NORETURN void die_push_simple(struct branch *branch, * Don't show advice for people who explicitly set * push.default. */ - if (push_default == PUSH_DEFAULT_UNSPECIFIED) + if (cfg->push_default == PUSH_DEFAULT_UNSPECIFIED) advice_pushdefault_maybe = _("\n" "To choose either option permanently, " "see push.default in 'git help config'.\n"); @@ -231,8 +232,9 @@ static void setup_default_push_refspecs(int *flags, struct remote *remote) struct branch *branch; const char *dst; int same_remote; + struct repo_config_values *cfg = repo_config_values(the_repository); - switch (push_default) { + switch (cfg->push_default) { case PUSH_DEFAULT_MATCHING: refspec_append(&rs, ":"); return; @@ -252,7 +254,7 @@ static void setup_default_push_refspecs(int *flags, struct remote *remote) dst = branch->refname; same_remote = !strcmp(remote->name, remote_for_branch(branch, NULL)); - switch (push_default) { + switch (cfg->push_default) { default: case PUSH_DEFAULT_UNSPECIFIED: case PUSH_DEFAULT_SIMPLE: diff --git a/environment.c b/environment.c index 20500658a20d67..66c1ac1ab8c0ee 100644 --- a/environment.c +++ b/environment.c @@ -58,7 +58,6 @@ enum eol core_eol = EOL_UNSET; int global_conv_flags_eol = CONV_EOL_RNDTRP_WARN; char *check_roundtrip_encoding; enum rebase_setup_type autorebase = AUTOREBASE_NEVER; -enum push_default_type push_default = PUSH_DEFAULT_UNSPECIFIED; #ifndef OBJECT_CREATION_MODE #define OBJECT_CREATION_MODE OBJECT_CREATION_USES_HARDLINKS #endif @@ -620,21 +619,23 @@ static int git_default_branch_config(const char *var, const char *value) static int git_default_push_config(const char *var, const char *value) { + struct repo_config_values *cfg = repo_config_values(the_repository); + if (!strcmp(var, "push.default")) { if (!value) return config_error_nonbool(var); else if (!strcmp(value, "nothing")) - push_default = PUSH_DEFAULT_NOTHING; + cfg->push_default = PUSH_DEFAULT_NOTHING; else if (!strcmp(value, "matching")) - push_default = PUSH_DEFAULT_MATCHING; + cfg->push_default = PUSH_DEFAULT_MATCHING; else if (!strcmp(value, "simple")) - push_default = PUSH_DEFAULT_SIMPLE; + cfg->push_default = PUSH_DEFAULT_SIMPLE; else if (!strcmp(value, "upstream")) - push_default = PUSH_DEFAULT_UPSTREAM; + cfg->push_default = PUSH_DEFAULT_UPSTREAM; else if (!strcmp(value, "tracking")) /* deprecated */ - push_default = PUSH_DEFAULT_UPSTREAM; + cfg->push_default = PUSH_DEFAULT_UPSTREAM; else if (!strcmp(value, "current")) - push_default = PUSH_DEFAULT_CURRENT; + cfg->push_default = PUSH_DEFAULT_CURRENT; else { error(_("malformed value for %s: %s"), var, value); return error(_("must be one of nothing, matching, simple, " @@ -726,6 +727,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->askpass_program = NULL; cfg->apply_default_whitespace = NULL; cfg->apply_default_ignorewhitespace = NULL; + cfg->push_default = PUSH_DEFAULT_UNSPECIFIED; cfg->apply_sparse_checkout = 0; cfg->branch_track = BRANCH_TRACK_REMOTE; cfg->trust_ctime = 1; diff --git a/environment.h b/environment.h index f450242ac03bad..17a3a628d2d299 100644 --- a/environment.h +++ b/environment.h @@ -87,6 +87,21 @@ extern const char * const local_repo_env[]; struct strvec; struct repository; + +/* + * NEEDSWORK: It would be better if these definitions could be moved to + * other more specific files, but care is needed to avoid circular + * inclusion issues. + */ +enum push_default_type { + PUSH_DEFAULT_NOTHING = 0, + PUSH_DEFAULT_MATCHING, + PUSH_DEFAULT_SIMPLE, + PUSH_DEFAULT_UPSTREAM, + PUSH_DEFAULT_CURRENT, + PUSH_DEFAULT_UNSPECIFIED +}; + struct repo_config_values { /* section "core" config values */ char *attributes_file; @@ -96,6 +111,7 @@ struct repo_config_values { char *askpass_program; char *apply_default_whitespace; char *apply_default_ignorewhitespace; + enum push_default_type push_default; int apply_sparse_checkout; int trust_ctime; int check_stat; @@ -197,16 +213,6 @@ enum rebase_setup_type { }; extern enum rebase_setup_type autorebase; -enum push_default_type { - PUSH_DEFAULT_NOTHING = 0, - PUSH_DEFAULT_MATCHING, - PUSH_DEFAULT_SIMPLE, - PUSH_DEFAULT_UPSTREAM, - PUSH_DEFAULT_CURRENT, - PUSH_DEFAULT_UNSPECIFIED -}; -extern enum push_default_type push_default; - enum object_creation_mode { OBJECT_CREATION_USES_HARDLINKS = 0, OBJECT_CREATION_USES_RENAMES = 1 diff --git a/remote.c b/remote.c index 00723b385e1d52..d48c01d37559df 100644 --- a/remote.c +++ b/remote.c @@ -1933,7 +1933,7 @@ static char *branch_get_push_1(struct repository *repo, if (remote->mirror) return tracking_for_push_dest(remote, branch->refname, err); - switch (push_default) { + switch (repo_config_values(repo)->push_default) { case PUSH_DEFAULT_NOTHING: return error_buf(err, _("push has no destination (push.default is 'nothing')")); From 1840ca005fbabc651f9018d88f0f9d8adb91b015 Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Tue, 14 Jul 2026 11:25:23 +0800 Subject: [PATCH 09/47] environment: move autorebase into repo_config_values The global variable 'autorebase' dictates whether a newly created branch should be configured to automatically rebase by default. Move it into 'struct repo_config_values' to continue the libification effort. The 'enum rebase_setup_type' definition is moved higher up in 'environment.h' so that it is visible to the repository-specific structure. The default state AUTOREBASE_NEVER is now correctly initialized in 'repo_config_values_init()'. Configuration parsing in 'git_default_branch_config()' is updated to write directly to the repository's configuration instance. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- branch.c | 2 +- environment.c | 10 +++++----- environment.h | 16 ++++++++-------- 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/branch.c b/branch.c index 243db7d0fc0226..e1c1f8c89dbc56 100644 --- a/branch.c +++ b/branch.c @@ -61,7 +61,7 @@ static int find_tracked_branch(struct remote *remote, void *priv) static int should_setup_rebase(const char *origin) { - switch (autorebase) { + switch (repo_config_values(the_repository)->autorebase) { case AUTOREBASE_NEVER: return 0; case AUTOREBASE_LOCAL: diff --git a/environment.c b/environment.c index 66c1ac1ab8c0ee..c0bf7577b7ad40 100644 --- a/environment.c +++ b/environment.c @@ -57,7 +57,6 @@ enum auto_crlf auto_crlf = AUTO_CRLF_FALSE; enum eol core_eol = EOL_UNSET; int global_conv_flags_eol = CONV_EOL_RNDTRP_WARN; char *check_roundtrip_encoding; -enum rebase_setup_type autorebase = AUTOREBASE_NEVER; #ifndef OBJECT_CREATION_MODE #define OBJECT_CREATION_MODE OBJECT_CREATION_USES_HARDLINKS #endif @@ -601,13 +600,13 @@ static int git_default_branch_config(const char *var, const char *value) if (!value) return config_error_nonbool(var); else if (!strcmp(value, "never")) - autorebase = AUTOREBASE_NEVER; + cfg->autorebase = AUTOREBASE_NEVER; else if (!strcmp(value, "local")) - autorebase = AUTOREBASE_LOCAL; + cfg->autorebase = AUTOREBASE_LOCAL; else if (!strcmp(value, "remote")) - autorebase = AUTOREBASE_REMOTE; + cfg->autorebase = AUTOREBASE_REMOTE; else if (!strcmp(value, "always")) - autorebase = AUTOREBASE_ALWAYS; + cfg->autorebase = AUTOREBASE_ALWAYS; else return error(_("malformed value for %s"), var); return 0; @@ -728,6 +727,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->apply_default_whitespace = NULL; cfg->apply_default_ignorewhitespace = NULL; cfg->push_default = PUSH_DEFAULT_UNSPECIFIED; + cfg->autorebase = AUTOREBASE_NEVER; cfg->apply_sparse_checkout = 0; cfg->branch_track = BRANCH_TRACK_REMOTE; cfg->trust_ctime = 1; diff --git a/environment.h b/environment.h index 17a3a628d2d299..46b2f0d861089e 100644 --- a/environment.h +++ b/environment.h @@ -102,6 +102,13 @@ enum push_default_type { PUSH_DEFAULT_UNSPECIFIED }; +enum rebase_setup_type { + AUTOREBASE_NEVER = 0, + AUTOREBASE_LOCAL, + AUTOREBASE_REMOTE, + AUTOREBASE_ALWAYS +}; + struct repo_config_values { /* section "core" config values */ char *attributes_file; @@ -112,6 +119,7 @@ struct repo_config_values { char *apply_default_whitespace; char *apply_default_ignorewhitespace; enum push_default_type push_default; + enum rebase_setup_type autorebase; int apply_sparse_checkout; int trust_ctime; int check_stat; @@ -205,14 +213,6 @@ extern unsigned long pack_size_limit_cfg; extern int protect_hfs; extern int protect_ntfs; -enum rebase_setup_type { - AUTOREBASE_NEVER = 0, - AUTOREBASE_LOCAL, - AUTOREBASE_REMOTE, - AUTOREBASE_ALWAYS -}; -extern enum rebase_setup_type autorebase; - enum object_creation_mode { OBJECT_CREATION_USES_HARDLINKS = 0, OBJECT_CREATION_USES_RENAMES = 1 From b5efc34b10b3484e18e7a55260f0f06e423eba28 Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Tue, 14 Jul 2026 11:25:24 +0800 Subject: [PATCH 10/47] environment: move object_creation_mode into repo_config_values The global variable 'object_creation_mode' controls how Git creates object files, specifically determining whether to use hardlinks or renames when moving temporary files into the object database. Move it into 'struct repo_config_values' to continue the libification effort. Move the 'enum object_creation_mode' definition higher up in 'environment.h' to ensure it is visible to the structure. Initialize the per-repository value to its default macro value OBJECT_CREATION_MODE inside 'repo_config_values_init()'. Update configuration parsing in 'git_default_core_config()' to write directly to the repository-specific configuration structure. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- environment.c | 6 +++--- environment.h | 12 ++++++------ object-file.c | 3 ++- 3 files changed, 11 insertions(+), 10 deletions(-) diff --git a/environment.c b/environment.c index c0bf7577b7ad40..ef3c032e0c0329 100644 --- a/environment.c +++ b/environment.c @@ -60,7 +60,6 @@ char *check_roundtrip_encoding; #ifndef OBJECT_CREATION_MODE #define OBJECT_CREATION_MODE OBJECT_CREATION_USES_HARDLINKS #endif -enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE; int grafts_keep_true_parents; unsigned long pack_size_limit_cfg; @@ -512,9 +511,9 @@ int git_default_core_config(const char *var, const char *value, if (!value) return config_error_nonbool(var); if (!strcmp(value, "rename")) - object_creation_mode = OBJECT_CREATION_USES_RENAMES; + cfg->object_creation_mode = OBJECT_CREATION_USES_RENAMES; else if (!strcmp(value, "link")) - object_creation_mode = OBJECT_CREATION_USES_HARDLINKS; + cfg->object_creation_mode = OBJECT_CREATION_USES_HARDLINKS; else die(_("invalid mode for object creation: %s"), value); return 0; @@ -728,6 +727,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->apply_default_ignorewhitespace = NULL; cfg->push_default = PUSH_DEFAULT_UNSPECIFIED; cfg->autorebase = AUTOREBASE_NEVER; + cfg->object_creation_mode = OBJECT_CREATION_MODE; cfg->apply_sparse_checkout = 0; cfg->branch_track = BRANCH_TRACK_REMOTE; cfg->trust_ctime = 1; diff --git a/environment.h b/environment.h index 46b2f0d861089e..a47a5c83dbd469 100644 --- a/environment.h +++ b/environment.h @@ -109,6 +109,11 @@ enum rebase_setup_type { AUTOREBASE_ALWAYS }; +enum object_creation_mode { + OBJECT_CREATION_USES_HARDLINKS = 0, + OBJECT_CREATION_USES_RENAMES = 1 +}; + struct repo_config_values { /* section "core" config values */ char *attributes_file; @@ -120,6 +125,7 @@ struct repo_config_values { char *apply_default_ignorewhitespace; enum push_default_type push_default; enum rebase_setup_type autorebase; + enum object_creation_mode object_creation_mode; int apply_sparse_checkout; int trust_ctime; int check_stat; @@ -213,12 +219,6 @@ extern unsigned long pack_size_limit_cfg; extern int protect_hfs; extern int protect_ntfs; -enum object_creation_mode { - OBJECT_CREATION_USES_HARDLINKS = 0, - OBJECT_CREATION_USES_RENAMES = 1 -}; -extern enum object_creation_mode object_creation_mode; - extern int grafts_keep_true_parents; const char *get_log_output_encoding(void); diff --git a/object-file.c b/object-file.c index e3d92bbda25f0a..26780904681a22 100644 --- a/object-file.c +++ b/object-file.c @@ -411,11 +411,12 @@ int finalize_object_file_flags(struct repository *repo, { unsigned retries = 0; int ret; + struct repo_config_values *cfg = repo_config_values(repo); retry: ret = 0; - if (object_creation_mode == OBJECT_CREATION_USES_RENAMES) + if (cfg->object_creation_mode == OBJECT_CREATION_USES_RENAMES) goto try_rename; else if (link(tmpfile, filename)) ret = errno; From 5eac7ac7151a27025386d48e21c4608115a16db6 Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Tue, 14 Jul 2026 11:25:25 +0800 Subject: [PATCH 11/47] repository: adjust the comment of config_values_private_ The configurations in 'struct config_values_private_' are not all parsed in 'git_default_config()'. For example, 'pager_program' is now parsed in 'pager.c'. Therefore, update the comment. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- repository.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repository.h b/repository.h index 36e2db26332c0e..9093e6af93db96 100644 --- a/repository.h +++ b/repository.h @@ -152,7 +152,7 @@ struct repository { /* Repository's compatibility hash algorithm. */ const struct git_hash_algo *compat_hash_algo; - /* Repository's config values parsed by git_default_config() */ + /* Repository-specific configuration values. */ struct repo_config_values config_values_private_; /* Repository's reference storage format, as serialized on disk. */ From 8aad1dfc006e129d3276ae6dd5cc2ba7ca2a8f4c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ren=C3=A9=20Scharfe?= Date: Tue, 14 Jul 2026 19:59:52 +0200 Subject: [PATCH 12/47] tempfile: add repo_create_tempfile{,_mode}() MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add variants of create_tempfile_mode() that handle arbitrary repositories. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano --- tempfile.c | 8 +++++++- tempfile.h | 11 +++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/tempfile.c b/tempfile.c index f0fdf582794ba5..3132eb43710036 100644 --- a/tempfile.c +++ b/tempfile.c @@ -135,6 +135,12 @@ static void deactivate_tempfile(struct tempfile *tempfile) /* Make sure errno contains a meaningful value on error */ struct tempfile *create_tempfile_mode(const char *path, int mode) +{ + return repo_create_tempfile_mode(the_repository, path, mode); +} + +struct tempfile *repo_create_tempfile_mode(struct repository *r, + const char *path, int mode) { struct tempfile *tempfile = new_tempfile(); @@ -150,7 +156,7 @@ struct tempfile *create_tempfile_mode(const char *path, int mode) return NULL; } activate_tempfile(tempfile); - if (adjust_shared_perm(the_repository, tempfile->filename.buf)) { + if (adjust_shared_perm(r, tempfile->filename.buf)) { int save_errno = errno; error("cannot fix permission bits on %s", tempfile->filename.buf); delete_tempfile(&tempfile); diff --git a/tempfile.h b/tempfile.h index 2227a095fd4289..2d17e4dad3faa5 100644 --- a/tempfile.h +++ b/tempfile.h @@ -4,6 +4,8 @@ #include "list.h" #include "strbuf.h" +struct repository; + /* * Handle temporary files. * @@ -94,11 +96,20 @@ struct tempfile { */ struct tempfile *create_tempfile_mode(const char *path, int mode); +struct tempfile *repo_create_tempfile_mode(struct repository *r, + const char *path, int mode); + static inline struct tempfile *create_tempfile(const char *path) { return create_tempfile_mode(path, 0666); } +static inline struct tempfile *repo_create_tempfile(struct repository *r, + const char *path) +{ + return repo_create_tempfile_mode(r, path, 0666); +} + /* * Register an existing file as a tempfile, meaning that it will be * deleted when the program exits. The tempfile is considered closed, From 8432a4dee1c1206730c3e6bc5845c32272330bec Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ren=C3=A9=20Scharfe?= Date: Tue, 14 Jul 2026 19:59:53 +0200 Subject: [PATCH 13/47] refs/packed: use repo_create_tempfile() MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Apply the config setting core.sharedRepository from the ref store base repository at hand instead of from the_repository. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano --- refs/packed-backend.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 0acde48c452513..7c4e8aca873555 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -1379,7 +1379,7 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re packed_refs_path = get_locked_file_path(&refs->lock); strbuf_addf(&sb, "%s.new", packed_refs_path); free(packed_refs_path); - refs->tempfile = create_tempfile(sb.buf); + refs->tempfile = repo_create_tempfile(refs->base.repo, sb.buf); if (!refs->tempfile) { strbuf_addf(err, "unable to create file %s: %s", sb.buf, strerror(errno)); From d43f701d326ef3fe1bf04c1de24fa025ab228af8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ren=C3=A9=20Scharfe?= Date: Tue, 14 Jul 2026 19:59:54 +0200 Subject: [PATCH 14/47] lockfile: add repo_hold_lock_file_for_update{,_timeout}{,_mode}() MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add variants of hold_lock_file_for_update_timeout_mode() that handle arbitrary repositories. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano --- lockfile.c | 30 ++++++++++++++++++++++-------- lockfile.h | 31 +++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+), 8 deletions(-) diff --git a/lockfile.c b/lockfile.c index 7add2f136abd85..100f60377174ef 100644 --- a/lockfile.c +++ b/lockfile.c @@ -2,11 +2,14 @@ * Copyright (c) 2005, Junio C Hamano */ +#define USE_THE_REPOSITORY_VARIABLE + #include "git-compat-util.h" #include "abspath.h" #include "gettext.h" #include "lockfile.h" #include "parse.h" +#include "repository.h" #include "strbuf.h" #include "wrapper.h" @@ -162,8 +165,8 @@ static int read_lock_pid(const char *pid_path, uintmax_t *pid_out) } /* Make sure errno contains a meaningful value on error */ -static int lock_file(struct lock_file *lk, const char *path, int flags, - int mode) +static int lock_file(struct repository *r, struct lock_file *lk, + const char *path, int flags, int mode) { struct strbuf base_path = STRBUF_INIT; struct strbuf lock_path = STRBUF_INIT; @@ -176,7 +179,7 @@ static int lock_file(struct lock_file *lk, const char *path, int flags, get_lock_path(&lock_path, base_path.buf); get_pid_path(&pid_path, base_path.buf); - lk->tempfile = create_tempfile_mode(lock_path.buf, mode); + lk->tempfile = repo_create_tempfile_mode(r, lock_path.buf, mode); if (lk->tempfile) lk->pid_tempfile = create_lock_pid_file(pid_path.buf, mode); @@ -200,8 +203,9 @@ static int lock_file(struct lock_file *lk, const char *path, int flags, * timeout_ms milliseconds. If timeout_ms is 0, try locking the file * exactly once. If timeout_ms is -1, try indefinitely. */ -static int lock_file_timeout(struct lock_file *lk, const char *path, - int flags, long timeout_ms, int mode) +static int lock_file_timeout(struct repository *r, struct lock_file *lk, + const char *path, int flags, long timeout_ms, + int mode) { int n = 1; int multiplier = 1; @@ -209,7 +213,7 @@ static int lock_file_timeout(struct lock_file *lk, const char *path, static int random_initialized = 0; if (timeout_ms == 0) - return lock_file(lk, path, flags, mode); + return lock_file(r, lk, path, flags, mode); if (!random_initialized) { srand((unsigned int)getpid()); @@ -223,7 +227,7 @@ static int lock_file_timeout(struct lock_file *lk, const char *path, long backoff_ms, wait_ms; int fd; - fd = lock_file(lk, path, flags, mode); + fd = lock_file(r, lk, path, flags, mode); if (fd >= 0) return fd; /* success */ @@ -308,7 +312,17 @@ int hold_lock_file_for_update_timeout_mode(struct lock_file *lk, const char *path, int flags, long timeout_ms, int mode) { - int fd = lock_file_timeout(lk, path, flags, timeout_ms, mode); + return repo_hold_lock_file_for_update_timeout_mode(the_repository, + lk, path, flags, + timeout_ms, mode); +} + +int repo_hold_lock_file_for_update_timeout_mode(struct repository *r, + struct lock_file *lk, + const char *path, int flags, + long timeout_ms, int mode) +{ + int fd = lock_file_timeout(r, lk, path, flags, timeout_ms, mode); if (fd < 0) { if (flags & LOCK_DIE_ON_ERROR) unable_to_lock_die(path, errno); diff --git a/lockfile.h b/lockfile.h index e7233f28dea5c7..1667612674b52a 100644 --- a/lockfile.h +++ b/lockfile.h @@ -189,6 +189,11 @@ int hold_lock_file_for_update_timeout_mode( struct lock_file *lk, const char *path, int flags, long timeout_ms, int mode); +int repo_hold_lock_file_for_update_timeout_mode(struct repository *r, + struct lock_file *lk, + const char *path, int flags, + long timeout_ms, int mode); + static inline int hold_lock_file_for_update_timeout( struct lock_file *lk, const char *path, int flags, long timeout_ms) @@ -197,6 +202,16 @@ static inline int hold_lock_file_for_update_timeout( timeout_ms, 0666); } +static inline int repo_hold_lock_file_for_update_timeout(struct repository *r, + struct lock_file *lk, + const char *path, + int flags, + long timeout_ms) +{ + return repo_hold_lock_file_for_update_timeout_mode(r, lk, path, flags, + timeout_ms, 0666); +} + /* * Attempt to create a lockfile for the file at `path` and return a * file descriptor for writing to it, or -1 on error. The flags @@ -208,6 +223,13 @@ static inline int hold_lock_file_for_update( return hold_lock_file_for_update_timeout(lk, path, flags, 0); } +static inline int repo_hold_lock_file_for_update(struct repository *r, + struct lock_file *lk, + const char *path, int flags) +{ + return repo_hold_lock_file_for_update_timeout(r, lk, path, flags, 0); +} + static inline int hold_lock_file_for_update_mode( struct lock_file *lk, const char *path, int flags, int mode) @@ -215,6 +237,15 @@ static inline int hold_lock_file_for_update_mode( return hold_lock_file_for_update_timeout_mode(lk, path, flags, 0, mode); } +static inline int repo_hold_lock_file_for_update_mode(struct repository *r, + struct lock_file *lk, + const char *path, + int flags, int mode) +{ + return repo_hold_lock_file_for_update_timeout_mode(r, lk, path, flags, + 0, mode); +} + /* * Return a nonzero value iff `lk` is currently locked. */ From 75e05f6bc3b3e04301d25327f54b2c9998471c1d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ren=C3=A9=20Scharfe?= Date: Tue, 14 Jul 2026 19:59:55 +0200 Subject: [PATCH 15/47] tempfile: stop using the_repository MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove the compatibility wrappers create_tempfile_mode() and create_tempfile() that have become unused. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano --- tempfile.c | 7 ------- tempfile.h | 7 ------- 2 files changed, 14 deletions(-) diff --git a/tempfile.c b/tempfile.c index 3132eb43710036..dc9ca4e6459c64 100644 --- a/tempfile.c +++ b/tempfile.c @@ -42,8 +42,6 @@ * file created by its parent. */ -#define USE_THE_REPOSITORY_VARIABLE - #include "git-compat-util.h" #include "abspath.h" #include "path.h" @@ -134,11 +132,6 @@ static void deactivate_tempfile(struct tempfile *tempfile) } /* Make sure errno contains a meaningful value on error */ -struct tempfile *create_tempfile_mode(const char *path, int mode) -{ - return repo_create_tempfile_mode(the_repository, path, mode); -} - struct tempfile *repo_create_tempfile_mode(struct repository *r, const char *path, int mode) { diff --git a/tempfile.h b/tempfile.h index 2d17e4dad3faa5..f571f3c609c04a 100644 --- a/tempfile.h +++ b/tempfile.h @@ -94,16 +94,9 @@ struct tempfile { * `core.sharedRepository`, so it is not guaranteed to have the given * mode. */ -struct tempfile *create_tempfile_mode(const char *path, int mode); - struct tempfile *repo_create_tempfile_mode(struct repository *r, const char *path, int mode); -static inline struct tempfile *create_tempfile(const char *path) -{ - return create_tempfile_mode(path, 0666); -} - static inline struct tempfile *repo_create_tempfile(struct repository *r, const char *path) { From 2e486bfbf75c5098406a07e6967ecb80a28c316a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ren=C3=A9=20Scharfe?= Date: Tue, 14 Jul 2026 19:59:56 +0200 Subject: [PATCH 16/47] use repo_hold_lock_file_for_update{,_mode,_timeout}() with custom repos MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Apply the config setting core.sharedRepository from the repository at hand instead of from the_repository. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano --- apply.c | 10 ++++++---- builtin/difftool.c | 2 +- builtin/gc.c | 2 +- builtin/history.c | 2 +- builtin/sparse-checkout.c | 3 ++- bundle.c | 4 ++-- commit-graph.c | 9 +++++---- config.c | 4 ++-- loose.c | 6 ++++-- midx-write.c | 7 ++++--- odb/source-files.c | 3 ++- refs/files-backend.c | 10 ++++++---- refs/packed-backend.c | 7 +++---- refs/packed-backend.h | 2 +- repack-midx.c | 3 ++- repository.c | 2 +- rerere.c | 6 +++--- 17 files changed, 46 insertions(+), 36 deletions(-) diff --git a/apply.c b/apply.c index 5e54453f799ea2..ac1bfc7f85a5d5 100644 --- a/apply.c +++ b/apply.c @@ -4287,7 +4287,8 @@ static int build_fake_ancestor(struct apply_state *state, struct patch *list) } } - hold_lock_file_for_update(&lock, state->fake_ancestor, LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(state->repo, &lock, state->fake_ancestor, + LOCK_DIE_ON_ERROR); res = write_locked_index(&result, &lock, COMMIT_LOCK); discard_index(&result); @@ -4945,9 +4946,10 @@ static int apply_patch(struct apply_state *state, state->update_index = (state->check_index || state->ita_only) && state->apply; if (state->update_index && !is_lock_file_locked(&state->lock_file)) { if (state->index_file) - hold_lock_file_for_update(&state->lock_file, - state->index_file, - LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(state->repo, + &state->lock_file, + state->index_file, + LOCK_DIE_ON_ERROR); else repo_hold_locked_index(state->repo, &state->lock_file, LOCK_DIE_ON_ERROR); diff --git a/builtin/difftool.c b/builtin/difftool.c index 26778f8515deef..99c01c92ef212a 100644 --- a/builtin/difftool.c +++ b/builtin/difftool.c @@ -636,7 +636,7 @@ static int run_dir_diff(struct repository *repo, struct lock_file lock = LOCK_INIT; strbuf_reset(&buf); strbuf_addf(&buf, "%s/wtindex", tmpdir.buf); - if (hold_lock_file_for_update(&lock, buf.buf, 0) < 0 || + if (repo_hold_lock_file_for_update(repo, &lock, buf.buf, 0) < 0 || write_locked_index(&wtindex, &lock, COMMIT_LOCK)) { ret = error("could not write %s", buf.buf); goto finish; diff --git a/builtin/gc.c b/builtin/gc.c index c26c93ee0fe4a3..01d034829c609b 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -1790,7 +1790,7 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts, struct repository *r = the_repository; char *lock_path = xstrfmt("%s/maintenance", r->objects->sources->path); - if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) { + if (repo_hold_lock_file_for_update(r, &lk, lock_path, LOCK_NO_DEREF) < 0) { /* * Another maintenance command is running. * diff --git a/builtin/history.c b/builtin/history.c index 091465a59e2f96..5b070fb3351408 100644 --- a/builtin/history.c +++ b/builtin/history.c @@ -764,7 +764,7 @@ static int write_ondisk_index(struct repository *repo, prime_cache_tree(repo, &index, tree); - if (hold_lock_file_for_update(&lock, path, 0) < 0) { + if (repo_hold_lock_file_for_update(repo, &lock, path, 0) < 0) { ret = error_errno(_("unable to acquire index lock")); goto out; } diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 0863d0fb460cf8..cb4a037b770291 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -341,7 +341,8 @@ static int write_patterns_and_update(struct repository *repo, if (safe_create_leading_directories(repo, sparse_filename)) die(_("failed to create directory for sparse-checkout file")); - hold_lock_file_for_update(&lk, sparse_filename, LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(repo, &lk, sparse_filename, + LOCK_DIE_ON_ERROR); result = update_working_directory(repo, pl); if (result) { diff --git a/bundle.c b/bundle.c index fd2db2c837df60..b64716f252b78f 100644 --- a/bundle.c +++ b/bundle.c @@ -519,8 +519,8 @@ int create_bundle(struct repository *r, const char *path, if (bundle_to_stdout) bundle_fd = 1; else - bundle_fd = hold_lock_file_for_update(&lock, path, - LOCK_DIE_ON_ERROR); + bundle_fd = repo_hold_lock_file_for_update(r, &lock, path, + LOCK_DIE_ON_ERROR); if (version == -1) version = min_version; diff --git a/commit-graph.c b/commit-graph.c index 801471a09802d9..063b545eb75c1e 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -2122,8 +2122,8 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx) if (ctx->split) { char *lock_name = get_commit_graph_chain_filename(ctx->odb_source); - hold_lock_file_for_update_mode(&lk, lock_name, - LOCK_DIE_ON_ERROR, 0444); + repo_hold_lock_file_for_update_mode(ctx->r, &lk, lock_name, + LOCK_DIE_ON_ERROR, 0444); free(lock_name); graph_layer = mks_tempfile_m(ctx->graph_name, 0444); @@ -2141,8 +2141,9 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx) f = hashfd(ctx->r->hash_algo, get_tempfile_fd(graph_layer), get_tempfile_path(graph_layer)); } else { - hold_lock_file_for_update_mode(&lk, ctx->graph_name, - LOCK_DIE_ON_ERROR, 0444); + repo_hold_lock_file_for_update_mode(ctx->r, &lk, + ctx->graph_name, + LOCK_DIE_ON_ERROR, 0444); f = hashfd(ctx->r->hash_algo, get_lock_file_fd(&lk), get_lock_file_path(&lk)); } diff --git a/config.c b/config.c index 6a0de86e3ae958..d29425ab8e8408 100644 --- a/config.c +++ b/config.c @@ -3034,7 +3034,7 @@ int repo_config_set_multivar_in_file_gently(struct repository *r, * The lock serves a purpose in addition to locking: the new * contents of .git/config will be written into it. */ - fd = hold_lock_file_for_update(&lock, config_filename, 0); + fd = repo_hold_lock_file_for_update(r, &lock, config_filename, 0); if (fd < 0) { error_errno(_("could not lock config file %s"), config_filename); ret = CONFIG_NO_LOCK; @@ -3379,7 +3379,7 @@ static int repo_config_copy_or_rename_section_in_file( if (!config_filename) config_filename = filename_buf = repo_git_path(r, "config"); - out_fd = hold_lock_file_for_update(&lock, config_filename, 0); + out_fd = repo_hold_lock_file_for_update(r, &lock, config_filename, 0); if (out_fd < 0) { ret = error(_("could not lock config file %s"), config_filename); goto out; diff --git a/loose.c b/loose.c index 0b626c1b854642..a79cafd38aae98 100644 --- a/loose.c +++ b/loose.c @@ -138,7 +138,8 @@ int repo_write_loose_object_map(struct repository *repo) return 0; repo_common_path_replace(repo, &path, "objects/loose-object-idx"); - fd = hold_lock_file_for_update_timeout(&lock, path.buf, LOCK_DIE_ON_ERROR, -1); + fd = repo_hold_lock_file_for_update_timeout(repo, &lock, path.buf, + LOCK_DIE_ON_ERROR, -1); iter = kh_begin(map); if (write_in_full(fd, loose_object_header, strlen(loose_object_header)) < 0) goto errout; @@ -180,7 +181,8 @@ static int write_one_object(struct odb_source_loose *loose, struct strbuf buf = STRBUF_INIT, path = STRBUF_INIT; strbuf_addf(&path, "%s/loose-object-idx", loose->base.path); - hold_lock_file_for_update_timeout(&lock, path.buf, LOCK_DIE_ON_ERROR, -1); + repo_hold_lock_file_for_update_timeout(loose->base.odb->repo, &lock, + path.buf, LOCK_DIE_ON_ERROR, -1); fd = open(path.buf, O_WRONLY | O_CREAT | O_APPEND, 0666); if (fd < 0) diff --git a/midx-write.c b/midx-write.c index 19e1cd10b7dee4..e0730d02545687 100644 --- a/midx-write.c +++ b/midx-write.c @@ -1627,8 +1627,8 @@ static int write_midx_internal(struct write_midx_opts *opts) struct strbuf lock_name = STRBUF_INIT; get_midx_chain_filename(opts->source, &lock_name); - hold_lock_file_for_update(&lk, lock_name.buf, - LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(r, &lk, lock_name.buf, + LOCK_DIE_ON_ERROR); strbuf_release(&lock_name); } @@ -1647,7 +1647,8 @@ static int write_midx_internal(struct write_midx_opts *opts) f = hashfd(r->hash_algo, get_tempfile_fd(incr), get_tempfile_path(incr)); } else { - hold_lock_file_for_update(&lk, midx_name.buf, LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(r, &lk, midx_name.buf, + LOCK_DIE_ON_ERROR); f = hashfd(r->hash_algo, get_lock_file_fd(&lk), get_lock_file_path(&lk)); } diff --git a/odb/source-files.c b/odb/source-files.c index 5bdd0429225397..3dd9e71e7341e5 100644 --- a/odb/source-files.c +++ b/odb/source-files.c @@ -217,7 +217,8 @@ static int odb_source_files_write_alternate(struct odb_source *source, int found = 0; int ret; - hold_lock_file_for_update(&lock, path, LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(source->odb->repo, &lock, path, + LOCK_DIE_ON_ERROR); out = fdopen_lock_file(&lock, "w"); if (!out) { ret = error_errno(_("unable to fdopen alternates lockfile")); diff --git a/refs/files-backend.c b/refs/files-backend.c index a4c7858787127d..2e82258844e872 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -791,7 +791,7 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs, goto error_return; } - if (hold_lock_file_for_update_timeout( + if (repo_hold_lock_file_for_update_timeout(refs->base.repo, &lock->lk, ref_file.buf, LOCK_NO_DEREF, get_files_ref_lock_timeout_ms(transaction->ref_store->repo)) < 0) { int myerr = errno; @@ -1199,8 +1199,8 @@ struct create_reflock_cb { static int create_reflock(const char *path, void *cb) { struct create_reflock_cb *data = cb; - return hold_lock_file_for_update_timeout( - data->lk, path, LOCK_NO_DEREF, + return repo_hold_lock_file_for_update_timeout( + data->repo, data->lk, path, LOCK_NO_DEREF, get_files_ref_lock_timeout_ms(data->repo)) < 0 ? -1 : 0; } @@ -3529,7 +3529,9 @@ static int files_reflog_expire(struct ref_store *ref_store, * work we need, including cleaning up if the program * exits unexpectedly. */ - if (hold_lock_file_for_update(&reflog_lock, log_file, 0) < 0) { + if (repo_hold_lock_file_for_update(ref_store->repo, + &reflog_lock, log_file, + 0) < 0) { struct strbuf err = STRBUF_INIT; unable_to_lock_message(log_file, errno, &err); error("%s", err.buf); diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 7c4e8aca873555..117f2d43681a6b 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -1232,10 +1232,9 @@ int packed_refs_lock(struct ref_store *ref_store, int flags, struct strbuf *err) * don't write new content to it, but rather to a separate * tempfile. */ - if (hold_lock_file_for_update_timeout( - &refs->lock, - refs->path, - flags, timeout_value) < 0) { + if (repo_hold_lock_file_for_update_timeout(ref_store->repo, &refs->lock, + refs->path, flags, + timeout_value) < 0) { unable_to_lock_message(refs->path, errno, err); return -1; } diff --git a/refs/packed-backend.h b/refs/packed-backend.h index 1db48e801d63d0..8a7b3238254d3d 100644 --- a/refs/packed-backend.h +++ b/refs/packed-backend.h @@ -21,7 +21,7 @@ struct ref_store *packed_ref_store_init(struct repository *repo, /* * Lock the packed-refs file for writing. Flags is passed to - * hold_lock_file_for_update(). Return 0 on success. On errors, write + * repo_hold_lock_file_for_update(). Return 0 on success. On errors, write * an error message to `err` and return a nonzero value. */ int packed_refs_lock(struct ref_store *ref_store, int flags, struct strbuf *err); diff --git a/repack-midx.c b/repack-midx.c index b6b1de718058da..78dbed3df6a71e 100644 --- a/repack-midx.c +++ b/repack-midx.c @@ -951,7 +951,8 @@ static int write_midx_incremental(struct repack_write_midx_opts *opts) lock_name.buf)) die_errno(_("unable to create leading directories of %s"), lock_name.buf); - hold_lock_file_for_update(&lf, lock_name.buf, LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(opts->existing->repo, &lf, lock_name.buf, + LOCK_DIE_ON_ERROR); if (!fdopen_lock_file(&lf, "w")) { ret = error_errno(_("unable to open multi-pack-index chain file")); diff --git a/repository.c b/repository.c index 187dd471c4e607..25583073085f61 100644 --- a/repository.c +++ b/repository.c @@ -466,5 +466,5 @@ int repo_hold_locked_index(struct repository *repo, { if (!repo->index_file) BUG("the repo hasn't been setup"); - return hold_lock_file_for_update(lf, repo->index_file, flags); + return repo_hold_lock_file_for_update(repo, lf, repo->index_file, flags); } diff --git a/rerere.c b/rerere.c index 8232542585cad4..2d1e99ec114294 100644 --- a/rerere.c +++ b/rerere.c @@ -911,9 +911,9 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & RERERE_READONLY) fd = 0; else - fd = hold_lock_file_for_update(&write_lock, - git_path_merge_rr(r), - LOCK_DIE_ON_ERROR); + fd = repo_hold_lock_file_for_update(r, &write_lock, + git_path_merge_rr(r), + LOCK_DIE_ON_ERROR); read_rr(r, merge_rr); return fd; } From fda513d6fea267f2b59a1c5a212d98e4b6ae4187 Mon Sep 17 00:00:00 2001 From: Travor Liu Date: Fri, 17 Jul 2026 21:52:45 +0800 Subject: [PATCH 17/47] gitweb: shorten index hashes with trailing file modes Diff index lines have included a trailing file mode since ec1fcc16af (Show original and resulting blob object info in diff output, 2005-10-07) when the old and new file modes match: index .. 100644 gitweb recognizes that trailing mode before it tries to shorten and link the object IDs. This appends the file-type annotation first, but the object-ID matcher requires the ID range to end the line. As a result, this common form keeps both full object IDs as plain text. That is inconsistent with other hash displays and makes commitdiff output wider than necessary. Recent gitweb changes have fixed mobile overflow in log, commit, blob and diff views; leaving two full object IDs in this header preserves an avoidable long line in the diff header. * gitweb/gitweb.perl: Remove the trailing mode before matching the index IDs, then append it again after the IDs have been shortened and linked. This preserves the mode display while letting ordinary and combined index lines use the existing object-ID formatting paths. * t/t9502-gitweb-standalone-parse-output.sh: Add coverage for that common form by rendering a commitdiff for a regular file modification. Check that the visible index line contains linked short blob IDs followed by the mode and file-type annotation, and that the full unlinked form is not emitted. Signed-off-by: Travor Liu Signed-off-by: Junio C Hamano --- gitweb/gitweb.perl | 18 +++++++++++++----- t/t9502-gitweb-standalone-parse-output.sh | 13 +++++++++++++ 2 files changed, 26 insertions(+), 5 deletions(-) diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl index fde804593b6ff3..8c2d9b8eefe4db 100755 --- a/gitweb/gitweb.perl +++ b/gitweb/gitweb.perl @@ -2339,12 +2339,14 @@ sub format_extended_diff_header_line { $line .= $cgi->a({-href=>$to->{'href'}, -class=>"path"}, esc_path($to->{'file'})); } - # match single - if ($line =~ m/\s(\d{6})$/) { - $line .= ' (' . - file_type_long($1) . - ')'; + + # Temporarily remove a trailing so an index line ends with its + # object IDs and can be shortened below. + my $mode; + if ($line =~ s/\s(\d{6})$//) { + $mode = $1; } + # match if ($line =~ oid_nlen_prefix_infix_regex($sha1_len, "index ", ",") | $line =~ oid_nlen_prefix_infix_regex($sha256_len, "index ", ",")) { @@ -2388,6 +2390,12 @@ sub format_extended_diff_header_line { my ($from_id, $to_id) = ($diffinfo->{'from_id'}, $diffinfo->{'to_id'}); $line =~ s!$from_id\.\.$to_id!$from_link..$to_link!; } + if (defined $mode) { + $line .= " $mode" . + ' (' . + file_type_long($mode) . + ')'; + } return $line . "
\n"; } diff --git a/t/t9502-gitweb-standalone-parse-output.sh b/t/t9502-gitweb-standalone-parse-output.sh index 81d562555799be..85f771650de034 100755 --- a/t/t9502-gitweb-standalone-parse-output.sh +++ b/t/t9502-gitweb-standalone-parse-output.sh @@ -115,6 +115,19 @@ test_expect_success 'snapshot: hierarchical branch name (xx/test)' ' ' test_debug 'cat gitweb.headers' +test_expect_success 'commitdiff: index line shortens hashes with mode' ' + old_blob=$(git rev-parse HEAD:foo) && + old_short=$(git rev-parse --short=7 HEAD:foo) && + echo changed >foo && + git commit -am "change foo" && + new_blob=$(git rev-parse HEAD:foo) && + new_short=$(git rev-parse --short=7 HEAD:foo) && + gitweb_run "p=.git;a=commitdiff;h=HEAD" && + test_grep ">${old_short}\\.\\.]*>${new_short} 100644 (file)" \ + gitweb.body && + test_grep ! "index ${old_blob}\\.\\.${new_blob} 100644" gitweb.body +' + # ---------------------------------------------------------------------- # forks of projects From 215d305f450ff0691d15daaa6ef72f77e8441b39 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 17 Jul 2026 11:32:09 +0200 Subject: [PATCH 18/47] odb: compute compat object ID in `odb_write_object_ext()` Repositories can have a compatibility hash configured, which means that such a repository is expected to maintain a mapping between canonical and compatibility object hashes. Maintaining this mapping is the responsibility of the object database sources, where we either store them as part of the loose objects map or in packfile indices v3 (once we gain support for this feature). But besides storing these compatibility hashes, the sources are also responsible for generating the compatibility hash in the first place. This is somewhat unnecessary though, as the compatibility hash should be computed the same no matter which source is being used. The consequence is that we need to duplicate this functionality across the different backends, which does not make a lot of sense. Refactor the code so that we instead compute the compatibility hash in `odb_write_object_ext()` and then pass the computed value to the sources. No callers need adjustment as there are none that write objects via the source interfaces directly. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- odb.c | 26 ++++++++++++++++++++++++-- odb.h | 10 ++++++---- odb/source-files.c | 2 +- odb/source-inmemory.c | 2 +- odb/source-loose.c | 24 +++--------------------- odb/source-packed.c | 2 +- odb/source.h | 4 ++-- 7 files changed, 38 insertions(+), 32 deletions(-) diff --git a/odb.c b/odb.c index cf6e7938c01e56..1d6538163b54ea 100644 --- a/odb.c +++ b/odb.c @@ -989,11 +989,33 @@ int odb_write_object_ext(struct object_database *odb, const void *buf, unsigned long len, enum object_type type, struct object_id *oid, - struct object_id *compat_oid, + const struct object_id *compat_oid_in, enum odb_write_object_flags flags) { + const struct git_hash_algo *compat = odb->repo->compat_hash_algo; + struct object_id compat_oid, *compat_oid_p = NULL; + + if (compat) { + const struct git_hash_algo *algo = odb->repo->hash_algo; + + if (compat_oid_in) { + oidcpy(&compat_oid, compat_oid_in); + } else if (type == OBJ_BLOB) { + hash_object_file(compat, buf, len, type, &compat_oid); + } else { + struct strbuf converted = STRBUF_INIT; + convert_object_file(odb->repo, &converted, algo, compat, + buf, len, type, 0); + hash_object_file(compat, converted.buf, converted.len, + type, &compat_oid); + strbuf_release(&converted); + } + + compat_oid_p = &compat_oid; + } + return odb_source_write_object(odb->sources, buf, len, type, - oid, compat_oid, flags); + oid, compat_oid_p, flags); } int odb_write_object_stream(struct object_database *odb, diff --git a/odb.h b/odb.h index 94754643d24997..066560113ebc78 100644 --- a/odb.h +++ b/odb.h @@ -585,9 +585,11 @@ enum odb_write_object_flags { /* * Write an object into the object database. The object is being written into - * the local alternate of the repository. If provided, the converted object ID - * as well as the compatibility object ID are written to the respective - * pointers. + * the local alternate of the repository. If provided, the object ID of the + * final object is written into `oid`. + * + * If the caller provides a `compat_oid`, then this compatibility object hash + * will be stored instead of computing the compatibility hash ad-hoc. * * Returns 0 on success, a negative error code otherwise. */ @@ -595,7 +597,7 @@ int odb_write_object_ext(struct object_database *odb, const void *buf, unsigned long len, enum object_type type, struct object_id *oid, - struct object_id *compat_oid, + const struct object_id *compat_oid, enum odb_write_object_flags flags); static inline int odb_write_object(struct object_database *odb, diff --git a/odb/source-files.c b/odb/source-files.c index 413875851135a5..3d9f5eca327948 100644 --- a/odb/source-files.c +++ b/odb/source-files.c @@ -163,7 +163,7 @@ static int odb_source_files_write_object(struct odb_source *source, const void *buf, size_t len, enum object_type type, struct object_id *oid, - struct object_id *compat_oid, + const struct object_id *compat_oid, enum odb_write_object_flags flags) { struct odb_source_files *files = odb_source_files_downcast(source); diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c index e47bfd8fccabf0..e727aba427a17d 100644 --- a/odb/source-inmemory.c +++ b/odb/source-inmemory.c @@ -231,7 +231,7 @@ static int odb_source_inmemory_write_object(struct odb_source *source, const void *buf, size_t len, enum object_type type, struct object_id *oid, - struct object_id *compat_oid UNUSED, + const struct object_id *compat_oid UNUSED, enum odb_write_object_flags flags UNUSED) { struct odb_source_inmemory *inmemory = odb_source_inmemory_downcast(source); diff --git a/odb/source-loose.c b/odb/source-loose.c index 3f7d04a56e36ce..ca223109cd46e4 100644 --- a/odb/source-loose.c +++ b/odb/source-loose.c @@ -585,32 +585,14 @@ static int odb_source_loose_freshen_object(struct odb_source *source, static int odb_source_loose_write_object(struct odb_source *source, const void *buf, size_t len, enum object_type type, struct object_id *oid, - struct object_id *compat_oid_in, + const struct object_id *compat_oid, enum odb_write_object_flags flags) { struct odb_source_loose *loose = odb_source_loose_downcast(source); const struct git_hash_algo *algo = source->odb->repo->hash_algo; - const struct git_hash_algo *compat = source->odb->repo->compat_hash_algo; - struct object_id compat_oid; char hdr[MAX_HEADER_LEN]; size_t hdrlen = sizeof(hdr); - /* Generate compat_oid */ - if (compat) { - if (compat_oid_in) - oidcpy(&compat_oid, compat_oid_in); - else if (type == OBJ_BLOB) - hash_object_file(compat, buf, len, type, &compat_oid); - else { - struct strbuf converted = STRBUF_INIT; - convert_object_file(source->odb->repo, &converted, algo, compat, - buf, len, type, 0); - hash_object_file(compat, converted.buf, converted.len, - type, &compat_oid); - strbuf_release(&converted); - } - } - /* Normally if we have it in the pack then we do not bother writing * it out into .git/objects/??/?{38} file. */ @@ -619,8 +601,8 @@ static int odb_source_loose_write_object(struct odb_source *source, return 0; if (write_loose_object(loose, oid, hdr, hdrlen, buf, len, 0, flags)) return -1; - if (compat) - return repo_add_loose_object_map(loose, oid, &compat_oid); + if (compat_oid) + return repo_add_loose_object_map(loose, oid, compat_oid); return 0; } diff --git a/odb/source-packed.c b/odb/source-packed.c index 8d9ce197cc2be6..af0d533375176f 100644 --- a/odb/source-packed.c +++ b/odb/source-packed.c @@ -530,7 +530,7 @@ static int odb_source_packed_write_object(struct odb_source *source UNUSED, size_t len UNUSED, enum object_type type UNUSED, struct object_id *oid UNUSED, - struct object_id *compat_oid UNUSED, + const struct object_id *compat_oid UNUSED, unsigned flags UNUSED) { return error("packed backend cannot write objects"); diff --git a/odb/source.h b/odb/source.h index cd63dba91f4e2f..b3c1ca3a6604fc 100644 --- a/odb/source.h +++ b/odb/source.h @@ -207,7 +207,7 @@ struct odb_source { const void *buf, size_t len, enum object_type type, struct object_id *oid, - struct object_id *compat_oid, + const struct object_id *compat_oid, enum odb_write_object_flags flags); /* @@ -417,7 +417,7 @@ static inline int odb_source_write_object(struct odb_source *source, const void *buf, unsigned long len, enum object_type type, struct object_id *oid, - struct object_id *compat_oid, + const struct object_id *compat_oid, enum odb_write_object_flags flags) { return source->write_object(source, buf, len, type, oid, From cb5192ea9ad42f6f7aa539496a5c59d76b8eaf0f Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 17 Jul 2026 11:32:10 +0200 Subject: [PATCH 19/47] t/u-odb-inmemory: implement wrapper for writing objects In the next commit we're about to change how objects are being written into the object database source. Prepare for this refactoring by introducing a wrapper function into our unit tests so that we don't have to adjust all callsites. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- t/unit-tests/u-odb-inmemory.c | 48 +++++++++++++++++------------------ 1 file changed, 23 insertions(+), 25 deletions(-) diff --git a/t/unit-tests/u-odb-inmemory.c b/t/unit-tests/u-odb-inmemory.c index 6844bfc37ccfdc..2dbc3ab1df689a 100644 --- a/t/unit-tests/u-odb-inmemory.c +++ b/t/unit-tests/u-odb-inmemory.c @@ -1,5 +1,6 @@ #include "unit-test.h" #include "hex.h" +#include "object-file.h" #include "odb/source-inmemory.h" #include "odb/streaming.h" #include "oidset.h" @@ -36,6 +37,16 @@ static void cl_assert_object_info(struct odb_source_inmemory *source, free(actual_content); } +static void cl_assert_write_object(struct odb_source_inmemory *source, + const char *content, + enum object_type type, + struct object_id *oid) +{ + size_t content_len = strlen(content); + cl_must_pass(odb_source_write_object(&source->base, content, content_len, + type, oid, NULL, 0)); +} + void test_odb_inmemory__initialize(void) { odb = odb_new(&repo, "", ""); @@ -78,8 +89,7 @@ void test_odb_inmemory__read_written_object(void) const char data[] = "foobar"; struct object_id written_oid; - cl_must_pass(odb_source_write_object(&source->base, data, strlen(data), - OBJ_BLOB, &written_oid, NULL, 0)); + cl_assert_write_object(source, data, OBJ_BLOB, &written_oid); cl_assert_equal_s(oid_to_hex(&written_oid), FOOBAR_OID); cl_assert_object_info(source, &written_oid, OBJ_BLOB, "foobar"); @@ -94,8 +104,7 @@ void test_odb_inmemory__read_stream_object(void) const char data[] = "foobar"; char buf[3] = { 0 }; - cl_must_pass(odb_source_write_object(&source->base, data, strlen(data), - OBJ_BLOB, &written_oid, NULL, 0)); + cl_assert_write_object(source, data, OBJ_BLOB, &written_oid); cl_must_pass(odb_source_read_object_stream(&stream, &source->base, &written_oid)); @@ -141,8 +150,7 @@ void test_odb_inmemory__for_each_object(void) strbuf_reset(&buf); strbuf_addf(&buf, "%d", i); - cl_must_pass(odb_source_write_object(&source->base, buf.buf, buf.len, - OBJ_BLOB, &written_oid, NULL, 0)); + cl_assert_write_object(source, buf.buf, OBJ_BLOB, &written_oid); cl_must_pass(oidset_insert(&expected_oids, &written_oid)); } @@ -174,12 +182,9 @@ void test_odb_inmemory__for_each_object_can_abort_iteration(void) struct object_id written_oid; unsigned counter = 0; - cl_must_pass(odb_source_write_object(&source->base, "1", 1, - OBJ_BLOB, &written_oid, NULL, 0)); - cl_must_pass(odb_source_write_object(&source->base, "2", 1, - OBJ_BLOB, &written_oid, NULL, 0)); - cl_must_pass(odb_source_write_object(&source->base, "3", 1, - OBJ_BLOB, &written_oid, NULL, 0)); + cl_assert_write_object(source, "1", OBJ_BLOB, &written_oid); + cl_assert_write_object(source, "2", OBJ_BLOB, &written_oid); + cl_assert_write_object(source, "3", OBJ_BLOB, &written_oid); cl_assert_equal_i(odb_source_for_each_object(&source->base, NULL, abort_after_two_objects, @@ -199,12 +204,9 @@ void test_odb_inmemory__count_objects(void) cl_must_pass(odb_source_count_objects(&source->base, 0, &count)); cl_assert_equal_u(count, 0); - cl_must_pass(odb_source_write_object(&source->base, "1", 1, - OBJ_BLOB, &written_oid, NULL, 0)); - cl_must_pass(odb_source_write_object(&source->base, "2", 1, - OBJ_BLOB, &written_oid, NULL, 0)); - cl_must_pass(odb_source_write_object(&source->base, "3", 1, - OBJ_BLOB, &written_oid, NULL, 0)); + cl_assert_write_object(source, "1", OBJ_BLOB, &written_oid); + cl_assert_write_object(source, "2", OBJ_BLOB, &written_oid); + cl_assert_write_object(source, "3", OBJ_BLOB, &written_oid); cl_must_pass(odb_source_count_objects(&source->base, 0, &count)); cl_assert_equal_u(count, 3); @@ -228,8 +230,7 @@ void test_odb_inmemory__find_abbrev_len(void) * * With only one blob written we expect a length of 4. */ - cl_must_pass(odb_source_write_object(&source->base, "368317", strlen("368317"), - OBJ_BLOB, &oid1, NULL, 0)); + cl_assert_write_object(source, "368317", OBJ_BLOB, &oid1); cl_must_pass(odb_source_find_abbrev_len(&source->base, &oid1, 4, &abbrev_len)); cl_assert_equal_u(abbrev_len, 4); @@ -238,8 +239,7 @@ void test_odb_inmemory__find_abbrev_len(void) * With both objects present, the shared 10-character prefix means we * need at least 11 characters to uniquely identify either object. */ - cl_must_pass(odb_source_write_object(&source->base, "514796", strlen("514796"), - OBJ_BLOB, &oid2, NULL, 0)); + cl_assert_write_object(source, "514796", OBJ_BLOB, &oid2); cl_must_pass(odb_source_find_abbrev_len(&source->base, &oid1, 4, &abbrev_len)); cl_assert_equal_u(abbrev_len, 11); @@ -257,9 +257,7 @@ void test_odb_inmemory__freshen_object(void) cl_must_pass(parse_oid_hex_algop(RANDOM_OID, &oid, &end, repo.hash_algo)); cl_assert_equal_i(odb_source_freshen_object(&source->base, &oid), 0); - cl_must_pass(odb_source_write_object(&source->base, "foobar", - strlen("foobar"), OBJ_BLOB, - &written_oid, NULL, 0)); + cl_assert_write_object(source, "foobar", OBJ_BLOB, &written_oid); cl_assert_equal_i(odb_source_freshen_object(&source->base, &written_oid), 1); From 46a586a7199a78d2280ddb22368378693541fff0 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 17 Jul 2026 11:32:11 +0200 Subject: [PATCH 20/47] odb: compute object hash in `odb_write_object_ext()` Same as in a preceding commit, compute the object hash in `odb_write_object_ext()` so that we can unify this logic. Besides unification, this change also allows us to lift the object existence check out of the "loose" backend into the generic layer, which will happen in the next commit. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- object-file.c | 35 ++++++++--------------------------- object-file.h | 4 ---- odb.c | 2 ++ odb/source-files.c | 2 +- odb/source-inmemory.c | 6 +++--- odb/source-loose.c | 12 +++++++----- odb/source-packed.c | 2 +- odb/source.h | 4 ++-- t/unit-tests/u-odb-inmemory.c | 1 + 9 files changed, 25 insertions(+), 43 deletions(-) diff --git a/object-file.c b/object-file.c index 5283292f1ea02d..9ca14f484dbd5d 100644 --- a/object-file.c +++ b/object-file.c @@ -316,31 +316,6 @@ int parse_loose_header(const char *hdr, struct object_info *oi) return 0; } -static void hash_object_body(const struct git_hash_algo *algo, struct git_hash_ctx *c, - const void *buf, size_t len, - struct object_id *oid, - char *hdr, size_t *hdrlen) -{ - git_hash_init(c, algo); - git_hash_update(c, hdr, *hdrlen); - git_hash_update(c, buf, len); - git_hash_final_oid(oid, c); -} - -void write_object_file_prepare(const struct git_hash_algo *algo, - const void *buf, size_t len, - enum object_type type, struct object_id *oid, - char *hdr, size_t *hdrlen) -{ - struct git_hash_ctx c; - - /* Generate the header */ - *hdrlen = format_object_header(hdr, *hdrlen, type, len); - - /* Hash (function pointers) computation */ - hash_object_body(algo, &c, buf, len, oid, hdr, hdrlen); -} - #define CHECK_COLLISION_DEST_VANISHED -2 static int check_collision(const char *source, const char *dest) @@ -476,10 +451,16 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf, size_t len, enum object_type type, struct object_id *oid) { + struct git_hash_ctx c; char hdr[MAX_HEADER_LEN]; - size_t hdrlen = sizeof(hdr); + int hdrlen; + + hdrlen = format_object_header(hdr, sizeof(hdr), type, len); - write_object_file_prepare(algo, buf, len, type, oid, hdr, &hdrlen); + git_hash_init(&c, algo); + git_hash_update(&c, hdr, hdrlen); + git_hash_update(&c, buf, len); + git_hash_final_oid(oid, &c); } struct transaction_packfile { diff --git a/object-file.h b/object-file.h index d04ffa6493ff17..08aafcda0df67b 100644 --- a/object-file.h +++ b/object-file.h @@ -134,10 +134,6 @@ int finalize_object_file_flags(struct repository *repo, void hash_object_file(const struct git_hash_algo *algo, const void *buf, size_t len, enum object_type type, struct object_id *oid); -void write_object_file_prepare(const struct git_hash_algo *algo, - const void *buf, size_t len, - enum object_type type, struct object_id *oid, - char *hdr, size_t *hdrlen); int write_loose_object(struct odb_source_loose *loose, const struct object_id *oid, char *hdr, int hdrlen, const void *buf, unsigned long len, diff --git a/odb.c b/odb.c index 1d6538163b54ea..4adbdf8a648982 100644 --- a/odb.c +++ b/odb.c @@ -995,6 +995,8 @@ int odb_write_object_ext(struct object_database *odb, const struct git_hash_algo *compat = odb->repo->compat_hash_algo; struct object_id compat_oid, *compat_oid_p = NULL; + hash_object_file(odb->repo->hash_algo, buf, len, type, oid); + if (compat) { const struct git_hash_algo *algo = odb->repo->hash_algo; diff --git a/odb/source-files.c b/odb/source-files.c index 3d9f5eca327948..06dfc8dd78aecb 100644 --- a/odb/source-files.c +++ b/odb/source-files.c @@ -162,7 +162,7 @@ static int odb_source_files_freshen_object(struct odb_source *source, static int odb_source_files_write_object(struct odb_source *source, const void *buf, size_t len, enum object_type type, - struct object_id *oid, + const struct object_id *oid, const struct object_id *compat_oid, enum odb_write_object_flags flags) { diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c index e727aba427a17d..963d5203178d97 100644 --- a/odb/source-inmemory.c +++ b/odb/source-inmemory.c @@ -230,15 +230,13 @@ static int odb_source_inmemory_count_objects(struct odb_source *source, static int odb_source_inmemory_write_object(struct odb_source *source, const void *buf, size_t len, enum object_type type, - struct object_id *oid, + const struct object_id *oid, const struct object_id *compat_oid UNUSED, enum odb_write_object_flags flags UNUSED) { struct odb_source_inmemory *inmemory = odb_source_inmemory_downcast(source); struct inmemory_object *object; - hash_object_file(source->odb->repo->hash_algo, buf, len, type, oid); - if (!inmemory->objects) { CALLOC_ARRAY(inmemory->objects, 1); oidtree_init(inmemory->objects); @@ -285,6 +283,8 @@ static int odb_source_inmemory_write_object_stream(struct odb_source *source, goto out; } + hash_object_file(source->odb->repo->hash_algo, data, total_read, OBJ_BLOB, oid); + ret = odb_source_inmemory_write_object(source, data, len, OBJ_BLOB, oid, NULL, 0); if (ret < 0) diff --git a/odb/source-loose.c b/odb/source-loose.c index ca223109cd46e4..d4715da6d1c55e 100644 --- a/odb/source-loose.c +++ b/odb/source-loose.c @@ -584,19 +584,21 @@ static int odb_source_loose_freshen_object(struct odb_source *source, static int odb_source_loose_write_object(struct odb_source *source, const void *buf, size_t len, - enum object_type type, struct object_id *oid, + enum object_type type, + const struct object_id *oid, const struct object_id *compat_oid, enum odb_write_object_flags flags) { struct odb_source_loose *loose = odb_source_loose_downcast(source); - const struct git_hash_algo *algo = source->odb->repo->hash_algo; char hdr[MAX_HEADER_LEN]; - size_t hdrlen = sizeof(hdr); + int hdrlen; + + hdrlen = format_object_header(hdr, sizeof(hdr), type, len); - /* Normally if we have it in the pack then we do not bother writing + /* + * Normally if we have it in the pack then we do not bother writing * it out into .git/objects/??/?{38} file. */ - write_object_file_prepare(algo, buf, len, type, oid, hdr, &hdrlen); if (odb_freshen_object(source->odb, oid)) return 0; if (write_loose_object(loose, oid, hdr, hdrlen, buf, len, 0, flags)) diff --git a/odb/source-packed.c b/odb/source-packed.c index af0d533375176f..f7f17064472f07 100644 --- a/odb/source-packed.c +++ b/odb/source-packed.c @@ -529,7 +529,7 @@ static int odb_source_packed_write_object(struct odb_source *source UNUSED, const void *buf UNUSED, size_t len UNUSED, enum object_type type UNUSED, - struct object_id *oid UNUSED, + const struct object_id *oid UNUSED, const struct object_id *compat_oid UNUSED, unsigned flags UNUSED) { diff --git a/odb/source.h b/odb/source.h index b3c1ca3a6604fc..c4e94c9d0d4615 100644 --- a/odb/source.h +++ b/odb/source.h @@ -206,7 +206,7 @@ struct odb_source { int (*write_object)(struct odb_source *source, const void *buf, size_t len, enum object_type type, - struct object_id *oid, + const struct object_id *oid, const struct object_id *compat_oid, enum odb_write_object_flags flags); @@ -416,7 +416,7 @@ static inline int odb_source_freshen_object(struct odb_source *source, static inline int odb_source_write_object(struct odb_source *source, const void *buf, unsigned long len, enum object_type type, - struct object_id *oid, + const struct object_id *oid, const struct object_id *compat_oid, enum odb_write_object_flags flags) { diff --git a/t/unit-tests/u-odb-inmemory.c b/t/unit-tests/u-odb-inmemory.c index 2dbc3ab1df689a..28a69fc24477c4 100644 --- a/t/unit-tests/u-odb-inmemory.c +++ b/t/unit-tests/u-odb-inmemory.c @@ -43,6 +43,7 @@ static void cl_assert_write_object(struct odb_source_inmemory *source, struct object_id *oid) { size_t content_len = strlen(content); + hash_object_file(repo.hash_algo, content, content_len, type, oid); cl_must_pass(odb_source_write_object(&source->base, content, content_len, type, oid, NULL, 0)); } From b688086b8fd56e1bbab36e6bb26e12844e6e6965 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 17 Jul 2026 11:32:12 +0200 Subject: [PATCH 21/47] odb: lift object existence check out of the "loose" backend Before writing a new loose object we first check whether the object already exists in any of the sources attached to the object database. This results in a couple of issues: - We have a layering violation, where the source needs to be aware of objects stored in any of the other sources. - Every backend would have to reimplement this check, which feels somewhat pointless. - It is not possible to easily write an object into a source in case the same object already exists in another source. Refactor the code and lift up the object existence check from the "loose" backend into the generic ODB layer. No callers need adjustment as none of them write via a specific source, but via the ODB layer. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- odb.c | 7 +++++++ odb/source-loose.c | 8 ++------ 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/odb.c b/odb.c index 4adbdf8a648982..bfeca76f4e4af9 100644 --- a/odb.c +++ b/odb.c @@ -997,6 +997,13 @@ int odb_write_object_ext(struct object_database *odb, hash_object_file(odb->repo->hash_algo, buf, len, type, oid); + /* + * We can skip the write in case we already have the object available. + * In that case, we only freshen its mtime. + */ + if (odb_freshen_object(odb, oid)) + return 0; + if (compat) { const struct git_hash_algo *algo = odb->repo->hash_algo; diff --git a/odb/source-loose.c b/odb/source-loose.c index d4715da6d1c55e..04af1a54a3f631 100644 --- a/odb/source-loose.c +++ b/odb/source-loose.c @@ -595,16 +595,12 @@ static int odb_source_loose_write_object(struct odb_source *source, hdrlen = format_object_header(hdr, sizeof(hdr), type, len); - /* - * Normally if we have it in the pack then we do not bother writing - * it out into .git/objects/??/?{38} file. - */ - if (odb_freshen_object(source->odb, oid)) - return 0; if (write_loose_object(loose, oid, hdr, hdrlen, buf, len, 0, flags)) return -1; + if (compat_oid) return repo_add_loose_object_map(loose, oid, compat_oid); + return 0; } From 8d9135dce3be50c961f23f5e89352b0fd1d61117 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 17 Jul 2026 11:32:13 +0200 Subject: [PATCH 22/47] odb: support setting mtime when writing objects The function `force_object_loose()` is used to loosen packed objects before repacking. It passes the pack's mtime along so that the newly written loose object inherits the same timestamp. This matters for object pruning, which uses the mtime to determine whether an object is old enough to be pruned. In a subsequent commit, `force_object_loose()` will be converted to use the generic `odb_source_write_object()` interface instead of calling `write_loose_object()` directly. But the generic interface doesn't yet support setting a specific mtime, which makes it impossible to implement the logic as of now. Prepare for the change by introducing a new `mtime` parameter to this function that we plumb through the stack. If set, the backends are instructed to set the object's mtime accordingly. If unset, the backends are expected to use the current time instead. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- builtin/pack-objects.c | 2 +- object-file.c | 29 ++++++++++++++++++++--------- object-file.h | 8 +++++--- odb.c | 6 +++--- odb/source-files.c | 10 ++++++---- odb/source-inmemory.c | 6 ++++-- odb/source-loose.c | 8 +++++--- odb/source-packed.c | 13 +++++++++++-- odb/source.h | 12 ++++++++---- read-cache.c | 2 +- t/unit-tests/u-odb-inmemory.c | 6 +++--- 11 files changed, 67 insertions(+), 35 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index ea5eab4cf841bf..e64a96f1a727ec 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -4642,7 +4642,7 @@ static void loosen_unused_packed_objects(void) !has_sha1_pack_kept_or_nonlocal(&oid) && !loosened_object_can_be_discarded(&oid, p->mtime)) { if (force_object_loose(the_repository->objects->sources, - &oid, p->mtime)) + &oid, &p->mtime)) die(_("unable to force loose object")); loosened_objects_nr++; } diff --git a/object-file.c b/object-file.c index 9ca14f484dbd5d..5b075309506334 100644 --- a/object-file.c +++ b/object-file.c @@ -67,9 +67,17 @@ const char *odb_loose_path(struct odb_source_loose *loose, } /* Returns 1 if we have successfully freshened the file, 0 otherwise. */ -static int freshen_file(const char *fn) +static int freshen_file(const char *fn, const time_t *mtime) { - return !utime(fn, NULL); + struct utimbuf times, *timesp = NULL; + + if (mtime) { + times.actime = *mtime; + times.modtime = *mtime; + timesp = × + } + + return !utime(fn, timesp); } /* @@ -79,11 +87,12 @@ static int freshen_file(const char *fn) * either does not exist on disk, or has a stale mtime and may be subject to * pruning). */ -int check_and_freshen_file(const char *fn, int freshen) +int check_and_freshen_file(const char *fn, int freshen, + const time_t *mtime) { if (access(fn, F_OK)) return 0; - if (freshen && !freshen_file(fn)) + if (freshen && !freshen_file(fn, mtime)) return 0; return 1; } @@ -706,7 +715,7 @@ static int end_loose_object_common(struct odb_source_loose *loose, int write_loose_object(struct odb_source_loose *loose, const struct object_id *oid, char *hdr, int hdrlen, const void *buf, unsigned long len, - time_t mtime, unsigned flags) + const time_t *mtime, unsigned flags) { int fd, ret; unsigned char compressed[4096]; @@ -751,9 +760,11 @@ int write_loose_object(struct odb_source_loose *loose, close_loose_object(loose, fd, tmp_file.buf); if (mtime) { - struct utimbuf utb; - utb.actime = mtime; - utb.modtime = mtime; + struct utimbuf utb = { + .actime = *mtime, + .modtime = *mtime, + }; + if (utime(tmp_file.buf, &utb) < 0 && !(flags & ODB_WRITE_OBJECT_SILENT)) warning_errno(_("failed utime() on %s"), tmp_file.buf); @@ -883,7 +894,7 @@ int odb_source_loose_write_stream(struct odb_source_loose *loose, } int force_object_loose(struct odb_source *source, - const struct object_id *oid, time_t mtime) + const struct object_id *oid, const time_t *mtime) { struct odb_source_files *files = odb_source_files_downcast(source); const struct git_hash_algo *compat = source->odb->repo->compat_hash_algo; diff --git a/object-file.h b/object-file.h index 08aafcda0df67b..9fd540afb6c6e7 100644 --- a/object-file.h +++ b/object-file.h @@ -99,7 +99,8 @@ int format_object_header(char *str, size_t size, enum object_type type, size_t objsize); int force_object_loose(struct odb_source *source, - const struct object_id *oid, time_t mtime); + const struct object_id *oid, + const time_t *mtime); /** * With in-core object data in "buf", rehash it to make sure the @@ -137,10 +138,11 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf, int write_loose_object(struct odb_source_loose *loose, const struct object_id *oid, char *hdr, int hdrlen, const void *buf, unsigned long len, - time_t mtime, unsigned flags); + const time_t *mtime, unsigned flags); /* Helper to check and "touch" a file */ -int check_and_freshen_file(const char *fn, int freshen); +int check_and_freshen_file(const char *fn, int freshen, + const time_t *mtime); /* * Open the loose object at path, check its hash, and return the contents, diff --git a/odb.c b/odb.c index bfeca76f4e4af9..dabd481f57dbc4 100644 --- a/odb.c +++ b/odb.c @@ -738,7 +738,7 @@ int odb_pretend_object(struct object_database *odb, return 0; return odb_source_write_object(odb->inmemory_objects, - buf, len, type, oid, NULL, 0); + buf, len, type, oid, NULL, NULL, 0); } void *odb_read_object(struct object_database *odb, @@ -829,7 +829,7 @@ int odb_freshen_object(struct object_database *odb, struct odb_source *source; odb_prepare_alternates(odb); for (source = odb->sources; source; source = source->next) - if (odb_source_freshen_object(source, oid)) + if (odb_source_freshen_object(source, oid, NULL)) return 1; return 0; } @@ -1024,7 +1024,7 @@ int odb_write_object_ext(struct object_database *odb, } return odb_source_write_object(odb->sources, buf, len, type, - oid, compat_oid_p, flags); + oid, compat_oid_p, NULL, flags); } int odb_write_object_stream(struct object_database *odb, diff --git a/odb/source-files.c b/odb/source-files.c index 06dfc8dd78aecb..4df4e1af6cf76b 100644 --- a/odb/source-files.c +++ b/odb/source-files.c @@ -150,11 +150,12 @@ static int odb_source_files_find_abbrev_len(struct odb_source *source, } static int odb_source_files_freshen_object(struct odb_source *source, - const struct object_id *oid) + const struct object_id *oid, + const time_t *mtime) { struct odb_source_files *files = odb_source_files_downcast(source); - if (odb_source_freshen_object(&files->packed->base, oid) || - odb_source_freshen_object(&files->loose->base, oid)) + if (odb_source_freshen_object(&files->packed->base, oid, mtime) || + odb_source_freshen_object(&files->loose->base, oid, mtime)) return 1; return 0; } @@ -164,11 +165,12 @@ static int odb_source_files_write_object(struct odb_source *source, enum object_type type, const struct object_id *oid, const struct object_id *compat_oid, + const time_t *mtime, enum odb_write_object_flags flags) { struct odb_source_files *files = odb_source_files_downcast(source); return odb_source_write_object(&files->loose->base, buf, len, type, - oid, compat_oid, flags); + oid, compat_oid, mtime, flags); } static int odb_source_files_write_object_stream(struct odb_source *source, diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c index 963d5203178d97..3e71611b8e0071 100644 --- a/odb/source-inmemory.c +++ b/odb/source-inmemory.c @@ -232,6 +232,7 @@ static int odb_source_inmemory_write_object(struct odb_source *source, enum object_type type, const struct object_id *oid, const struct object_id *compat_oid UNUSED, + const time_t *mtime UNUSED, enum odb_write_object_flags flags UNUSED) { struct odb_source_inmemory *inmemory = odb_source_inmemory_downcast(source); @@ -286,7 +287,7 @@ static int odb_source_inmemory_write_object_stream(struct odb_source *source, hash_object_file(source->odb->repo->hash_algo, data, total_read, OBJ_BLOB, oid); ret = odb_source_inmemory_write_object(source, data, len, OBJ_BLOB, oid, - NULL, 0); + NULL, NULL, 0); if (ret < 0) goto out; @@ -296,7 +297,8 @@ static int odb_source_inmemory_write_object_stream(struct odb_source *source, } static int odb_source_inmemory_freshen_object(struct odb_source *source, - const struct object_id *oid) + const struct object_id *oid, + const time_t *mtime UNUSED) { struct odb_source_inmemory *inmemory = odb_source_inmemory_downcast(source); if (find_cached_object(inmemory, oid)) diff --git a/odb/source-loose.c b/odb/source-loose.c index 04af1a54a3f631..520a30157cf0d6 100644 --- a/odb/source-loose.c +++ b/odb/source-loose.c @@ -574,12 +574,13 @@ static int odb_source_loose_count_objects(struct odb_source *source, } static int odb_source_loose_freshen_object(struct odb_source *source, - const struct object_id *oid) + const struct object_id *oid, + const time_t *mtime) { struct odb_source_loose *loose = odb_source_loose_downcast(source); static struct strbuf path = STRBUF_INIT; odb_loose_path(loose, &path, oid); - return !!check_and_freshen_file(path.buf, 1); + return !!check_and_freshen_file(path.buf, 1, mtime); } static int odb_source_loose_write_object(struct odb_source *source, @@ -587,6 +588,7 @@ static int odb_source_loose_write_object(struct odb_source *source, enum object_type type, const struct object_id *oid, const struct object_id *compat_oid, + const time_t *mtime, enum odb_write_object_flags flags) { struct odb_source_loose *loose = odb_source_loose_downcast(source); @@ -595,7 +597,7 @@ static int odb_source_loose_write_object(struct odb_source *source, hdrlen = format_object_header(hdr, sizeof(hdr), type, len); - if (write_loose_object(loose, oid, hdr, hdrlen, buf, len, 0, flags)) + if (write_loose_object(loose, oid, hdr, hdrlen, buf, len, mtime, flags)) return -1; if (compat_oid) diff --git a/odb/source-packed.c b/odb/source-packed.c index f7f17064472f07..5e5da9bc547f9a 100644 --- a/odb/source-packed.c +++ b/odb/source-packed.c @@ -507,18 +507,26 @@ static int odb_source_packed_find_abbrev_len(struct odb_source *source, } static int odb_source_packed_freshen_object(struct odb_source *source, - const struct object_id *oid) + const struct object_id *oid, + const time_t *mtime) { struct odb_source_packed *packed = odb_source_packed_downcast(source); + struct utimbuf times, *timesp = NULL; struct pack_entry e; + if (mtime) { + times.actime = *mtime; + times.modtime = *mtime; + timesp = × + } + if (!find_pack_entry(packed, oid, &e)) return 0; if (e.p->is_cruft) return 0; if (e.p->freshened) return 1; - if (utime(e.p->pack_name, NULL)) + if (utime(e.p->pack_name, timesp)) return 0; e.p->freshened = 1; @@ -531,6 +539,7 @@ static int odb_source_packed_write_object(struct odb_source *source UNUSED, enum object_type type UNUSED, const struct object_id *oid UNUSED, const struct object_id *compat_oid UNUSED, + const time_t *mtime UNUSED, unsigned flags UNUSED) { return error("packed backend cannot write objects"); diff --git a/odb/source.h b/odb/source.h index c4e94c9d0d4615..fc04dd5cda8800 100644 --- a/odb/source.h +++ b/odb/source.h @@ -190,7 +190,8 @@ struct odb_source { * has been freshened. */ int (*freshen_object)(struct odb_source *source, - const struct object_id *oid); + const struct object_id *oid, + const time_t *mtime); /* * This callback is expected to persist the given object into the @@ -208,6 +209,7 @@ struct odb_source { enum object_type type, const struct object_id *oid, const struct object_id *compat_oid, + const time_t *mtime, enum odb_write_object_flags flags); /* @@ -403,9 +405,10 @@ static inline int odb_source_find_abbrev_len(struct odb_source *source, * not exist. */ static inline int odb_source_freshen_object(struct odb_source *source, - const struct object_id *oid) + const struct object_id *oid, + const time_t *mtime) { - return source->freshen_object(source, oid); + return source->freshen_object(source, oid, mtime); } /* @@ -418,10 +421,11 @@ static inline int odb_source_write_object(struct odb_source *source, enum object_type type, const struct object_id *oid, const struct object_id *compat_oid, + const time_t *mtime, enum odb_write_object_flags flags) { return source->write_object(source, buf, len, type, oid, - compat_oid, flags); + compat_oid, mtime, flags); } /* diff --git a/read-cache.c b/read-cache.c index 3510b49edf70be..c67930177f7639 100644 --- a/read-cache.c +++ b/read-cache.c @@ -2342,7 +2342,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) */ static void freshen_shared_index(const char *shared_index, int warn) { - if (!check_and_freshen_file(shared_index, 1) && warn) + if (!check_and_freshen_file(shared_index, 1, NULL) && warn) warning(_("could not freshen shared index '%s'"), shared_index); } diff --git a/t/unit-tests/u-odb-inmemory.c b/t/unit-tests/u-odb-inmemory.c index 28a69fc24477c4..ddf2db5c811fb8 100644 --- a/t/unit-tests/u-odb-inmemory.c +++ b/t/unit-tests/u-odb-inmemory.c @@ -45,7 +45,7 @@ static void cl_assert_write_object(struct odb_source_inmemory *source, size_t content_len = strlen(content); hash_object_file(repo.hash_algo, content, content_len, type, oid); cl_must_pass(odb_source_write_object(&source->base, content, content_len, - type, oid, NULL, 0)); + type, oid, NULL, NULL, 0)); } void test_odb_inmemory__initialize(void) @@ -256,11 +256,11 @@ void test_odb_inmemory__freshen_object(void) const char *end; cl_must_pass(parse_oid_hex_algop(RANDOM_OID, &oid, &end, repo.hash_algo)); - cl_assert_equal_i(odb_source_freshen_object(&source->base, &oid), 0); + cl_assert_equal_i(odb_source_freshen_object(&source->base, &oid, NULL), 0); cl_assert_write_object(source, "foobar", OBJ_BLOB, &written_oid); cl_assert_equal_i(odb_source_freshen_object(&source->base, - &written_oid), 1); + &written_oid, NULL), 1); odb_source_free(&source->base); } From 0e002f6dc74841324687284e6a3ae4a6061c0c73 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 17 Jul 2026 11:32:14 +0200 Subject: [PATCH 23/47] object-file: fix memory leak in `force_object_loose()` We return an error when converting the given object to the compatibility hash algorithm fails. This early return causes a memory leak though, because we don't free the content buffer that we've already read before via `odb_read_object_info_extended()`. Plug the memory leak by creating a common exit path where the buffer gets free'd. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- object-file.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/object-file.c b/object-file.c index 5b075309506334..067a63a4f1e13b 100644 --- a/object-file.c +++ b/object-file.c @@ -898,7 +898,7 @@ int force_object_loose(struct odb_source *source, { struct odb_source_files *files = odb_source_files_downcast(source); const struct git_hash_algo *compat = source->odb->repo->compat_hash_algo; - void *buf; + void *buf = NULL; size_t len; struct object_info oi = OBJECT_INFO_INIT; struct object_id compat_oid; @@ -916,19 +916,29 @@ int force_object_loose(struct odb_source *source, oi.typep = &type; oi.sizep = &len; oi.contentp = &buf; - if (odb_read_object_info_extended(source->odb, oid, &oi, 0)) - return error(_("cannot read object for %s"), oid_to_hex(oid)); + if (odb_read_object_info_extended(source->odb, oid, &oi, 0)) { + ret = error(_("cannot read object for %s"), oid_to_hex(oid)); + goto out; + } + if (compat) { - if (repo_oid_to_algop(source->odb->repo, oid, compat, &compat_oid)) - return error(_("cannot map object %s to %s"), - oid_to_hex(oid), compat->name); + if (repo_oid_to_algop(source->odb->repo, oid, compat, &compat_oid)) { + ret = error(_("cannot map object %s to %s"), + oid_to_hex(oid), compat->name); + goto out; + } } + hdrlen = format_object_header(hdr, sizeof(hdr), type, len); ret = write_loose_object(files->loose, oid, hdr, hdrlen, buf, len, mtime, 0); - if (!ret && compat) + if (ret) + goto out; + + if (compat) ret = repo_add_loose_object_map(files->loose, oid, &compat_oid); - free(buf); +out: + free(buf); return ret; } From 627d3b7ab5dc76f0d4e4ef7a42184524df8f2ab1 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 17 Jul 2026 11:32:15 +0200 Subject: [PATCH 24/47] object-file: force objects loose via generic interface When repacking objects we may end up "loosening" objects via `force_objects_loose()`. The implementation of this logic still sits with "object-file.c" even though it is ultimately an implementation detail of the "files" backend. Moving this logic around is non-trivial though as we depend on `write_loose_object()`, which is an internal implementation detail of how we write loose objects. Until now it wasn't possible to use the generic function `odb_source_write_object()` though, because the "loose" implementation thereof would skip writing the object in case it already exists in any other source. This restriction was lifted over the preceding commits though, where this object existence check is now handled on the object database level and not on the individual source level anymore. Consequently, it is now possible to use generic interfaces. Refactor the code accordingly so that we can move the logic around in a subsequent commit. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- object-file.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/object-file.c b/object-file.c index 067a63a4f1e13b..89825feed0c1b1 100644 --- a/object-file.c +++ b/object-file.c @@ -898,13 +898,11 @@ int force_object_loose(struct odb_source *source, { struct odb_source_files *files = odb_source_files_downcast(source); const struct git_hash_algo *compat = source->odb->repo->compat_hash_algo; - void *buf = NULL; - size_t len; struct object_info oi = OBJECT_INFO_INIT; - struct object_id compat_oid; + struct object_id compat_oid, *compat_oid_p = NULL; enum object_type type; - char hdr[MAX_HEADER_LEN]; - int hdrlen; + void *buf = NULL; + size_t len; int ret; for (struct odb_source *s = source->odb->sources; s; s = s->next) { @@ -927,15 +925,12 @@ int force_object_loose(struct odb_source *source, oid_to_hex(oid), compat->name); goto out; } - } - hdrlen = format_object_header(hdr, sizeof(hdr), type, len); - ret = write_loose_object(files->loose, oid, hdr, hdrlen, buf, len, mtime, 0); - if (ret) - goto out; + compat_oid_p = &compat_oid; + } - if (compat) - ret = repo_add_loose_object_map(files->loose, oid, &compat_oid); + ret = odb_source_write_object(&files->loose->base, buf, len, type, oid, + compat_oid_p, mtime, 0); out: free(buf); From a6c1e16b480a797a61a47616772e7254f7759053 Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 17 Jul 2026 11:32:16 +0200 Subject: [PATCH 25/47] object-file: move `force_object_loose()` In the preceding commits we have refactored `force_object_loose()` to not call internal functions anymore for writing the object. Instead, it now only uses generic functions that are accessible to all callers. Consequently, we can now easily move the function to its only caller, which is git-pack-objects(1). Do so. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- builtin/pack-objects.c | 46 ++++++++++++++++++++++++++++++++++++++++++ object-file.c | 44 ---------------------------------------- object-file.h | 4 ---- 3 files changed, 46 insertions(+), 48 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index e64a96f1a727ec..bb3bc486e8be90 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -32,6 +32,7 @@ #include "list.h" #include "packfile.h" #include "object-file.h" +#include "object-file-convert.h" #include "odb.h" #include "odb/streaming.h" #include "replace-object.h" @@ -4622,6 +4623,51 @@ static int loosened_object_can_be_discarded(const struct object_id *oid, return 1; } +static int force_object_loose(struct odb_source *source, + const struct object_id *oid, + const time_t *mtime) +{ + struct odb_source_files *files = odb_source_files_downcast(source); + const struct git_hash_algo *compat = source->odb->repo->compat_hash_algo; + struct object_info oi = OBJECT_INFO_INIT; + struct object_id compat_oid, *compat_oid_p = NULL; + enum object_type type; + void *buf = NULL; + size_t len; + int ret; + + for (struct odb_source *s = source->odb->sources; s; s = s->next) { + struct odb_source_files *files = odb_source_files_downcast(s); + if (!odb_source_read_object_info(&files->loose->base, oid, NULL, 0)) + return 0; + } + + oi.typep = &type; + oi.sizep = &len; + oi.contentp = &buf; + if (odb_read_object_info_extended(source->odb, oid, &oi, 0)) { + ret = error(_("cannot read object for %s"), oid_to_hex(oid)); + goto out; + } + + if (compat) { + if (repo_oid_to_algop(source->odb->repo, oid, compat, &compat_oid)) { + ret = error(_("cannot map object %s to %s"), + oid_to_hex(oid), compat->name); + goto out; + } + + compat_oid_p = &compat_oid; + } + + ret = odb_source_write_object(&files->loose->base, buf, len, type, oid, + compat_oid_p, mtime, 0); + +out: + free(buf); + return ret; +} + static void loosen_unused_packed_objects(void) { struct packed_git *p; diff --git a/object-file.c b/object-file.c index 89825feed0c1b1..b867d8d9de10af 100644 --- a/object-file.c +++ b/object-file.c @@ -893,50 +893,6 @@ int odb_source_loose_write_stream(struct odb_source_loose *loose, return err; } -int force_object_loose(struct odb_source *source, - const struct object_id *oid, const time_t *mtime) -{ - struct odb_source_files *files = odb_source_files_downcast(source); - const struct git_hash_algo *compat = source->odb->repo->compat_hash_algo; - struct object_info oi = OBJECT_INFO_INIT; - struct object_id compat_oid, *compat_oid_p = NULL; - enum object_type type; - void *buf = NULL; - size_t len; - int ret; - - for (struct odb_source *s = source->odb->sources; s; s = s->next) { - struct odb_source_files *files = odb_source_files_downcast(s); - if (!odb_source_read_object_info(&files->loose->base, oid, NULL, 0)) - return 0; - } - - oi.typep = &type; - oi.sizep = &len; - oi.contentp = &buf; - if (odb_read_object_info_extended(source->odb, oid, &oi, 0)) { - ret = error(_("cannot read object for %s"), oid_to_hex(oid)); - goto out; - } - - if (compat) { - if (repo_oid_to_algop(source->odb->repo, oid, compat, &compat_oid)) { - ret = error(_("cannot map object %s to %s"), - oid_to_hex(oid), compat->name); - goto out; - } - - compat_oid_p = &compat_oid; - } - - ret = odb_source_write_object(&files->loose->base, buf, len, type, oid, - compat_oid_p, mtime, 0); - -out: - free(buf); - return ret; -} - /* * We can't use the normal fsck_error_function() for index_mem(), * because we don't yet have a valid oid for it to report. Instead, diff --git a/object-file.h b/object-file.h index 9fd540afb6c6e7..31781a9c5308d8 100644 --- a/object-file.h +++ b/object-file.h @@ -98,10 +98,6 @@ int for_each_file_in_obj_subdir(unsigned int subdir_nr, int format_object_header(char *str, size_t size, enum object_type type, size_t objsize); -int force_object_loose(struct odb_source *source, - const struct object_id *oid, - const time_t *mtime); - /** * With in-core object data in "buf", rehash it to make sure the * object name actually matches "oid" to detect object corruption. From 810f8b203303f27543537d3ceadc90b065ed9c8f Mon Sep 17 00:00:00 2001 From: Patrick Steinhardt Date: Fri, 17 Jul 2026 11:32:17 +0200 Subject: [PATCH 26/47] object-file: move logic to write loose objects The logic to write loose objects is split up across "object-file.c" and "odb/source-loose.c". This split is somewhat weird, but it is the result of two things: - `force_object_loose()` used to reach into internals of how exactly we write objects. - The logic of writing objects is intertwined with potentially starting a transaction. We have refactored `force_object_loose()` over preceding commits to work via generic interfaces now, so this reason doesn't exist anymore. But the second reason still does, as our management of "files" transactions and their ad-hoc creation is still very messy. This area definitely requires further work, and that work is indeed ongoing. That being said, we can already move the writing logic into the "loose" backend rather easily. All we have to do is to expose two functions that relate to the transactions. Expose these two functions and move the writing logic into the "loose" backend accordingly so that it becomes more self-contained. Note that this requires us to drop a reference to `the_repository` in favor of using the source's repository in `start_loose_object_common()`. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano --- object-file.c | 360 +-------------------------------------------- object-file.h | 22 +-- odb/source-loose.c | 354 +++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 357 insertions(+), 379 deletions(-) diff --git a/object-file.c b/object-file.c index b867d8d9de10af..bdc97d7943af25 100644 --- a/object-file.c +++ b/object-file.c @@ -491,7 +491,7 @@ struct odb_transaction_files { const char *prefix; }; -static int odb_transaction_files_prepare(struct odb_transaction *base) +int odb_transaction_files_prepare(struct odb_transaction *base) { struct odb_transaction_files *transaction = container_of_or_null(base, struct odb_transaction_files, base); @@ -514,8 +514,8 @@ static int odb_transaction_files_prepare(struct odb_transaction *base) return 0; } -static void odb_transaction_files_fsync(struct odb_transaction *base, - int fd, const char *filename) +void odb_transaction_files_fsync(struct odb_transaction *base, + int fd, const char *filename) { struct odb_transaction_files *transaction = container_of_or_null(base, struct odb_transaction_files, base); @@ -539,360 +539,6 @@ static void odb_transaction_files_fsync(struct odb_transaction *base, } } -/* Finalize a file on disk, and close it. */ -static void close_loose_object(struct odb_source_loose *loose, - int fd, const char *filename) -{ - if (loose->base.will_destroy) - goto out; - - if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) - odb_transaction_files_fsync(loose->base.odb->transaction, fd, filename); - else if (fsync_object_files > 0) - fsync_or_die(fd, filename); - else - fsync_component_or_die(FSYNC_COMPONENT_LOOSE_OBJECT, fd, - filename); - -out: - if (close(fd) != 0) - die_errno(_("error when closing loose object file")); -} - -/* Size of directory component, including the ending '/' */ -static inline int directory_size(const char *filename) -{ - const char *s = strrchr(filename, '/'); - if (!s) - return 0; - return s - filename + 1; -} - -/* - * This creates a temporary file in the same directory as the final - * 'filename' - * - * We want to avoid cross-directory filename renames, because those - * can have problems on various filesystems (FAT, NFS, Coda). - */ -static int create_tmpfile(struct repository *repo, - struct strbuf *tmp, const char *filename) -{ - int fd, dirlen = directory_size(filename); - - strbuf_reset(tmp); - strbuf_add(tmp, filename, dirlen); - strbuf_addstr(tmp, "tmp_obj_XXXXXX"); - fd = git_mkstemp_mode(tmp->buf, 0444); - if (fd < 0 && dirlen && errno == ENOENT) { - /* - * Make sure the directory exists; note that the contents - * of the buffer are undefined after mkstemp returns an - * error, so we have to rewrite the whole buffer from - * scratch. - */ - strbuf_reset(tmp); - strbuf_add(tmp, filename, dirlen - 1); - if (mkdir(tmp->buf, 0777) && errno != EEXIST) - return -1; - if (adjust_shared_perm(repo, tmp->buf)) - return -1; - - /* Try again */ - strbuf_addstr(tmp, "/tmp_obj_XXXXXX"); - fd = git_mkstemp_mode(tmp->buf, 0444); - } - return fd; -} - -/** - * Common steps for loose object writers to start writing loose - * objects: - * - * - Create tmpfile for the loose object. - * - Setup zlib stream for compression. - * - Start to feed header to zlib stream. - * - * Returns a "fd", which should later be provided to - * end_loose_object_common(). - */ -static int start_loose_object_common(struct odb_source_loose *loose, - struct strbuf *tmp_file, - const char *filename, unsigned flags, - git_zstream *stream, - unsigned char *buf, size_t buflen, - struct git_hash_ctx *c, struct git_hash_ctx *compat_c, - char *hdr, int hdrlen) -{ - const struct git_hash_algo *algo = loose->base.odb->repo->hash_algo; - const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; - int fd; - struct repo_config_values *cfg = repo_config_values(the_repository); - - fd = create_tmpfile(loose->base.odb->repo, tmp_file, filename); - if (fd < 0) { - if (flags & ODB_WRITE_OBJECT_SILENT) - return -1; - else if (errno == EACCES) - return error(_("insufficient permission for adding " - "an object to repository database %s"), - loose->base.path); - else - return error_errno( - _("unable to create temporary file")); - } - - /* Setup zlib stream for compression */ - git_deflate_init(stream, cfg->zlib_compression_level); - stream->next_out = buf; - stream->avail_out = buflen; - git_hash_init(c, algo); - if (compat && compat_c) - git_hash_init(compat_c, compat); - - /* Start to feed header to zlib stream */ - stream->next_in = (unsigned char *)hdr; - stream->avail_in = hdrlen; - while (git_deflate(stream, 0) == Z_OK) - ; /* nothing */ - git_hash_update(c, hdr, hdrlen); - if (compat && compat_c) - git_hash_update(compat_c, hdr, hdrlen); - - return fd; -} - -/** - * Common steps for the inner git_deflate() loop for writing loose - * objects. Returns what git_deflate() returns. - */ -static int write_loose_object_common(struct odb_source_loose *loose, - struct git_hash_ctx *c, struct git_hash_ctx *compat_c, - git_zstream *stream, const int flush, - unsigned char *in0, const int fd, - unsigned char *compressed, - const size_t compressed_len) -{ - const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; - int ret; - - ret = git_deflate(stream, flush ? Z_FINISH : 0); - git_hash_update(c, in0, stream->next_in - in0); - if (compat && compat_c) - git_hash_update(compat_c, in0, stream->next_in - in0); - if (write_in_full(fd, compressed, stream->next_out - compressed) < 0) - die_errno(_("unable to write loose object file")); - stream->next_out = compressed; - stream->avail_out = compressed_len; - - return ret; -} - -/** - * Common steps for loose object writers to end writing loose objects: - * - * - End the compression of zlib stream. - * - Get the calculated oid to "oid". - */ -static int end_loose_object_common(struct odb_source_loose *loose, - struct git_hash_ctx *c, struct git_hash_ctx *compat_c, - git_zstream *stream, struct object_id *oid, - struct object_id *compat_oid) -{ - const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; - int ret; - - ret = git_deflate_end_gently(stream); - if (ret != Z_OK) - return ret; - git_hash_final_oid(oid, c); - if (compat && compat_c) - git_hash_final_oid(compat_oid, compat_c); - - return Z_OK; -} - -int write_loose_object(struct odb_source_loose *loose, - const struct object_id *oid, char *hdr, - int hdrlen, const void *buf, unsigned long len, - const time_t *mtime, unsigned flags) -{ - int fd, ret; - unsigned char compressed[4096]; - git_zstream stream; - struct git_hash_ctx c; - struct object_id parano_oid; - static struct strbuf tmp_file = STRBUF_INIT; - static struct strbuf filename = STRBUF_INIT; - - if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) - odb_transaction_files_prepare(loose->base.odb->transaction); - - odb_loose_path(loose, &filename, oid); - - fd = start_loose_object_common(loose, &tmp_file, filename.buf, flags, - &stream, compressed, sizeof(compressed), - &c, NULL, hdr, hdrlen); - if (fd < 0) - return -1; - - /* Then the data itself.. */ - stream.next_in = (void *)buf; - stream.avail_in = len; - do { - unsigned char *in0 = stream.next_in; - - ret = write_loose_object_common(loose, &c, NULL, &stream, 1, in0, fd, - compressed, sizeof(compressed)); - } while (ret == Z_OK); - - if (ret != Z_STREAM_END) - die(_("unable to deflate new object %s (%d)"), oid_to_hex(oid), - ret); - ret = end_loose_object_common(loose, &c, NULL, &stream, ¶no_oid, NULL); - if (ret != Z_OK) - die(_("deflateEnd on object %s failed (%d)"), oid_to_hex(oid), - ret); - if (!oideq(oid, ¶no_oid)) - die(_("confused by unstable object source data for %s"), - oid_to_hex(oid)); - - close_loose_object(loose, fd, tmp_file.buf); - - if (mtime) { - struct utimbuf utb = { - .actime = *mtime, - .modtime = *mtime, - }; - - if (utime(tmp_file.buf, &utb) < 0 && - !(flags & ODB_WRITE_OBJECT_SILENT)) - warning_errno(_("failed utime() on %s"), tmp_file.buf); - } - - return finalize_object_file_flags(loose->base.odb->repo, tmp_file.buf, filename.buf, - FOF_SKIP_COLLISION_CHECK); -} - -int odb_source_loose_write_stream(struct odb_source_loose *loose, - struct odb_write_stream *in_stream, size_t len, - struct object_id *oid) -{ - const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; - struct object_id compat_oid; - int fd, ret, err = 0, flush = 0; - unsigned char compressed[4096]; - git_zstream stream; - struct git_hash_ctx c, compat_c; - struct strbuf tmp_file = STRBUF_INIT; - struct strbuf filename = STRBUF_INIT; - unsigned char buf[8192]; - int dirlen; - char hdr[MAX_HEADER_LEN]; - int hdrlen; - - if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) - odb_transaction_files_prepare(loose->base.odb->transaction); - - /* Since oid is not determined, save tmp file to odb path. */ - strbuf_addf(&filename, "%s/", loose->base.path); - hdrlen = format_object_header(hdr, sizeof(hdr), OBJ_BLOB, len); - - /* - * Common steps for write_loose_object and stream_loose_object to - * start writing loose objects: - * - * - Create tmpfile for the loose object. - * - Setup zlib stream for compression. - * - Start to feed header to zlib stream. - */ - fd = start_loose_object_common(loose, &tmp_file, filename.buf, 0, - &stream, compressed, sizeof(compressed), - &c, &compat_c, hdr, hdrlen); - if (fd < 0) { - err = -1; - goto cleanup; - } - - /* Then the data itself.. */ - do { - unsigned char *in0 = stream.next_in; - - if (!stream.avail_in && !in_stream->is_finished) { - ssize_t read_len = odb_write_stream_read(in_stream, buf, - sizeof(buf)); - if (read_len < 0) { - close(fd); - err = -1; - goto cleanup; - } - - stream.avail_in = read_len; - stream.next_in = buf; - in0 = buf; - /* All data has been read. */ - if (in_stream->is_finished) - flush = 1; - } - ret = write_loose_object_common(loose, &c, &compat_c, &stream, flush, in0, fd, - compressed, sizeof(compressed)); - /* - * Unlike write_loose_object(), we do not have the entire - * buffer. If we get Z_BUF_ERROR due to too few input bytes, - * then we'll replenish them in the next input_stream->read() - * call when we loop. - */ - } while (ret == Z_OK || ret == Z_BUF_ERROR); - - if (stream.total_in != len + hdrlen) - die(_("write stream object %"PRIuMAX" != %"PRIuMAX), (uintmax_t)stream.total_in, - (uintmax_t)len + hdrlen); - - /* - * Common steps for write_loose_object and stream_loose_object to - * end writing loose object: - * - * - End the compression of zlib stream. - * - Get the calculated oid. - */ - if (ret != Z_STREAM_END) - die(_("unable to stream deflate new object (%d)"), ret); - ret = end_loose_object_common(loose, &c, &compat_c, &stream, oid, &compat_oid); - if (ret != Z_OK) - die(_("deflateEnd on stream object failed (%d)"), ret); - close_loose_object(loose, fd, tmp_file.buf); - - if (odb_freshen_object(loose->base.odb, oid)) { - unlink_or_warn(tmp_file.buf); - goto cleanup; - } - odb_loose_path(loose, &filename, oid); - - /* We finally know the object path, and create the missing dir. */ - dirlen = directory_size(filename.buf); - if (dirlen) { - struct strbuf dir = STRBUF_INIT; - strbuf_add(&dir, filename.buf, dirlen); - - if (safe_create_dir_in_gitdir(loose->base.odb->repo, dir.buf) && - errno != EEXIST) { - err = error_errno(_("unable to create directory %s"), dir.buf); - strbuf_release(&dir); - goto cleanup; - } - strbuf_release(&dir); - } - - err = finalize_object_file_flags(loose->base.odb->repo, tmp_file.buf, filename.buf, - FOF_SKIP_COLLISION_CHECK); - if (!err && compat) - err = repo_add_loose_object_map(loose, oid, &compat_oid); -cleanup: - strbuf_release(&tmp_file); - strbuf_release(&filename); - return err; -} - /* * We can't use the normal fsck_error_function() for index_mem(), * because we don't yet have a valid oid for it to report. Instead, diff --git a/object-file.h b/object-file.h index 31781a9c5308d8..805f2cfa289661 100644 --- a/object-file.h +++ b/object-file.h @@ -24,20 +24,6 @@ int index_path(struct index_state *istate, struct object_id *oid, const char *pa struct object_info; struct odb_source; -/* - * Write the given stream into the loose object source. The only difference - * from the generic implementation of this function is that we don't perform an - * object existence check here. - * - * TODO: We should stop exposing this function altogether and move it into - * "odb/source-loose.c". This requires a couple of refactorings though to make - * `force_object_loose()` generic and is thus postponed to a later point in - * time. - */ -int odb_source_loose_write_stream(struct odb_source_loose *source, - struct odb_write_stream *stream, size_t len, - struct object_id *oid); - /* * Put in `buf` the name of the file in the local object database that * would be used to store a loose object with the specified oid. @@ -131,10 +117,6 @@ int finalize_object_file_flags(struct repository *repo, void hash_object_file(const struct git_hash_algo *algo, const void *buf, size_t len, enum object_type type, struct object_id *oid); -int write_loose_object(struct odb_source_loose *loose, - const struct object_id *oid, char *hdr, - int hdrlen, const void *buf, unsigned long len, - const time_t *mtime, unsigned flags); /* Helper to check and "touch" a file */ int check_and_freshen_file(const char *fn, int freshen, @@ -195,4 +177,8 @@ int odb_transaction_files_begin(struct odb_source *source, struct odb_transaction **out, enum odb_transaction_flags flags); +int odb_transaction_files_prepare(struct odb_transaction *base); +void odb_transaction_files_fsync(struct odb_transaction *base, + int fd, const char *filename); + #endif /* OBJECT_FILE_H */ diff --git a/odb/source-loose.c b/odb/source-loose.c index 520a30157cf0d6..ef0e9192777c4a 100644 --- a/odb/source-loose.c +++ b/odb/source-loose.c @@ -11,8 +11,11 @@ #include "odb/source-loose.h" #include "odb/streaming.h" #include "oidtree.h" +#include "path.h" #include "repository.h" #include "strbuf.h" +#include "tempfile.h" +#include "write-or-die.h" static int append_loose_object(const struct object_id *oid, const char *path UNUSED, @@ -583,6 +586,241 @@ static int odb_source_loose_freshen_object(struct odb_source *source, return !!check_and_freshen_file(path.buf, 1, mtime); } +/* Finalize a file on disk, and close it. */ +static void close_loose_object(struct odb_source_loose *loose, + int fd, const char *filename) +{ + if (loose->base.will_destroy) + goto out; + + if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) + odb_transaction_files_fsync(loose->base.odb->transaction, fd, filename); + else if (fsync_object_files > 0) + fsync_or_die(fd, filename); + else + fsync_component_or_die(FSYNC_COMPONENT_LOOSE_OBJECT, fd, + filename); + +out: + if (close(fd) != 0) + die_errno(_("error when closing loose object file")); +} + +/* Size of directory component, including the ending '/' */ +static inline int directory_size(const char *filename) +{ + const char *s = strrchr(filename, '/'); + if (!s) + return 0; + return s - filename + 1; +} + +/* + * This creates a temporary file in the same directory as the final + * 'filename' + * + * We want to avoid cross-directory filename renames, because those + * can have problems on various filesystems (FAT, NFS, Coda). + */ +static int create_tmpfile(struct repository *repo, + struct strbuf *tmp, const char *filename) +{ + int fd, dirlen = directory_size(filename); + + strbuf_reset(tmp); + strbuf_add(tmp, filename, dirlen); + strbuf_addstr(tmp, "tmp_obj_XXXXXX"); + fd = git_mkstemp_mode(tmp->buf, 0444); + if (fd < 0 && dirlen && errno == ENOENT) { + /* + * Make sure the directory exists; note that the contents + * of the buffer are undefined after mkstemp returns an + * error, so we have to rewrite the whole buffer from + * scratch. + */ + strbuf_reset(tmp); + strbuf_add(tmp, filename, dirlen - 1); + if (mkdir(tmp->buf, 0777) && errno != EEXIST) + return -1; + if (adjust_shared_perm(repo, tmp->buf)) + return -1; + + /* Try again */ + strbuf_addstr(tmp, "/tmp_obj_XXXXXX"); + fd = git_mkstemp_mode(tmp->buf, 0444); + } + return fd; +} + +/** + * Common steps for loose object writers to start writing loose + * objects: + * + * - Create tmpfile for the loose object. + * - Setup zlib stream for compression. + * - Start to feed header to zlib stream. + * + * Returns a "fd", which should later be provided to + * end_loose_object_common(). + */ +static int start_loose_object_common(struct odb_source_loose *loose, + struct strbuf *tmp_file, + const char *filename, unsigned flags, + git_zstream *stream, + unsigned char *buf, size_t buflen, + struct git_hash_ctx *c, struct git_hash_ctx *compat_c, + char *hdr, int hdrlen) +{ + const struct git_hash_algo *algo = loose->base.odb->repo->hash_algo; + const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; + int fd; + struct repo_config_values *cfg = repo_config_values(loose->base.odb->repo); + + fd = create_tmpfile(loose->base.odb->repo, tmp_file, filename); + if (fd < 0) { + if (flags & ODB_WRITE_OBJECT_SILENT) + return -1; + else if (errno == EACCES) + return error(_("insufficient permission for adding " + "an object to repository database %s"), + loose->base.path); + else + return error_errno( + _("unable to create temporary file")); + } + + /* Setup zlib stream for compression */ + git_deflate_init(stream, cfg->zlib_compression_level); + stream->next_out = buf; + stream->avail_out = buflen; + git_hash_init(c, algo); + if (compat && compat_c) + git_hash_init(compat_c, compat); + + /* Start to feed header to zlib stream */ + stream->next_in = (unsigned char *)hdr; + stream->avail_in = hdrlen; + while (git_deflate(stream, 0) == Z_OK) + ; /* nothing */ + git_hash_update(c, hdr, hdrlen); + if (compat && compat_c) + git_hash_update(compat_c, hdr, hdrlen); + + return fd; +} + +/** + * Common steps for the inner git_deflate() loop for writing loose + * objects. Returns what git_deflate() returns. + */ +static int write_loose_object_common(struct odb_source_loose *loose, + struct git_hash_ctx *c, struct git_hash_ctx *compat_c, + git_zstream *stream, const int flush, + unsigned char *in0, const int fd, + unsigned char *compressed, + const size_t compressed_len) +{ + const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; + int ret; + + ret = git_deflate(stream, flush ? Z_FINISH : 0); + git_hash_update(c, in0, stream->next_in - in0); + if (compat && compat_c) + git_hash_update(compat_c, in0, stream->next_in - in0); + if (write_in_full(fd, compressed, stream->next_out - compressed) < 0) + die_errno(_("unable to write loose object file")); + stream->next_out = compressed; + stream->avail_out = compressed_len; + + return ret; +} + +/** + * Common steps for loose object writers to end writing loose objects: + * + * - End the compression of zlib stream. + * - Get the calculated oid to "oid". + */ +static int end_loose_object_common(struct odb_source_loose *loose, + struct git_hash_ctx *c, struct git_hash_ctx *compat_c, + git_zstream *stream, struct object_id *oid, + struct object_id *compat_oid) +{ + const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; + int ret; + + ret = git_deflate_end_gently(stream); + if (ret != Z_OK) + return ret; + git_hash_final_oid(oid, c); + if (compat && compat_c) + git_hash_final_oid(compat_oid, compat_c); + + return Z_OK; +} + +static int write_loose_object(struct odb_source_loose *loose, + const struct object_id *oid, char *hdr, + int hdrlen, const void *buf, unsigned long len, + const time_t *mtime, unsigned flags) +{ + int fd, ret; + unsigned char compressed[4096]; + git_zstream stream; + struct git_hash_ctx c; + struct object_id parano_oid; + static struct strbuf tmp_file = STRBUF_INIT; + static struct strbuf filename = STRBUF_INIT; + + if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) + odb_transaction_files_prepare(loose->base.odb->transaction); + + odb_loose_path(loose, &filename, oid); + + fd = start_loose_object_common(loose, &tmp_file, filename.buf, flags, + &stream, compressed, sizeof(compressed), + &c, NULL, hdr, hdrlen); + if (fd < 0) + return -1; + + /* Then the data itself.. */ + stream.next_in = (void *)buf; + stream.avail_in = len; + do { + unsigned char *in0 = stream.next_in; + + ret = write_loose_object_common(loose, &c, NULL, &stream, 1, in0, fd, + compressed, sizeof(compressed)); + } while (ret == Z_OK); + + if (ret != Z_STREAM_END) + die(_("unable to deflate new object %s (%d)"), oid_to_hex(oid), + ret); + ret = end_loose_object_common(loose, &c, NULL, &stream, ¶no_oid, NULL); + if (ret != Z_OK) + die(_("deflateEnd on object %s failed (%d)"), oid_to_hex(oid), + ret); + if (!oideq(oid, ¶no_oid)) + die(_("confused by unstable object source data for %s"), + oid_to_hex(oid)); + + close_loose_object(loose, fd, tmp_file.buf); + + if (mtime) { + struct utimbuf utb = { + .actime = *mtime, + .modtime = *mtime, + }; + + if (utime(tmp_file.buf, &utb) < 0 && + !(flags & ODB_WRITE_OBJECT_SILENT)) + warning_errno(_("failed utime() on %s"), tmp_file.buf); + } + + return finalize_object_file_flags(loose->base.odb->repo, tmp_file.buf, filename.buf, + FOF_SKIP_COLLISION_CHECK); +} + static int odb_source_loose_write_object(struct odb_source *source, const void *buf, size_t len, enum object_type type, @@ -611,12 +849,120 @@ static int odb_source_loose_write_object_stream(struct odb_source *source, size_t len, struct object_id *oid) { + struct odb_source_loose *loose = odb_source_loose_downcast(source); + const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; + struct object_id compat_oid; + int fd, ret, err = 0, flush = 0; + unsigned char compressed[4096]; + git_zstream stream; + struct git_hash_ctx c, compat_c; + struct strbuf tmp_file = STRBUF_INIT; + struct strbuf filename = STRBUF_INIT; + unsigned char buf[8192]; + int dirlen; + char hdr[MAX_HEADER_LEN]; + int hdrlen; + + if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) + odb_transaction_files_prepare(loose->base.odb->transaction); + + /* Since oid is not determined, save tmp file to odb path. */ + strbuf_addf(&filename, "%s/", loose->base.path); + hdrlen = format_object_header(hdr, sizeof(hdr), OBJ_BLOB, len); + /* - * TODO: the implementation should be moved here, see the comment on - * the called function in "object-file.h". + * Common steps for write_loose_object and stream_loose_object to + * start writing loose objects: + * + * - Create tmpfile for the loose object. + * - Setup zlib stream for compression. + * - Start to feed header to zlib stream. */ - struct odb_source_loose *loose = odb_source_loose_downcast(source); - return odb_source_loose_write_stream(loose, in_stream, len, oid); + fd = start_loose_object_common(loose, &tmp_file, filename.buf, 0, + &stream, compressed, sizeof(compressed), + &c, &compat_c, hdr, hdrlen); + if (fd < 0) { + err = -1; + goto cleanup; + } + + /* Then the data itself.. */ + do { + unsigned char *in0 = stream.next_in; + + if (!stream.avail_in && !in_stream->is_finished) { + ssize_t read_len = odb_write_stream_read(in_stream, buf, + sizeof(buf)); + if (read_len < 0) { + close(fd); + err = -1; + goto cleanup; + } + + stream.avail_in = read_len; + stream.next_in = buf; + in0 = buf; + /* All data has been read. */ + if (in_stream->is_finished) + flush = 1; + } + ret = write_loose_object_common(loose, &c, &compat_c, &stream, flush, in0, fd, + compressed, sizeof(compressed)); + /* + * Unlike write_loose_object(), we do not have the entire + * buffer. If we get Z_BUF_ERROR due to too few input bytes, + * then we'll replenish them in the next input_stream->read() + * call when we loop. + */ + } while (ret == Z_OK || ret == Z_BUF_ERROR); + + if (stream.total_in != len + hdrlen) + die(_("write stream object %"PRIuMAX" != %"PRIuMAX), (uintmax_t)stream.total_in, + (uintmax_t)len + hdrlen); + + /* + * Common steps for write_loose_object and stream_loose_object to + * end writing loose object: + * + * - End the compression of zlib stream. + * - Get the calculated oid. + */ + if (ret != Z_STREAM_END) + die(_("unable to stream deflate new object (%d)"), ret); + ret = end_loose_object_common(loose, &c, &compat_c, &stream, oid, &compat_oid); + if (ret != Z_OK) + die(_("deflateEnd on stream object failed (%d)"), ret); + close_loose_object(loose, fd, tmp_file.buf); + + if (odb_freshen_object(loose->base.odb, oid)) { + unlink_or_warn(tmp_file.buf); + goto cleanup; + } + odb_loose_path(loose, &filename, oid); + + /* We finally know the object path, and create the missing dir. */ + dirlen = directory_size(filename.buf); + if (dirlen) { + struct strbuf dir = STRBUF_INIT; + strbuf_add(&dir, filename.buf, dirlen); + + if (safe_create_dir_in_gitdir(loose->base.odb->repo, dir.buf) && + errno != EEXIST) { + err = error_errno(_("unable to create directory %s"), dir.buf); + strbuf_release(&dir); + goto cleanup; + } + strbuf_release(&dir); + } + + err = finalize_object_file_flags(loose->base.odb->repo, tmp_file.buf, filename.buf, + FOF_SKIP_COLLISION_CHECK); + if (!err && compat) + err = repo_add_loose_object_map(loose, oid, &compat_oid); +cleanup: + strbuf_release(&tmp_file); + strbuf_release(&filename); + return err; } static int odb_source_loose_begin_transaction(struct odb_source *source UNUSED, From 8817d7931aab41d4423fbf588f1ab2247070d816 Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Mon, 20 Jul 2026 18:53:32 +0800 Subject: [PATCH 27/47] read-cache: remove redundant extern declarations The 'read-cache.c' file already includes 'environment.h', which provides the extern declarations for variables like 'trust_executable_bit' and 'has_symlinks'. Remove the redundant extern declarations inside 'st_mode_from_ce()' to clean up the code. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- read-cache.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/read-cache.c b/read-cache.c index 21829102ae275e..1ced8c630ef637 100644 --- a/read-cache.c +++ b/read-cache.c @@ -205,8 +205,6 @@ void fill_stat_cache_info(struct index_state *istate, struct cache_entry *ce, st static unsigned int st_mode_from_ce(const struct cache_entry *ce) { - extern int trust_executable_bit, has_symlinks; - switch (ce->ce_mode & S_IFMT) { case S_IFLNK: return has_symlinks ? S_IFLNK : (S_IFREG | 0644); From b7b6e9e02f6b6b84848b2458d5d2b3d47260d4bb Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Mon, 20 Jul 2026 18:53:33 +0800 Subject: [PATCH 28/47] read-cache: pass 'repo' to 'ce_mode_from_stat()' The ce_mode_from_stat() function is a performance-critical static inline helper in 'read-cache.h'. As we migrate configuration variables into the repository struct, this helper needs access to the repository context. Update the signature of ce_mode_from_stat() to take a 'struct repository *' parameter, and update all callers to pass the appropriate repository instance. To prepare for the overhead of replacing cheap global variable accesses with getter functions, the boolean expressions are reordered to evaluate 'S_ISREG(mode)' first. While at it, add a comment for ce_mode_from_stat(). Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- apply.c | 2 +- builtin/update-index.c | 2 +- diff-lib.c | 10 +++++----- read-cache.c | 2 +- read-cache.h | 15 ++++++++++++--- 5 files changed, 20 insertions(+), 11 deletions(-) diff --git a/apply.c b/apply.c index 249248d4f205ca..26286eb57b4f52 100644 --- a/apply.c +++ b/apply.c @@ -3894,7 +3894,7 @@ static int check_preimage(struct apply_state *state, BUG("ce_mode == 0 for path '%s'", old_name); if (trust_executable_bit || !S_ISREG(st->st_mode)) - st_mode = ce_mode_from_stat(*ce, st->st_mode); + st_mode = ce_mode_from_stat(state->repo, *ce, st->st_mode); else if (*ce) st_mode = (*ce)->ce_mode; else diff --git a/builtin/update-index.c b/builtin/update-index.c index 3d6646c318b98e..afd672a9145b4f 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -294,7 +294,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len ce->ce_flags = create_ce_flags(0); ce->ce_namelen = len; fill_stat_cache_info(the_repository->index, ce, st); - ce->ce_mode = ce_mode_from_stat(old, st->st_mode); + ce->ce_mode = ce_mode_from_stat(the_repository, old, st->st_mode); if (index_path(the_repository->index, &ce->oid, path, st, info_only ? 0 : INDEX_WRITE_OBJECT)) { diff --git a/diff-lib.c b/diff-lib.c index ae91027a024eec..46cae637ecda83 100644 --- a/diff-lib.c +++ b/diff-lib.c @@ -160,7 +160,7 @@ void run_diff_files(struct rev_info *revs, unsigned int option) changed = check_removed(ce, &st); if (!changed) - wt_mode = ce_mode_from_stat(ce, st.st_mode); + wt_mode = ce_mode_from_stat(revs->repo, ce, st.st_mode); else { if (changed < 0) { perror(ce->name); @@ -193,7 +193,7 @@ void run_diff_files(struct rev_info *revs, unsigned int option) num_compare_stages++; oidcpy(&dpath->parent[stage - 2].oid, &nce->oid); - dpath->parent[stage-2].mode = ce_mode_from_stat(nce, mode); + dpath->parent[stage-2].mode = ce_mode_from_stat(revs->repo, nce, mode); dpath->parent[stage-2].status = DIFF_STATUS_MODIFIED; } @@ -262,7 +262,7 @@ void run_diff_files(struct rev_info *revs, unsigned int option) continue; } else if (revs->diffopt.ita_invisible_in_index && ce_intent_to_add(ce)) { - newmode = ce_mode_from_stat(ce, st.st_mode); + newmode = ce_mode_from_stat(revs->repo, ce, st.st_mode); diff_addremove(&revs->diffopt, '+', newmode, null_oid(the_hash_algo), 0, ce->name, 0); continue; @@ -270,7 +270,7 @@ void run_diff_files(struct rev_info *revs, unsigned int option) changed = match_stat_with_submodule(&revs->diffopt, ce, &st, ce_option, &dirty_submodule); - newmode = ce_mode_from_stat(ce, st.st_mode); + newmode = ce_mode_from_stat(revs->repo, ce, st.st_mode); } if (!changed && !dirty_submodule) { @@ -338,7 +338,7 @@ static int get_stat_data(const struct cache_entry *ce, changed = match_stat_with_submodule(diffopt, ce, &st, 0, dirty_submodule); if (changed) { - mode = ce_mode_from_stat(ce, st.st_mode); + mode = ce_mode_from_stat(diffopt->repo, ce, st.st_mode); oid = null_oid(the_hash_algo); } } diff --git a/read-cache.c b/read-cache.c index 1ced8c630ef637..a9f059174305eb 100644 --- a/read-cache.c +++ b/read-cache.c @@ -750,7 +750,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st, int pos = index_name_pos_also_unmerged(istate, path, namelen); ent = (0 <= pos) ? istate->cache[pos] : NULL; - ce->ce_mode = ce_mode_from_stat(ent, st_mode); + ce->ce_mode = ce_mode_from_stat(istate->repo, ent, st_mode); } /* When core.ignorecase=true, determine if a directory of the same name but differing diff --git a/read-cache.h b/read-cache.h index 043da1f1aae706..af8c657ecbeb48 100644 --- a/read-cache.h +++ b/read-cache.h @@ -4,15 +4,24 @@ #include "read-cache-ll.h" #include "object.h" #include "pathspec.h" +#include "environment.h" -static inline unsigned int ce_mode_from_stat(const struct cache_entry *ce, +/* + * Determine the appropriate index mode for a file based on its stat() + * information and the existing cache entry (if any). + * + * This function handles degradation for filesystems that lack + * symlink support or reliable executable bits. + */ +static inline unsigned int ce_mode_from_stat(struct repository *repo UNUSED, + const struct cache_entry *ce, unsigned int mode) { extern int trust_executable_bit, has_symlinks; - if (!has_symlinks && S_ISREG(mode) && + if (S_ISREG(mode) && !has_symlinks && ce && S_ISLNK(ce->ce_mode)) return ce->ce_mode; - if (!trust_executable_bit && S_ISREG(mode)) { + if (S_ISREG(mode) && !trust_executable_bit) { if (ce && S_ISREG(ce->ce_mode)) return ce->ce_mode; return create_ce_mode(0666); From 99c79dabb09a298863cd96398305d2957e60d022 Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Mon, 20 Jul 2026 18:53:34 +0800 Subject: [PATCH 29/47] environment: move trust_executable_bit into repo_config_values Move the global 'trust_executable_bit' configuration into the repository-specific 'repo_config_values' struct. To ensure code readability, the getter function 'repo_trust_executable_bit()' has been introduced. Callers access this configuration by passing in 'repo' when possible, and explicitly fall back to 'the_repository' the rest of time. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- apply.c | 2 +- environment.c | 11 +++++++++-- environment.h | 4 +++- read-cache.c | 6 +++--- read-cache.h | 6 +++--- 5 files changed, 19 insertions(+), 10 deletions(-) diff --git a/apply.c b/apply.c index 26286eb57b4f52..edb1502414e4be 100644 --- a/apply.c +++ b/apply.c @@ -3893,7 +3893,7 @@ static int check_preimage(struct apply_state *state, if (*ce && !(*ce)->ce_mode) BUG("ce_mode == 0 for path '%s'", old_name); - if (trust_executable_bit || !S_ISREG(st->st_mode)) + if (repo_trust_executable_bit(state->repo) || !S_ISREG(st->st_mode)) st_mode = ce_mode_from_stat(state->repo, *ce, st->st_mode); else if (*ce) st_mode = (*ce)->ce_mode; diff --git a/environment.c b/environment.c index fc3ed8bb1c7a66..32b110c40505be 100644 --- a/environment.c +++ b/environment.c @@ -41,7 +41,6 @@ static int pack_compression_seen; static int zlib_compression_seen; -int trust_executable_bit = 1; int trust_ctime = 1; int check_stat = 1; int has_symlinks = 1; @@ -142,6 +141,13 @@ int is_bare_repository(void) return is_bare_repository_cfg && !repo_get_work_tree(the_repository); } +int repo_trust_executable_bit(struct repository *repo) +{ + return repo->initialized + ? repo_config_values(repo)->trust_executable_bit + : 1; +} + int have_git_dir(void) { return startup_info->have_repository @@ -305,7 +311,7 @@ int git_default_core_config(const char *var, const char *value, /* This needs a better name */ if (!strcmp(var, "core.filemode")) { - trust_executable_bit = git_config_bool(var, value); + cfg->trust_executable_bit = git_config_bool(var, value); return 0; } if (!strcmp(var, "core.trustctime")) { @@ -720,5 +726,6 @@ void repo_config_values_init(struct repo_config_values *cfg) { cfg->attributes_file = NULL; cfg->apply_sparse_checkout = 0; + cfg->trust_executable_bit = 1; cfg->branch_track = BRANCH_TRACK_REMOTE; } diff --git a/environment.h b/environment.h index 9eb97b3869c9b1..c15456fc0d0b39 100644 --- a/environment.h +++ b/environment.h @@ -91,6 +91,7 @@ struct repo_config_values { /* section "core" config values */ char *attributes_file; int apply_sparse_checkout; + int trust_executable_bit; /* section "branch" config values */ enum branch_track branch_track; @@ -123,6 +124,8 @@ int git_default_config(const char *, const char *, int git_default_core_config(const char *var, const char *value, const struct config_context *ctx, void *cb); +int repo_trust_executable_bit(struct repository *repo); + void repo_config_values_init(struct repo_config_values *cfg); /* @@ -158,7 +161,6 @@ int is_bare_repository(void); extern char *git_work_tree_cfg; /* Environment bits from configuration mechanism */ -extern int trust_executable_bit; extern int trust_ctime; extern int check_stat; extern int has_symlinks; diff --git a/read-cache.c b/read-cache.c index a9f059174305eb..90789ff049edd0 100644 --- a/read-cache.c +++ b/read-cache.c @@ -209,7 +209,7 @@ static unsigned int st_mode_from_ce(const struct cache_entry *ce) case S_IFLNK: return has_symlinks ? S_IFLNK : (S_IFREG | 0644); case S_IFREG: - return (ce->ce_mode & (trust_executable_bit ? 0755 : 0644)) | S_IFREG; + return (ce->ce_mode & (repo_trust_executable_bit(the_repository) ? 0755 : 0644)) | S_IFREG; case S_IFGITLINK: return S_IFDIR | 0755; case S_IFDIR: @@ -319,7 +319,7 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st) /* We consider only the owner x bit to be relevant for * "mode changes" */ - if (trust_executable_bit && + if (repo_trust_executable_bit(the_repository) && (0100 & (ce->ce_mode ^ st->st_mode))) changed |= MODE_CHANGED; break; @@ -740,7 +740,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st, ce->ce_flags |= CE_INTENT_TO_ADD; - if (trust_executable_bit && has_symlinks) { + if (repo_trust_executable_bit(istate->repo) && has_symlinks) { ce->ce_mode = create_ce_mode(st_mode); } else { /* If there is an existing entry, pick the mode bits and type diff --git a/read-cache.h b/read-cache.h index af8c657ecbeb48..4b54cfc57c8de0 100644 --- a/read-cache.h +++ b/read-cache.h @@ -13,15 +13,15 @@ * This function handles degradation for filesystems that lack * symlink support or reliable executable bits. */ -static inline unsigned int ce_mode_from_stat(struct repository *repo UNUSED, +static inline unsigned int ce_mode_from_stat(struct repository *repo, const struct cache_entry *ce, unsigned int mode) { - extern int trust_executable_bit, has_symlinks; + extern int has_symlinks; if (S_ISREG(mode) && !has_symlinks && ce && S_ISLNK(ce->ce_mode)) return ce->ce_mode; - if (S_ISREG(mode) && !trust_executable_bit) { + if (S_ISREG(mode) && !repo_trust_executable_bit(repo)) { if (ce && S_ISREG(ce->ce_mode)) return ce->ce_mode; return create_ce_mode(0666); From df2bc04e6937ba35b9a905a65dd0137627b46c8b Mon Sep 17 00:00:00 2001 From: Tian Yuchen Date: Mon, 20 Jul 2026 18:53:35 +0800 Subject: [PATCH 30/47] environment: move has_symlinks into repo_config_values Move the global 'has_symlinks' configuration into the repository-specific 'repo_config_values' struct. Introduce 'repo_has_symlinks()' getter for readability. Callers access this configuration by passing in 'repo' when possible, and explicitly fall back to 'the_repository' the rest of the time. Introduce 'platform_has_symlinks()' macro to allow platform specific-customization, primarily to help MinGW. Platforms can override this in their respective headers. Mentored-by: Christian Couder Mentored-by: Ayush Chandekar Mentored-by: Olamide Caleb Bello Signed-off-by: Tian Yuchen Signed-off-by: Junio C Hamano --- apply.c | 2 +- combine-diff.c | 2 +- compat/mingw.c | 17 +++++++++++++---- compat/mingw.h | 3 +++ entry.c | 3 ++- environment.c | 12 ++++++++++-- environment.h | 4 +++- git-compat-util.h | 4 ++++ read-cache.c | 7 ++++--- read-cache.h | 3 +-- 10 files changed, 42 insertions(+), 15 deletions(-) diff --git a/apply.c b/apply.c index edb1502414e4be..b748192ee2c1f6 100644 --- a/apply.c +++ b/apply.c @@ -4511,7 +4511,7 @@ static int try_create_file(struct apply_state *state, const char *path, return !!mkdir(path, 0777); } - if (has_symlinks && S_ISLNK(mode)) + if (repo_has_symlinks(state->repo) && S_ISLNK(mode)) /* Although buf:size is counted string, it also is NUL * terminated. */ diff --git a/combine-diff.c b/combine-diff.c index b7998620687ed7..80e5c46e9b26ff 100644 --- a/combine-diff.c +++ b/combine-diff.c @@ -1078,7 +1078,7 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent, /* if symlinks don't work, assume symlink if all parents * are symlinks */ - is_file = has_symlinks; + is_file = repo_has_symlinks(rev->repo); for (i = 0; !is_file && i < num_parent; i++) is_file = !S_ISLNK(elem->parent[i].mode); if (!is_file) diff --git a/compat/mingw.c b/compat/mingw.c index aa7525f419cb64..47819119292d8e 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -7,6 +7,7 @@ #include "config.h" #include "dir.h" #include "environment.h" +#include "repository.h" #include "gettext.h" #include "run-command.h" #include "strbuf.h" @@ -1043,7 +1044,7 @@ int mingw_chdir(const char *dirname) if (xutftowcs_path(wdirname, dirname) < 0) return -1; - if (has_symlinks) { + if (repo_has_symlinks(the_repository)) { HANDLE hnd = CreateFileW(wdirname, 0, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL); @@ -2903,7 +2904,7 @@ int symlink(const char *target, const char *link) int len; /* fail if symlinks are disabled or API is not supported (WinXP) */ - if (!has_symlinks) { + if (!repo_has_symlinks(the_repository)) { errno = ENOSYS; return -1; } @@ -3173,15 +3174,23 @@ static void setup_windows_environment(void) if (!tmp && (tmp = getenv("USERPROFILE"))) setenv("HOME", tmp, 1); } +} +int mingw_platform_has_symlinks(void) +{ + static int has_symlinks = -1; /* * Change 'core.symlinks' default to false, unless native symlinks are * enabled in MSys2 (via 'MSYS=winsymlinks:nativestrict'). Thus we can * run the test suite (which doesn't obey config files) with or without * symlink support. */ - if (!(tmp = getenv("MSYS")) || !strstr(tmp, "winsymlinks:nativestrict")) - has_symlinks = 0; + if (has_symlinks < 0) { + const char *tmp = getenv("MSYS"); + has_symlinks = (tmp && strstr(tmp, "winsymlinks:nativestrict")) ? 1 : 0; + } + + return has_symlinks; } static void get_current_user_sid(PSID *sid, HANDLE *linked_token) diff --git a/compat/mingw.h b/compat/mingw.h index 444daedfa52469..df02aeb632d8c3 100644 --- a/compat/mingw.h +++ b/compat/mingw.h @@ -208,6 +208,9 @@ void open_in_gdb(void); */ int err_win_to_posix(DWORD winerr); +int mingw_platform_has_symlinks(void); +#define platform_has_symlinks() mingw_platform_has_symlinks() + #ifndef NO_UNIX_SOCKETS int mingw_have_unix_sockets(void); #undef have_unix_sockets diff --git a/entry.c b/entry.c index 7817aee362ed9e..5913a8b51f1895 100644 --- a/entry.c +++ b/entry.c @@ -321,7 +321,8 @@ static int write_entry(struct cache_entry *ce, char *path, struct conv_attrs *ca * We can't make a real symlink; write out a regular file entry * with the symlink destination as its contents. */ - if (!has_symlinks || to_tempfile) + if (!repo_has_symlinks(state->istate && state->istate->repo ? + state->istate->repo : the_repository) || to_tempfile) goto write_file_entry; ret = symlink(new_blob, path); diff --git a/environment.c b/environment.c index 32b110c40505be..e351043446001d 100644 --- a/environment.c +++ b/environment.c @@ -43,7 +43,6 @@ static int zlib_compression_seen; int trust_ctime = 1; int check_stat = 1; -int has_symlinks = 1; int minimum_abbrev = 4, default_abbrev = -1; int ignore_case; int assume_unchanged; @@ -148,6 +147,13 @@ int repo_trust_executable_bit(struct repository *repo) : 1; } +int repo_has_symlinks(struct repository *repo) +{ + return repo->initialized + ? repo_config_values(repo)->has_symlinks + : platform_has_symlinks(); +} + int have_git_dir(void) { return startup_info->have_repository @@ -336,7 +342,8 @@ int git_default_core_config(const char *var, const char *value, } if (!strcmp(var, "core.symlinks")) { - has_symlinks = git_config_bool(var, value); + struct repo_config_values *cfg = repo_config_values(the_repository); + cfg->has_symlinks = git_config_bool(var, value); return 0; } @@ -727,5 +734,6 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->attributes_file = NULL; cfg->apply_sparse_checkout = 0; cfg->trust_executable_bit = 1; + cfg->has_symlinks = platform_has_symlinks(); cfg->branch_track = BRANCH_TRACK_REMOTE; } diff --git a/environment.h b/environment.h index c15456fc0d0b39..8f54c481e92a06 100644 --- a/environment.h +++ b/environment.h @@ -92,6 +92,7 @@ struct repo_config_values { char *attributes_file; int apply_sparse_checkout; int trust_executable_bit; + int has_symlinks; /* section "branch" config values */ enum branch_track branch_track; @@ -126,6 +127,8 @@ int git_default_core_config(const char *var, const char *value, int repo_trust_executable_bit(struct repository *repo); +int repo_has_symlinks(struct repository *repo); + void repo_config_values_init(struct repo_config_values *cfg); /* @@ -163,7 +166,6 @@ extern char *git_work_tree_cfg; /* Environment bits from configuration mechanism */ extern int trust_ctime; extern int check_stat; -extern int has_symlinks; extern int minimum_abbrev, default_abbrev; extern int ignore_case; extern int assume_unchanged; diff --git a/git-compat-util.h b/git-compat-util.h index 88097764078538..a0f901ce793ed6 100644 --- a/git-compat-util.h +++ b/git-compat-util.h @@ -245,6 +245,10 @@ static inline int git_is_dir_sep(int c) #define is_dir_sep git_is_dir_sep #endif +#ifndef platform_has_symlinks +#define platform_has_symlinks() 1 +#endif + #ifndef offset_1st_component static inline int git_offset_1st_component(const char *path) { diff --git a/read-cache.c b/read-cache.c index 90789ff049edd0..046b1e649e7552 100644 --- a/read-cache.c +++ b/read-cache.c @@ -207,7 +207,7 @@ static unsigned int st_mode_from_ce(const struct cache_entry *ce) { switch (ce->ce_mode & S_IFMT) { case S_IFLNK: - return has_symlinks ? S_IFLNK : (S_IFREG | 0644); + return repo_has_symlinks(the_repository) ? S_IFLNK : (S_IFREG | 0644); case S_IFREG: return (ce->ce_mode & (repo_trust_executable_bit(the_repository) ? 0755 : 0644)) | S_IFREG; case S_IFGITLINK: @@ -325,7 +325,7 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st) break; case S_IFLNK: if (!S_ISLNK(st->st_mode) && - (has_symlinks || !S_ISREG(st->st_mode))) + (repo_has_symlinks(the_repository) || !S_ISREG(st->st_mode))) changed |= TYPE_CHANGED; break; case S_IFGITLINK: @@ -740,7 +740,8 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st, ce->ce_flags |= CE_INTENT_TO_ADD; - if (repo_trust_executable_bit(istate->repo) && has_symlinks) { + if (repo_trust_executable_bit(istate->repo) && + repo_has_symlinks(istate->repo)) { ce->ce_mode = create_ce_mode(st_mode); } else { /* If there is an existing entry, pick the mode bits and type diff --git a/read-cache.h b/read-cache.h index 4b54cfc57c8de0..ab9d40aa81f95e 100644 --- a/read-cache.h +++ b/read-cache.h @@ -17,8 +17,7 @@ static inline unsigned int ce_mode_from_stat(struct repository *repo, const struct cache_entry *ce, unsigned int mode) { - extern int has_symlinks; - if (S_ISREG(mode) && !has_symlinks && + if (S_ISREG(mode) && !repo_has_symlinks(repo) && ce && S_ISLNK(ce->ce_mode)) return ce->ce_mode; if (S_ISREG(mode) && !repo_trust_executable_bit(repo)) { From dd674df3267112248b0012ce614168055a416920 Mon Sep 17 00:00:00 2001 From: Ted Nyman Date: Mon, 20 Jul 2026 15:31:20 -0700 Subject: [PATCH 31/47] pathspec: use match for sparse-index expansion checks The pathspec parser computes `len` and `nowildcard_len` from `item.match`, which includes any prefix added when a command is run from a subdirectory. `item.original` can still contain the shorter, unprefixed argument. Using `item.original + item.nowildcard_len` in `pathspec_needs_expanded_index()` can therefore read past the end of the allocation. AddressSanitizer reports a heap-buffer-overflow for prefixed wildcard pathspecs passed to `git rm` and `git reset` with a sparse index. The mismatch dates back to 4d1cfc1351 ("reset: make --mixed sparse-aware", 2021-11-29), which introduced the helper using `item.original`. b29ad38322 ("pathspec.h: move pathspec_needs_expanded_index() from reset.c to here", 2022-08-07) later moved it to `pathspec.c` and preserved the affected comparisons. Use `item.match` consistently when checking whether a pathspec can match a sparse-directory entry. Add coverage for prefixed wildcard pathspecs so both commands keep the index sparse. Signed-off-by: Ted Nyman Reviewed-by: Taylor Blau Signed-off-by: Junio C Hamano --- pathspec.c | 12 ++++++------ t/t1092-sparse-checkout-compatibility.sh | 7 +++++++ 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/pathspec.c b/pathspec.c index f78b22709ccb67..281858f21f9c59 100644 --- a/pathspec.c +++ b/pathspec.c @@ -847,9 +847,9 @@ int pathspec_needs_expanded_index(struct index_state *istate, * - not-in-cone/bar*: may need expanded index * - **.c: may need expanded index */ - if (strspn(item.original + item.nowildcard_len, "*") == + if (strspn(item.match + item.nowildcard_len, "*") == (unsigned int)(item.len - item.nowildcard_len) && - path_in_cone_mode_sparse_checkout(item.original, istate)) + path_in_cone_mode_sparse_checkout(item.match, istate)) continue; for (pos = 0; pos < istate->cache_nr; pos++) { @@ -865,7 +865,7 @@ int pathspec_needs_expanded_index(struct index_state *istate, */ if ((unsigned int)item.nowildcard_len > ce_namelen(ce) && - !strncmp(item.original, ce->name, + !strncmp(item.match, ce->name, ce_namelen(ce))) { res = 1; break; @@ -876,13 +876,13 @@ int pathspec_needs_expanded_index(struct index_state *istate, * directory and the pathspec does not match the whole * directory, need to expand the index. */ - if (!strncmp(item.original, ce->name, item.nowildcard_len) && - wildmatch(item.original, ce->name, 0)) { + if (!strncmp(item.match, ce->name, item.nowildcard_len) && + wildmatch(item.match, ce->name, 0)) { res = 1; break; } } - } else if (!path_in_cone_mode_sparse_checkout(item.original, istate) && + } else if (!path_in_cone_mode_sparse_checkout(item.match, istate) && !matches_skip_worktree(pathspec, i, &skip_worktree_seen)) res = 1; diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 8186da5c887c56..d4a9c0bbf2d0fd 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -2119,6 +2119,13 @@ test_expect_success 'sparse index is not expanded: rm' ' ensure_not_expanded rm -r deep ' +test_expect_success 'sparse index is not expanded: prefixed wildcard pathspec' ' + init_repos && + + ensure_not_expanded -C deep rm --dry-run -- "a*" && + ensure_not_expanded -C deep reset base -- "a*" +' + test_expect_success 'grep with and --cached' ' init_repos && From 1be6ebe264031dc2175fec50c36df25fafd25084 Mon Sep 17 00:00:00 2001 From: Ted Nyman Date: Mon, 20 Jul 2026 15:31:21 -0700 Subject: [PATCH 32/47] stash: avoid sparse-index expansion for in-cone paths `git stash push -- ` expands a sparse index before checking whether the pathspec matches any tracked paths. This is unnecessary when the pathspec is wholly inside the sparse-checkout cone and makes a path-limited stash proportional to the size of the full index. Use `pathspec_needs_expanded_index()` to expand only when a pathspec can match part of a sparse-directory entry, as `git rm` and `git reset` already do. Keep the full-index behavior for pathspecs that need it. Add compatibility coverage for literal, prefixed, wildcard, file, multiple, staged, and missing pathspecs. Add the corresponding path-limited stash case to p2000. On a cone-mode repository with 349,525 tracked paths and 49 sparse index entries, the best of three runs changed from 18.87s to 0.06s. Trace2 reported four index expansions before this change and none after it. Signed-off-by: Ted Nyman Reviewed-by: Taylor Blau Signed-off-by: Junio C Hamano --- builtin/stash.c | 4 +- t/perf/p2000-sparse-operations.sh | 1 + t/t1092-sparse-checkout-compatibility.sh | 55 ++++++++++++++++++++++++ 3 files changed, 58 insertions(+), 2 deletions(-) diff --git a/builtin/stash.c b/builtin/stash.c index c4809f299a313b..72c52571f8c06c 100644 --- a/builtin/stash.c +++ b/builtin/stash.c @@ -1702,8 +1702,8 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q if (!include_untracked && ps->nr) { char *ps_matched = xcalloc(ps->nr, 1); - /* TODO: audit for interaction with sparse-index. */ - ensure_full_index(the_repository->index); + if (pathspec_needs_expanded_index(the_repository->index, ps)) + ensure_full_index(the_repository->index); for (size_t i = 0; i < the_repository->index->cache_nr; i++) ce_path_match(the_repository->index, the_repository->index->cache[i], ps, ps_matched); diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index aadf22bc2f0bb2..548a61cd9064bc 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -108,6 +108,7 @@ test_perf_on_all () { test_perf_on_all git status test_perf_on_all 'git stash && git stash pop' +test_perf_on_all "git stash push -- $SPARSE_CONE/a && git stash pop" test_perf_on_all 'echo >>new && git stash -u && git stash pop' test_perf_on_all git add -A test_perf_on_all git add . diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index d4a9c0bbf2d0fd..c3f7bfe551a247 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -1598,6 +1598,61 @@ test_expect_success 'sparse-index is not expanded: stash' ' ensure_not_expanded stash pop ' +test_expect_success 'sparse-index is not expanded: stash in-cone pathspec' ' + init_repos && + + echo unrelated >>sparse-index/deep/e && + echo literal >>sparse-index/deep/a && + ensure_not_expanded stash push -- deep/a && + test_grep ! literal sparse-index/deep/a && + test_grep unrelated sparse-index/deep/e && + ensure_not_expanded stash pop && + test_grep literal sparse-index/deep/a && + + echo prefixed >>sparse-index/deep/a && + ensure_not_expanded -C deep stash push -- a && + test_grep ! prefixed sparse-index/deep/a && + test_grep unrelated sparse-index/deep/e && + ensure_not_expanded stash pop && + test_grep prefixed sparse-index/deep/a && + + echo wildcard >>sparse-index/deep/a && + ensure_not_expanded stash push -- "deep/a*" && + test_grep ! wildcard sparse-index/deep/a && + test_grep unrelated sparse-index/deep/e && + ensure_not_expanded stash pop && + test_grep wildcard sparse-index/deep/a && + + echo pathspec-file >>sparse-index/deep/a && + echo deep/a >pathspec-file && + ensure_not_expanded stash push --pathspec-from-file=../pathspec-file && + test_grep ! pathspec-file sparse-index/deep/a && + test_grep unrelated sparse-index/deep/e && + ensure_not_expanded stash pop && + test_grep pathspec-file sparse-index/deep/a && + + echo multiple-a >>sparse-index/deep/a && + echo multiple-e >>sparse-index/deep/e && + ensure_not_expanded stash push -- deep/a deep/e && + test_grep ! multiple-a sparse-index/deep/a && + test_grep ! multiple-e sparse-index/deep/e && + ensure_not_expanded stash pop && + test_grep multiple-a sparse-index/deep/a && + test_grep multiple-e sparse-index/deep/e && + + echo staged >>sparse-index/deep/a && + git -C sparse-index add deep/a && + ensure_not_expanded stash push --staged -- deep/a && + test_grep ! staged sparse-index/deep/a && + test_grep unrelated sparse-index/deep/e && + ensure_not_expanded stash pop --index && + test_grep staged sparse-index/deep/a && + test_must_fail git -C sparse-index diff --cached --quiet -- deep/a && + + ensure_not_expanded ! stash push -- deep/does-not-exist && + test_grep "did not match any file" sparse-index-error +' + test_expect_success 'describe tested on all' ' init_repos && From 113d62b141a80bf136268df51b4597dbec4355d0 Mon Sep 17 00:00:00 2001 From: Shlok Kulshreshtha Date: Tue, 21 Jul 2026 12:27:36 +0530 Subject: [PATCH 33/47] userdiff: add support for Swift Add a built-in userdiff driver for the Swift programming language so that diff hunk headers and word diffs work out of the box for ".swift" files. The funcname pattern is built for Swift's own declaration grammar: an optional run of attributes ("@objc", "@available(iOS 13, *)", ...), followed by an optional run of lowercase modifiers ("public", "static", "final", ...), followed by a declaration keyword (func, class, struct, enum, protocol, extension, actor, init, deinit, subscript). The keyword is followed by a boundary that allows whitespace, "(" (init/subscript), "?" or "!" (failable init), or "<" (generics), while still acting as a word boundary so e.g. "initialize(" does not match. The word regex recognizes Swift identifiers, hexadecimal, octal, binary, integer and floating-point literals, and the language's operators. Signed-off-by: Shlok Kulshreshtha Acked-by: Johannes Sixt Signed-off-by: Junio C Hamano --- Documentation/gitattributes.adoc | 2 ++ t/t4018/swift-actor | 5 +++++ t/t4018/swift-attribute-with-args | 7 +++++++ t/t4018/swift-class | 5 +++++ t/t4018/swift-enum | 5 +++++ t/t4018/swift-extension | 5 +++++ t/t4018/swift-failable-init | 7 +++++++ t/t4018/swift-func | 5 +++++ t/t4018/swift-generic-subscript | 7 +++++++ t/t4018/swift-init | 7 +++++++ t/t4018/swift-inline-attribute | 7 +++++++ t/t4018/swift-modifiers | 4 ++++ t/t4018/swift-protocol | 5 +++++ t/t4018/swift-struct | 5 +++++ userdiff.c | 10 ++++++++++ 15 files changed, 86 insertions(+) create mode 100644 t/t4018/swift-actor create mode 100644 t/t4018/swift-attribute-with-args create mode 100644 t/t4018/swift-class create mode 100644 t/t4018/swift-enum create mode 100644 t/t4018/swift-extension create mode 100644 t/t4018/swift-failable-init create mode 100644 t/t4018/swift-func create mode 100644 t/t4018/swift-generic-subscript create mode 100644 t/t4018/swift-init create mode 100644 t/t4018/swift-inline-attribute create mode 100644 t/t4018/swift-modifiers create mode 100644 t/t4018/swift-protocol create mode 100644 t/t4018/swift-struct diff --git a/Documentation/gitattributes.adoc b/Documentation/gitattributes.adoc index bd76167a45eb71..9fea75f96f928f 100644 --- a/Documentation/gitattributes.adoc +++ b/Documentation/gitattributes.adoc @@ -914,6 +914,8 @@ patterns are available: - `scheme` suitable for source code in most Lisp dialects, including Scheme, Emacs Lisp, Common Lisp, and Clojure. +- `swift` suitable for source code in the Swift language. + - `tex` suitable for source code for LaTeX documents. diff --git a/t/t4018/swift-actor b/t/t4018/swift-actor new file mode 100644 index 00000000000000..e4852f40a7ebd8 --- /dev/null +++ b/t/t4018/swift-actor @@ -0,0 +1,5 @@ +actor RIGHT { + let a = 1 + // a comment + let b = ChangeMe +} diff --git a/t/t4018/swift-attribute-with-args b/t/t4018/swift-attribute-with-args new file mode 100644 index 00000000000000..22b1ee32f110bd --- /dev/null +++ b/t/t4018/swift-attribute-with-args @@ -0,0 +1,7 @@ +struct View { + @available(iOS 13, *) public func RIGHT() { + let a = 1 + // a comment + print(ChangeMe) + } +} diff --git a/t/t4018/swift-class b/t/t4018/swift-class new file mode 100644 index 00000000000000..c3a93360277f81 --- /dev/null +++ b/t/t4018/swift-class @@ -0,0 +1,5 @@ +class RIGHT { + let a = 1 + // a comment + let b = ChangeMe +} diff --git a/t/t4018/swift-enum b/t/t4018/swift-enum new file mode 100644 index 00000000000000..0a843029930409 --- /dev/null +++ b/t/t4018/swift-enum @@ -0,0 +1,5 @@ +enum RIGHT { + case first + // a comment + case ChangeMe +} diff --git a/t/t4018/swift-extension b/t/t4018/swift-extension new file mode 100644 index 00000000000000..cbc18ab6ef42c8 --- /dev/null +++ b/t/t4018/swift-extension @@ -0,0 +1,5 @@ +extension RIGHT { + static let a = 1 + // a comment + static let b = ChangeMe +} diff --git a/t/t4018/swift-failable-init b/t/t4018/swift-failable-init new file mode 100644 index 00000000000000..4bbd6217c9147a --- /dev/null +++ b/t/t4018/swift-failable-init @@ -0,0 +1,7 @@ +class Bar { + init?(RIGHT: Int) { + let x = 0 + // a comment + print(ChangeMe) + } +} diff --git a/t/t4018/swift-func b/t/t4018/swift-func new file mode 100644 index 00000000000000..1fecae0911d411 --- /dev/null +++ b/t/t4018/swift-func @@ -0,0 +1,5 @@ +func RIGHT(x: Int) -> Int { + let y = x + // a comment + return ChangeMe +} diff --git a/t/t4018/swift-generic-subscript b/t/t4018/swift-generic-subscript new file mode 100644 index 00000000000000..423cb589412ea5 --- /dev/null +++ b/t/t4018/swift-generic-subscript @@ -0,0 +1,7 @@ +struct Container { + subscript(index: Int) -> Int { + let a = 0 + // a comment + return ChangeMe + } +} diff --git a/t/t4018/swift-init b/t/t4018/swift-init new file mode 100644 index 00000000000000..dc7a298f3806b5 --- /dev/null +++ b/t/t4018/swift-init @@ -0,0 +1,7 @@ +class Foo { + init(RIGHT: Int) { + let x = 0 + // a comment + print(ChangeMe) + } +} diff --git a/t/t4018/swift-inline-attribute b/t/t4018/swift-inline-attribute new file mode 100644 index 00000000000000..2374c4b6036efa --- /dev/null +++ b/t/t4018/swift-inline-attribute @@ -0,0 +1,7 @@ +class Service { + @objc func RIGHT() { + let path = "/api" + // a comment + log(ChangeMe) + } +} diff --git a/t/t4018/swift-modifiers b/t/t4018/swift-modifiers new file mode 100644 index 00000000000000..9d80685a786bea --- /dev/null +++ b/t/t4018/swift-modifiers @@ -0,0 +1,4 @@ +public static func RIGHT() -> Int { + // a comment + return ChangeMe +} diff --git a/t/t4018/swift-protocol b/t/t4018/swift-protocol new file mode 100644 index 00000000000000..07c39ec2a3ca89 --- /dev/null +++ b/t/t4018/swift-protocol @@ -0,0 +1,5 @@ +protocol RIGHT { + var first: Int { get } + // a comment + var second: ChangeMe { get } +} diff --git a/t/t4018/swift-struct b/t/t4018/swift-struct new file mode 100644 index 00000000000000..e399ed77597656 --- /dev/null +++ b/t/t4018/swift-struct @@ -0,0 +1,5 @@ +struct RIGHT { + let a = 1 + // a comment + let b = ChangeMe +} diff --git a/userdiff.c b/userdiff.c index b5412e6bc3ecd3..7129bf148266b7 100644 --- a/userdiff.c +++ b/userdiff.c @@ -362,6 +362,16 @@ PATTERNS("scheme", "\\|([^|\\\\]|\\\\.)*\\|" /* All other words should be delimited by spaces or parentheses. */ "|([^][)(}{ \t])+"), +PATTERNS("swift", + "^[ \t]*((@[A-Za-z_][A-Za-z0-9_]*(\\([^()]*\\))?[ \t]+)*([a-z]+[ \t]+)*(func|init|deinit|subscript|class|struct|enum|protocol|extension|actor)[ \t(?!<].*)$", + /* -- */ + "[a-zA-Z_][a-zA-Z0-9_]*" + /* hexadecimal, octal, and binary literals */ + "|0[xX][0-9a-fA-F_]+|0[oO][0-7_]+|0[bB][01_]+" + /* integers and floating-point numbers */ + "|[0-9][0-9_]*([.][0-9_]+)?([eE][-+]?[0-9]+)?" + /* unary and binary operators */ + "|[-+*/%<>=!&|^~?]=|&&|\\|\\||<<=?|>>=?|\\?\\?|\\.\\.[.<]|->"), PATTERNS("tex", "^(\\\\((sub)*section|chapter|part)\\*{0,1}\\{.*)$", "\\\\[a-zA-Z@]+|\\\\.|([a-zA-Z0-9]|[^\x01-\x7f])+"), { .name = "default", .binary = -1 }, From 840eb9a1c54f76b1e16b249eb3f4240c3622daf3 Mon Sep 17 00:00:00 2001 From: Pablo Sabater Date: Fri, 24 Jul 2026 12:54:12 +0200 Subject: [PATCH 34/47] transport-helper: fix memory leak of helper on disconnect disconnect_helper() only frees data inside of the if(data->helper) block [1]. When the transport is disconnected without the helper being fully started, data->name allocated in transport_helper_init() is never freed. Move FREE_AND_NULL(data->name) outside the conditional block so it's always freed on disconnect. [1]: https://lore.kernel.org/git/05fbadbae2184479c87c37675dde7bd79b3e32ab.1716465556.git.ps@pks.im/ Mentored-by: Karthik Nayak Mentored-by: Chandra Pratap Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- transport-helper.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/transport-helper.c b/transport-helper.c index 80f90eb7bace6f..f1950707883dbc 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -266,9 +266,9 @@ static int disconnect_helper(struct transport *transport) close(data->helper->out); fclose(data->out); res = finish_command(data->helper); - FREE_AND_NULL(data->name); FREE_AND_NULL(data->helper); } + FREE_AND_NULL(data->name); return res; } From f05e099dea8ceeae5990f77ed6f4dc91d4be4694 Mon Sep 17 00:00:00 2001 From: Eric Ju Date: Fri, 24 Jul 2026 12:54:13 +0200 Subject: [PATCH 35/47] cat-file: declare loop counter inside for() Declare loop counters in the for statement when they are only used within the loop body, limiting their scope and improving readability. While updating the loop counters, use size_t instead of int for counters that iterate over object counts. Update the 'nr' parameter of dispatch_calls() to size_t as all callers already pass a value of that type. Helped-by: Christian Couder Signed-off-by: Eric Ju Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- builtin/cat-file.c | 13 ++++--------- fetch-pack.c | 3 +-- 2 files changed, 5 insertions(+), 11 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index b4b99a73da9696..03afc44c5ed482 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -721,14 +721,12 @@ static void dispatch_calls(struct batch_options *opt, struct strbuf *output, struct expand_data *data, struct queued_cmd *cmd, - int nr) + size_t nr) { - int i; - if (!opt->buffer_output) die(_("flush is only for --buffer mode")); - for (i = 0; i < nr; i++) + for (size_t i = 0; i < nr; i++) cmd[i].fn(opt, cmd[i].line, output, data); fflush(stdout); @@ -736,9 +734,7 @@ static void dispatch_calls(struct batch_options *opt, static void free_cmds(struct queued_cmd *cmd, size_t *nr) { - size_t i; - - for (i = 0; i < *nr; i++) + for (size_t i = 0; i < *nr; i++) FREE_AND_NULL(cmd[i].line); *nr = 0; @@ -765,7 +761,6 @@ static void batch_objects_command(struct batch_options *opt, size_t alloc = 0, nr = 0; while (strbuf_getdelim_strip_crlf(&input, stdin, opt->input_delim) != EOF) { - int i; const struct parse_cmd *cmd = NULL; const char *p = NULL, *cmd_end; struct queued_cmd call = {0}; @@ -775,7 +770,7 @@ static void batch_objects_command(struct batch_options *opt, if (isspace(*input.buf)) die(_("whitespace before command: '%s'"), input.buf); - for (i = 0; i < ARRAY_SIZE(commands); i++) { + for (size_t i = 0; i < ARRAY_SIZE(commands); i++) { if (!skip_prefix(input.buf, commands[i].name, &cmd_end)) continue; diff --git a/fetch-pack.c b/fetch-pack.c index 29c41132ee0495..9eb8fc539986b7 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1388,9 +1388,8 @@ static void write_fetch_command_and_capabilities(struct strbuf *req_buf, if (advertise_sid && server_supports_v2("session-id")) packet_buf_write(req_buf, "session-id=%s", trace2_session_id()); if (server_options && server_options->nr) { - int i; ensure_server_supports_v2("server-option"); - for (i = 0; i < server_options->nr; i++) + for (size_t i = 0; i < server_options->nr; i++) packet_buf_write(req_buf, "server-option=%s", server_options->items[i].string); } From fb60c49d18d1d52b6b81680586754168bc3ebc2c Mon Sep 17 00:00:00 2001 From: Eric Ju Date: Fri, 24 Jul 2026 12:54:14 +0200 Subject: [PATCH 36/47] t1006: extract helper functions into new 'lib-cat-file.sh' Extract utility functions from the cat-file's test script 't1006-cat-file.sh' into a new 'lib-cat-file.sh' dedicated library file. A subsequent commit will need these functions. This improves the code reuse and readability, enabling future cat-file tests to share these helpers without duplicating code. While at it update the style of this line to follow coding guidelines: . "$TEST_DIRECTORY/lib-loose.sh" to . "$TEST_DIRECTORY"/lib-loose.sh Signed-off-by: Eric Ju Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- t/lib-cat-file.sh | 16 ++++++++++++++++ t/t1006-cat-file.sh | 15 ++------------- 2 files changed, 18 insertions(+), 13 deletions(-) create mode 100644 t/lib-cat-file.sh diff --git a/t/lib-cat-file.sh b/t/lib-cat-file.sh new file mode 100644 index 00000000000000..7c2e8770164ce3 --- /dev/null +++ b/t/lib-cat-file.sh @@ -0,0 +1,16 @@ +# Library of git-cat-file related test functions. + +# Print a string without a trailing newline. +echo_without_newline () { + printf '%s' "$*" +} + +# Print a string without newlines and replace them with a NUL character (\0). +echo_without_newline_nul () { + echo_without_newline "$@" | tr '\n' '\0' +} + +# Calculate the length of a string. +strlen () { + echo_without_newline "$1" | wc -c | sed -e 's/^ *//' +} diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 8e2c52652c5185..cf65bfc88f14eb 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -3,7 +3,8 @@ test_description='git cat-file' . ./test-lib.sh -. "$TEST_DIRECTORY/lib-loose.sh" +. "$TEST_DIRECTORY"/lib-loose.sh +. "$TEST_DIRECTORY"/lib-cat-file.sh test_cmdmode_usage () { test_expect_code 129 "$@" 2>err && @@ -99,18 +100,6 @@ do ' done -echo_without_newline () { - printf '%s' "$*" -} - -echo_without_newline_nul () { - echo_without_newline "$@" | tr '\n' '\0' -} - -strlen () { - echo_without_newline "$1" | wc -c | sed -e 's/^ *//' -} - run_tests () { type=$1 object_name="$2" From b54d5e19f068a73bdbde5a9434fe72902764b958 Mon Sep 17 00:00:00 2001 From: Pablo Sabater Date: Fri, 24 Jul 2026 12:54:15 +0200 Subject: [PATCH 37/47] fetch-pack: drop the static advertise_sid variable write_fetch_command_and_capabilities() is moved to 'connect.c' in a subsequent commit. To prepare for that, drop the static variable usage of advertise_sid. Currently advertise_sid is set in fetch_pack_config() by reading "transfer.advertisesid". It is used in three places: 1. In do_fetch_pack(), to clear it when the server lacks support: if (!server_supports("session-id")) advertise_sid = 0; 2. In find_common(), to advertise the session id over protocol v0/v1: if (advertise_sid) strbuf_addf(&c, " session-id=%s", trace2_session_id()); 3. In write_fetch_command_and_capabilities(), to advertise it over protocol v2: if (advertise_sid && server_supports_v2("session-id")) packet_buf_write(req_buf, "session-id=%s", trace2_session_id()); About 1, the check only guards the v0/v1 path, and the v2 path already checks server support inline in its condition. Follow the same pattern and fold the check into the condition in find_common(). About 2 and 3, replace the static variable with a local read via repo_config_get_bool() in each function. Because repo_config_get_bool() leaves advertise_sid as is if it is not set, initialize it to 0, matching its default. Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- fetch-pack.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 9eb8fc539986b7..65ebfec09fc62d 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -49,7 +49,6 @@ static int fetch_fsck_objects = -1; static int transfer_fsck_objects = -1; static int agent_supported; static int server_supports_filtering; -static int advertise_sid; static struct shallow_lock shallow_lock; static const char *alternate_shallow_file; static struct strbuf fsck_msg_types = STRBUF_INIT; @@ -363,6 +362,9 @@ static int find_common(struct fetch_negotiator *negotiator, size_t state_len = 0; struct packet_reader reader; struct oidset negotiation_include_oids = OIDSET_INIT; + int advertise_sid = 0; + + repo_config_get_bool(the_repository, "transfer.advertisesid", &advertise_sid); if (args->stateless_rpc && multi_ack == 1) die(_("the option '%s' requires '%s'"), "--stateless-rpc", "multi_ack_detailed"); @@ -414,7 +416,7 @@ static int find_common(struct fetch_negotiator *negotiator, if (deepen_not_ok) strbuf_addstr(&c, " deepen-not"); if (agent_supported) strbuf_addf(&c, " agent=%s", git_user_agent_sanitized()); - if (advertise_sid) + if (advertise_sid && server_supports("session-id")) strbuf_addf(&c, " session-id=%s", trace2_session_id()); if (args->filter_options.choice) strbuf_addstr(&c, " filter"); @@ -1160,9 +1162,6 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args, (int)agent_len, agent_feature); } - if (!server_supports("session-id")) - advertise_sid = 0; - if (server_supports("shallow")) print_verbose(args, _("Server supports %s"), "shallow"); else if (args->depth > 0 || is_repository_shallow(r)) @@ -1380,6 +1379,9 @@ static void write_fetch_command_and_capabilities(struct strbuf *req_buf, const struct string_list *server_options) { const char *hash_name; + int advertise_sid = 0; + + repo_config_get_bool(the_repository, "transfer.advertisesid", &advertise_sid); ensure_server_supports_v2("fetch"); packet_buf_write(req_buf, "command=fetch"); @@ -1998,7 +2000,6 @@ static void fetch_pack_config(void) repo_config_get_bool(the_repository, "repack.usedeltabaseoffset", &prefer_ofs_delta); repo_config_get_bool(the_repository, "fetch.fsckobjects", &fetch_fsck_objects); repo_config_get_bool(the_repository, "transfer.fsckobjects", &transfer_fsck_objects); - repo_config_get_bool(the_repository, "transfer.advertisesid", &advertise_sid); if (!uri_protocols.nr) { char *str; From b72559caa4ca3733b7694fa27364c4a9ac364159 Mon Sep 17 00:00:00 2001 From: Pablo Sabater Date: Fri, 24 Jul 2026 12:54:16 +0200 Subject: [PATCH 38/47] fetch-pack: use unsigned int for hash_algo variable hash_algo_by_name() returns unsigned int, but it is stored in hash_algo variable as int. This goes unnoticed because of: DISABLE_SIGN_COMPARE_WARNINGS On 'fetch-pack.c' On a subsequent commit this function will be moved to 'connect.c' that would notice this. Change hash_algo variable type to match its return type, also make it const because they are never modified. Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- fetch-pack.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fetch-pack.c b/fetch-pack.c index 65ebfec09fc62d..f1e64160fc92a6 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1397,7 +1397,7 @@ static void write_fetch_command_and_capabilities(struct strbuf *req_buf, } if (server_feature_v2("object-format", &hash_name)) { - int hash_algo = hash_algo_by_name(hash_name); + const unsigned int hash_algo = hash_algo_by_name(hash_name); if (hash_algo_by_ptr(the_hash_algo) != hash_algo) die(_("mismatched algorithms: client %s; server %s"), the_hash_algo->name, hash_name); From 40865f70d44d0665b0a0bd59acb8e18091d505a9 Mon Sep 17 00:00:00 2001 From: Pablo Sabater Date: Fri, 24 Jul 2026 12:54:17 +0200 Subject: [PATCH 39/47] fetch-pack: move write_fetch_command_and_capabilities() to connect.c In a subsequent commit write_fetch_command_and_capabilities() will be refactored to a more general-purpose function, making it more accessible to additional commands in the future. Move write_fetch_command_and_capabilities() to 'connect.c', where there are similar purpose functions. Because string_list is only used as a pointer, use a forward declaration [1]. [1]: https://lore.kernel.org/git/Z0RIqUAoEob8lGfM@pks.im/ Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- connect.c | 34 ++++++++++++++++++++++++++++++++++ connect.h | 4 ++++ fetch-pack.c | 34 ---------------------------------- 3 files changed, 38 insertions(+), 34 deletions(-) diff --git a/connect.c b/connect.c index 47e39d2a7316ae..c09947cc5615bc 100644 --- a/connect.c +++ b/connect.c @@ -700,6 +700,40 @@ int server_supports(const char *feature) return !!server_feature_value(feature, NULL); } +void write_fetch_command_and_capabilities(struct strbuf *req_buf, + const struct string_list *server_options) +{ + const char *hash_name; + int advertise_sid = 0; + + repo_config_get_bool(the_repository, "transfer.advertisesid", &advertise_sid); + + ensure_server_supports_v2("fetch"); + packet_buf_write(req_buf, "command=fetch"); + if (server_supports_v2("agent")) + packet_buf_write(req_buf, "agent=%s", git_user_agent_sanitized()); + if (advertise_sid && server_supports_v2("session-id")) + packet_buf_write(req_buf, "session-id=%s", trace2_session_id()); + if (server_options && server_options->nr) { + ensure_server_supports_v2("server-option"); + for (size_t i = 0; i < server_options->nr; i++) + packet_buf_write(req_buf, "server-option=%s", + server_options->items[i].string); + } + + if (server_feature_v2("object-format", &hash_name)) { + const unsigned int hash_algo = hash_algo_by_name(hash_name); + if (hash_algo_by_ptr(the_hash_algo) != hash_algo) + die(_("mismatched algorithms: client %s; server %s"), + the_hash_algo->name, hash_name); + packet_buf_write(req_buf, "object-format=%s", the_hash_algo->name); + } else if (hash_algo_by_ptr(the_hash_algo) != GIT_HASH_SHA1_LEGACY) { + die(_("the server does not support algorithm '%s'"), + the_hash_algo->name); + } + packet_buf_delim(req_buf); +} + static const char *url_scheme_name(enum url_scheme scheme) { switch (scheme) { diff --git a/connect.h b/connect.h index aa482a37fb4da4..c4f6ea4b0adcbf 100644 --- a/connect.h +++ b/connect.h @@ -34,4 +34,8 @@ void check_stateless_delimiter(int stateless_rpc, struct packet_reader *reader, const char *error); +struct string_list; +void write_fetch_command_and_capabilities(struct strbuf *req_buf, + const struct string_list *server_options); + #endif diff --git a/fetch-pack.c b/fetch-pack.c index f1e64160fc92a6..f7789e8456eb38 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1375,40 +1375,6 @@ static int add_haves(struct fetch_negotiator *negotiator, return haves_added; } -static void write_fetch_command_and_capabilities(struct strbuf *req_buf, - const struct string_list *server_options) -{ - const char *hash_name; - int advertise_sid = 0; - - repo_config_get_bool(the_repository, "transfer.advertisesid", &advertise_sid); - - ensure_server_supports_v2("fetch"); - packet_buf_write(req_buf, "command=fetch"); - if (server_supports_v2("agent")) - packet_buf_write(req_buf, "agent=%s", git_user_agent_sanitized()); - if (advertise_sid && server_supports_v2("session-id")) - packet_buf_write(req_buf, "session-id=%s", trace2_session_id()); - if (server_options && server_options->nr) { - ensure_server_supports_v2("server-option"); - for (size_t i = 0; i < server_options->nr; i++) - packet_buf_write(req_buf, "server-option=%s", - server_options->items[i].string); - } - - if (server_feature_v2("object-format", &hash_name)) { - const unsigned int hash_algo = hash_algo_by_name(hash_name); - if (hash_algo_by_ptr(the_hash_algo) != hash_algo) - die(_("mismatched algorithms: client %s; server %s"), - the_hash_algo->name, hash_name); - packet_buf_write(req_buf, "object-format=%s", the_hash_algo->name); - } else if (hash_algo_by_ptr(the_hash_algo) != GIT_HASH_SHA1_LEGACY) { - die(_("the server does not support algorithm '%s'"), - the_hash_algo->name); - } - packet_buf_delim(req_buf); -} - static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, struct fetch_pack_args *args, const struct ref *wants, struct oidset *common, From eb1fbc64b705deaf600bbba777616209280f43a5 Mon Sep 17 00:00:00 2001 From: Pablo Sabater Date: Fri, 24 Jul 2026 12:54:18 +0200 Subject: [PATCH 40/47] connect: make write_fetch_command_and_capabilities() more generic Refactor write_fetch_command_and_capabilities(), enabling it to serve both fetch and additional commands. In this context, "command" refers to the "operations" supported by Git's wire protocol Documentation/gitprotocol-v2.adoc, such as a Git subcommand (e.g., git-fetch(1)) or a server-side operation like "object-info" as implemented in commit a2ba162cda (object-info: support for retrieving object info, 2021-04-20). Refactor the function signature to accept a command instead of the hardcoded "fetch". Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- connect.c | 8 ++++---- connect.h | 8 ++++++-- fetch-pack.c | 4 ++-- 3 files changed, 12 insertions(+), 8 deletions(-) diff --git a/connect.c b/connect.c index c09947cc5615bc..127ed4a2e6b4c0 100644 --- a/connect.c +++ b/connect.c @@ -700,16 +700,16 @@ int server_supports(const char *feature) return !!server_feature_value(feature, NULL); } -void write_fetch_command_and_capabilities(struct strbuf *req_buf, - const struct string_list *server_options) +void write_command_and_capabilities(struct strbuf *req_buf, const char *command, + const struct string_list *server_options) { const char *hash_name; int advertise_sid = 0; repo_config_get_bool(the_repository, "transfer.advertisesid", &advertise_sid); - ensure_server_supports_v2("fetch"); - packet_buf_write(req_buf, "command=fetch"); + ensure_server_supports_v2(command); + packet_buf_write(req_buf, "command=%s", command); if (server_supports_v2("agent")) packet_buf_write(req_buf, "agent=%s", git_user_agent_sanitized()); if (advertise_sid && server_supports_v2("session-id")) diff --git a/connect.h b/connect.h index c4f6ea4b0adcbf..957e5fe2b9eccb 100644 --- a/connect.h +++ b/connect.h @@ -35,7 +35,11 @@ void check_stateless_delimiter(int stateless_rpc, const char *error); struct string_list; -void write_fetch_command_and_capabilities(struct strbuf *req_buf, - const struct string_list *server_options); +/* + * Write a protocol v2 command request, along with the capability + * advertisements, into req_buf. + */ +void write_command_and_capabilities(struct strbuf *req_buf, const char *command, + const struct string_list *server_options); #endif diff --git a/fetch-pack.c b/fetch-pack.c index f7789e8456eb38..3695059cd55b03 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1386,7 +1386,7 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, int done_sent = 0; struct strbuf req_buf = STRBUF_INIT; - write_fetch_command_and_capabilities(&req_buf, args->server_options); + write_command_and_capabilities(&req_buf, "fetch", args->server_options); if (args->use_thin_pack) packet_buf_write(&req_buf, "thin-pack"); @@ -2253,7 +2253,7 @@ void negotiate_using_fetch(const struct oid_array *negotiation_restrict_tips, the_repository, "%d", negotiation_round); strbuf_reset(&req_buf); - write_fetch_command_and_capabilities(&req_buf, server_options); + write_command_and_capabilities(&req_buf, "fetch", server_options); packet_buf_write(&req_buf, "wait-for-done"); From 24f082e6f45e39dc36f55a3344696d707740f56e Mon Sep 17 00:00:00 2001 From: Calvin Wan Date: Fri, 24 Jul 2026 12:54:19 +0200 Subject: [PATCH 41/47] fetch-pack: move fetch initialization There are some variables initialized at the start of the do_fetch_pack_v2() state machine. Currently, they are initialized in FETCH_CHECK_LOCAL, which is the initial state set at the beginning of the function. However, a subsequent patch will allow for another initial state, while still requiring these initialized variables. Move the initialization to be before the state machine, so that they are set regardless of the initial state. Note that there is no change in behavior, because we're moving code from the beginning of the first state to just before the execution of the state machine. Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- fetch-pack.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 3695059cd55b03..922a9b25812c68 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1735,18 +1735,18 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, reader.me = "fetch-pack"; } + /* v2 supports these by default */ + allow_unadvertised_object_request |= ALLOW_REACHABLE_SHA1; + use_sideband = 2; + if (args->depth > 0 || args->deepen_since || args->deepen_not) + args->deepen = 1; + while (state != FETCH_DONE) { switch (state) { case FETCH_CHECK_LOCAL: sort_ref_list(&ref, ref_compare_name); QSORT(sought, nr_sought, cmp_ref_by_name); - /* v2 supports these by default */ - allow_unadvertised_object_request |= ALLOW_REACHABLE_SHA1; - use_sideband = 2; - if (args->depth > 0 || args->deepen_since || args->deepen_not) - args->deepen = 1; - /* Filter 'ref' by 'sought' and those that aren't local */ mark_complete_and_common_ref(negotiator, args, &ref); filter_refs(args, &ref, sought, nr_sought); From e44aca614596d49ca802f1602eaf9a4850c3c07d Mon Sep 17 00:00:00 2001 From: Pablo Sabater Date: Fri, 24 Jul 2026 12:54:20 +0200 Subject: [PATCH 42/47] protocol-caps: check object existence regardless of the attributes requested Currently, send_info() only checks for existence when the attribute 'size' is also requested. Requesting a bare OID, without attributes only echoes back the OID. Extract the existence check to be done regardless of the number of attributes requested. While at it, introduce a wrapper called get_object_info() similar to odb_read_object_info() that returns OBJ_BAD on fail and adds OBJECT_INFO_SKIP_FETCH_OBJECT and OBJECT_INFO_QUICK flags. OBJECT_INFO_SKIP_FETCH_OBJECT is so a server with a partial clone doesn't trigger fetching objects when it gets an object-info request with an OID that is not available locally. A server should only report what it has locally. Tighten the condition used to determine whether an object is recognized. get_object_info() returns OBJ_BAD for unknown objects, but OBJ_NONE (0) can also mean "not found". Change the check from '< 0' to '<= OBJ_NONE' to cover both as unrecognized. With this patch, a bare OID has two possible responses: 1. Recognized OID: the server answers with "" 2. Unrecognized OID: the server answers with " SP" Update the object-info section in 'gitprotocol-v2.adoc': - Require full obj-oid explicitly. - Fix parentheses. - Define obj-size explicitly. - Make obj-size optional in obj-info and document the behavior for unrecognized object IDs. - Describe the attr header as zero or more pkt-lines, one per attribute, matching what the server implements. A request with no attributes gets no header. Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- Documentation/gitprotocol-v2.adoc | 21 +++++++---- protocol-caps.c | 45 +++++++++++++++++++--- t/t5701-git-serve.sh | 63 +++++++++++++++++++++++++++++++ 3 files changed, 115 insertions(+), 14 deletions(-) diff --git a/Documentation/gitprotocol-v2.adoc b/Documentation/gitprotocol-v2.adoc index 2beb70595fc1e5..7bf62014c3917d 100644 --- a/Documentation/gitprotocol-v2.adoc +++ b/Documentation/gitprotocol-v2.adoc @@ -568,21 +568,26 @@ An `object-info` request takes the following arguments: oid Indicates to the server an object which the client wants to obtain - information for. + information for. They must be full OIDs. -The response of `object-info` is a list of the requested object ids -and associated requested information, each separated by a single space. +The response of `object-info` consists of one pkt-line per requested attribute, +echoing the attributes the server will report, followed by one pkt-line per +requested object id with its information, each field separated by a single +space. output = info flush-pkt - info = PKT-LINE(attrs) LF) - *PKT-LINE(obj-info LF) - - attrs = attr | attrs SP attrs + info = *PKT-LINE(attr LF) + *PKT-LINE(obj-info LF) attr = "size" - obj-info = obj-id SP obj-size + obj-size = 1*DIGIT + + obj-info = obj-id [SP [obj-size]] + +If the server does not recognize the OID, the response will be ` SP` +regardless of the number of attributes requested. bundle-uri ~~~~~~~~~~ diff --git a/protocol-caps.c b/protocol-caps.c index 8858ea4489d714..02261be14d817a 100644 --- a/protocol-caps.c +++ b/protocol-caps.c @@ -30,6 +30,32 @@ static int parse_oid(const char *line, struct string_list *oid_str_list) return 1; } +/* + * odb_read_object_info_extended() wrapper. Similar to odb_read_object_info() + * but uses the flags: + * + * - OBJECT_INFO_SKIP_FETCH_OBJECT so a server won't fetch an object when a + * object-info request asks for an OID that it doesn't have. + * + * - OBJECT_INFO_QUICK to avoid re-scanning packs when the object is not found. + */ +static enum object_type get_object_info(struct object_database *odb, + const struct object_id *oid, + size_t *sizep) +{ + enum object_type type; + struct object_info oi = OBJECT_INFO_INIT; + + oi.typep = &type; + oi.sizep = sizep; + if (odb_read_object_info_extended(odb, oid, &oi, + OBJECT_INFO_LOOKUP_REPLACE | + OBJECT_INFO_SKIP_FETCH_OBJECT | + OBJECT_INFO_QUICK) < 0) + return OBJ_BAD; + return type; +} + /* * Validates and send requested info back to the client. Any errors detected * are returned as they are detected. @@ -62,15 +88,22 @@ static void send_info(struct repository *r, struct packet_writer *writer, strbuf_addstr(&send_buffer, oid_str); + /* + * Check the existence of the object first. + * If an object is not recognized by the server append SP to + * the response. + */ + if (get_object_info(r->objects, &oid, &object_size) <= OBJ_NONE) { + strbuf_addstr(&send_buffer, " "); + goto write; + } + if (info->size) { - if (odb_read_object_info(r->objects, &oid, &object_size) < 0) { - strbuf_addstr(&send_buffer, " "); - } else { - strbuf_addf(&send_buffer, " %"PRIuMAX, - (uintmax_t)object_size); - } + strbuf_addf(&send_buffer, " %"PRIuMAX, + (uintmax_t)object_size); } +write: packet_writer_write(writer, "%s", send_buffer.buf); strbuf_reset(&send_buffer); } diff --git a/t/t5701-git-serve.sh b/t/t5701-git-serve.sh index d4c28bae39e2ad..cacff4456c6fea 100755 --- a/t/t5701-git-serve.sh +++ b/t/t5701-git-serve.sh @@ -7,6 +7,8 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +unknown_oid=$(printf "test" | git hash-object --stdin) + test_expect_success 'setup to generate files with expected content' ' printf "agent=git/%s" "$(git version | cut -d" " -f3)" >agent_capability && @@ -364,6 +366,67 @@ test_expect_success 'basics of object-info' ' test_cmp expect actual ' +test_expect_success 'bare OID request' ' + test_config transfer.advertiseObjectInfo true && + + test-tool pkt-line pack >in <<-EOF && + command=object-info + object-format=$(test_oid algo) + 0001 + oid $(git rev-parse two:two.t) + 0000 + EOF + + cat >expect <<-EOF && + $(git rev-parse two:two.t) + 0000 + EOF + + test-tool serve-v2 --stateless-rpc out && + test-tool pkt-line unpack actual && + test_cmp expect actual +' + +test_expect_success 'object-info with bare unrecognized OID' ' + test_config transfer.advertiseObjectInfo true && + + test-tool pkt-line pack >in <<-EOF && + command=object-info + object-format=$(test_oid algo) + 0001 + oid $unknown_oid + 0000 + EOF + + printf "%s \n" "$unknown_oid" >expect && + printf "0000\n" >>expect && + + test-tool serve-v2 --stateless-rpc out && + test-tool pkt-line unpack actual && + test_cmp expect actual +' + +test_expect_success 'object-info with size for unrecognized OID' ' + test_config transfer.advertiseObjectInfo true && + + test-tool pkt-line pack >in <<-EOF && + command=object-info + object-format=$(test_oid algo) + 0001 + size + oid $unknown_oid + 0000 + EOF + + printf "size\n" >expect && + printf "%s \n" "$unknown_oid" >>expect && + printf "0000\n" >>expect && + + test-tool serve-v2 --stateless-rpc out && + test-tool pkt-line unpack actual && + test_cmp expect actual +' + test_expect_success 'test capability advertisement with uploadpack.advertiseBundleURIs' ' test_config uploadpack.advertiseBundleURIs true && From 7dcbd35a120c04e30462407c4c9446aaaba9068f Mon Sep 17 00:00:00 2001 From: Calvin Wan Date: Fri, 24 Jul 2026 12:54:21 +0200 Subject: [PATCH 43/47] serve: advertise object-info feature In order for a client to know what object-info components a server can provide, advertise supported object-info features. This allows a client to decide whether to query the server for object-info or fetch as a fallback. Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- serve.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/serve.c b/serve.c index 49a6e39b1dd25c..2b07d922b3dde1 100644 --- a/serve.c +++ b/serve.c @@ -89,7 +89,7 @@ static void session_id_receive(struct repository *r UNUSED, trace2_data_string("transfer", NULL, "client-sid", client_sid); } -static int object_info_advertise(struct repository *r, struct strbuf *value UNUSED) +static int object_info_advertise(struct repository *r, struct strbuf *value) { if (advertise_object_info == -1 && repo_config_get_bool(r, "transfer.advertiseobjectinfo", @@ -97,6 +97,9 @@ static int object_info_advertise(struct repository *r, struct strbuf *value UNUS /* disabled by default */ advertise_object_info = 0; } + /* Currently only size is supported */ + if (value && advertise_object_info) + strbuf_addstr(value, "size"); return advertise_object_info; } From 12a19eec1d7c0121e0c82c95a2300fdbad1f18f2 Mon Sep 17 00:00:00 2001 From: Calvin Wan Date: Fri, 24 Jul 2026 12:54:22 +0200 Subject: [PATCH 44/47] transport: add client support for object-info Sometimes, it is beneficial to retrieve information about an object without downloading it entirely. The server-side logic for this functionality was implemented in commit "a2ba162cda (object-info: support for retrieving object info, 2021-04-20)." And the wire format is documented at https://git-scm.com/docs/protocol-v2#_object_info. Introduce client-side support for the object-info capability. Add its own function for object-info separate from existing fetch infrastructure. Currently, the client supports requesting a list of OIDs with the size attribute from a v2 server. If the server does not advertise this feature (i.e., transfer.advertiseobjectinfo is set to false), the client returns an error and exits. Note that: 1. The entire request is written into req_buf before being sent to the remote. This approach follows the pattern used in the send_fetch_request() logic within 'fetch-pack.c'. Streaming the request is not addressed in this patch. 2. A new field 'unrecognized' has been added to object_info. This new field is set at fetch_object_info() when the object is unrecognized by the server. Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- Makefile | 1 + fetch-object-info.c | 138 +++++++++++++++++++++++++++++++++++++++++++ fetch-object-info.h | 22 +++++++ meson.build | 1 + odb.h | 6 ++ transport-helper.c | 10 ++++ transport-internal.h | 8 +++ transport.c | 45 ++++++++++++++ transport.h | 9 +++ 9 files changed, 240 insertions(+) create mode 100644 fetch-object-info.c create mode 100644 fetch-object-info.h diff --git a/Makefile b/Makefile index 1f3f099f5c5705..d450e0277e4242 100644 --- a/Makefile +++ b/Makefile @@ -1158,6 +1158,7 @@ LIB_OBJS += ewah/ewah_io.o LIB_OBJS += ewah/ewah_rlw.o LIB_OBJS += exec-cmd.o LIB_OBJS += fetch-negotiator.o +LIB_OBJS += fetch-object-info.o LIB_OBJS += fetch-pack.o LIB_OBJS += fmt-merge-msg.o LIB_OBJS += fsck.o diff --git a/fetch-object-info.c b/fetch-object-info.c new file mode 100644 index 00000000000000..30475a1e873d61 --- /dev/null +++ b/fetch-object-info.c @@ -0,0 +1,138 @@ +#include "git-compat-util.h" +#include "gettext.h" +#include "hex.h" +#include "pkt-line.h" +#include "connect.h" +#include "oid-array.h" +#include "odb.h" +#include "fetch-object-info.h" +#include "string-list.h" + +/* Sends object-info command and its arguments into the request buffer. */ +static void send_object_info_request(const int fd_out, struct object_info_args *args) +{ + struct strbuf req_buf = STRBUF_INIT; + + write_command_and_capabilities(&req_buf, "object-info", args->server_options); + + if (unsorted_string_list_has_string(args->object_info_options, "size")) + packet_buf_write(&req_buf, "size"); + else if (args->object_info_options->nr) + BUG("only size should be in object_info_options"); + + if (args->oids) + for (size_t i = 0; i < args->oids->nr; i++) + packet_buf_write(&req_buf, "oid %s", oid_to_hex(&args->oids->oid[i])); + + packet_buf_flush(&req_buf); + if (write_in_full(fd_out, req_buf.buf, req_buf.len) < 0) + die_errno(_("unable to write request to remote")); + + strbuf_release(&req_buf); +} + +static int parse_object_size(const char *s, size_t *res) +{ + uintmax_t uim; + + if (!s[0] || s[strspn(s, "0123456789")]) + return -1; + errno = 0; + uim = strtoumax(s, NULL, 10); + if (errno || uim > SIZE_MAX) + return -1; + *res = uim; + return 0; +} + +int fetch_object_info(const enum protocol_version version, struct object_info_args *args, + struct packet_reader *reader, struct object_info *object_info_data, + const int stateless_rpc, const int fd_out) +{ + int size_index = -1; + + switch (version) { + case protocol_v2: + if (!server_supports_v2("object-info")) + die(_("object-info capability is not enabled on the server")); + send_object_info_request(fd_out, args); + break; + case protocol_v1: + case protocol_v0: + die(_("object-info requires protocol v2")); + case protocol_unknown_version: + BUG("unknown protocol version"); + } + + for (size_t i = 0; i < args->object_info_options->nr; i++) { + if (packet_reader_read(reader) != PACKET_READ_NORMAL) { + check_stateless_delimiter(stateless_rpc, reader, + "stateless delimiter expected"); + return -1; + } + + if (!string_list_has_string(args->object_info_options, reader->line)) + return -1; + + if (!strcmp(reader->line, "size")) { + /* + * i is the number of supported options which currently + * is only size. No risk of overflow. + */ + size_index = (int)i; + for (size_t j = 0; j < args->oids->nr; j++) + object_info_data[j].sizep = + xcalloc(1, sizeof(*object_info_data[j].sizep)); + } else { + BUG("only size is supported"); + } + } + + for (size_t i = 0; + packet_reader_read(reader) == PACKET_READ_NORMAL && + i < args->oids->nr; + i++) { + struct string_list object_info_values = STRING_LIST_INIT_DUP; + + string_list_split(&object_info_values, reader->line, " ", -1); + + if (strcmp(object_info_values.items[0].string, + oid_to_hex(&args->oids->oid[i]))) + die(_("object-info: expected OID: %s, got %s"), + oid_to_hex(&args->oids->oid[i]), + object_info_values.items[0].string); + + /* + * If the response is two elements but the second one is an + * empty string, that means that the OID is unrecognized by the + * server. + */ + if (object_info_values.nr >= 2 && + !strcmp(object_info_values.items[1].string, "")) { + object_info_data[i].unrecognized = 1; + string_list_clear(&object_info_values, 0); + continue; + } + + /* + * Because we filter the options to be only the supported by + * the server we expect the server to answer with the same + * number of attributes requested. + */ + if (args->object_info_options->nr + 1 != object_info_values.nr) + die("object-info: unexpected number of attributes: %s", + reader->line); + + if (size_index >= 0 && + parse_object_size(object_info_values.items[size_index + 1].string, + object_info_data[i].sizep)) + die("object-info: ref %s has invalid size %s", + object_info_values.items[0].string, + object_info_values.items[size_index + 1].string); + + string_list_clear(&object_info_values, 0); + } + check_stateless_delimiter(stateless_rpc, reader, "stateless delimiter expected"); + + return 0; +} diff --git a/fetch-object-info.h b/fetch-object-info.h new file mode 100644 index 00000000000000..31aad98408a29b --- /dev/null +++ b/fetch-object-info.h @@ -0,0 +1,22 @@ +#ifndef FETCH_OBJECT_INFO_H +#define FETCH_OBJECT_INFO_H + +#include "pkt-line.h" +#include "protocol.h" + +struct object_info_args { + struct string_list *object_info_options; + const struct string_list *server_options; + struct oid_array *oids; +}; + +struct object_info; +/* + * Sends git-cat-file object-info command into the request buf and read the + * results from packets. + */ +int fetch_object_info(enum protocol_version version, struct object_info_args *args, + struct packet_reader *reader, struct object_info *object_info_data, + int stateless_rpc, int fd_out); + +#endif /* FETCH_OBJECT_INFO_H */ diff --git a/meson.build b/meson.build index 9434b56960ba80..dfefcd3475bb76 100644 --- a/meson.build +++ b/meson.build @@ -359,6 +359,7 @@ libgit_sources = [ 'ewah/ewah_rlw.c', 'exec-cmd.c', 'fetch-negotiator.c', + 'fetch-object-info.c', 'fetch-pack.c', 'fmt-merge-msg.c', 'fsck.c', diff --git a/odb.h b/odb.h index 94754643d24997..88a37febbfa479 100644 --- a/odb.h +++ b/odb.h @@ -339,6 +339,12 @@ struct object_info { * or multiple times in the same source. */ struct odb_source_info *source_infop; + + /* + * object-info protocol specific. Set by the protocol when the remote + * does not recognize the requested object. + */ + unsigned int unrecognized:1; }; /* diff --git a/transport-helper.c b/transport-helper.c index f1950707883dbc..623463dcea891a 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -784,6 +784,15 @@ static int fetch_refs(struct transport *transport, return -1; } +static int fetch_object_info_helper(struct transport *transport) +{ + get_helper(transport); + if (process_connect(transport, 0)) + return transport->vtable->fetch_object_info(transport); + + die(_("object-info requires protocol v2")); +} + struct push_update_ref_state { struct ref *hint; struct ref_push_report *report; @@ -1330,6 +1339,7 @@ static struct transport_vtable vtable = { .get_refs_list = get_refs_list, .get_bundle_uri = get_bundle_uri, .fetch_refs = fetch_refs, + .fetch_object_info = fetch_object_info_helper, .push_refs = push_refs, .connect = connect_helper, .disconnect = release_helper diff --git a/transport-internal.h b/transport-internal.h index 051f3ab0dc95ec..60db0bedcdb9ae 100644 --- a/transport-internal.h +++ b/transport-internal.h @@ -45,6 +45,14 @@ struct transport_vtable { **/ int (*fetch_refs)(struct transport *transport, int refs_nr, struct ref **refs); + /* + * Fetch object info (only size currently) from remote without + * downloading the objects. + * + * Uses object-info capability of v2 protocol. + */ + int (*fetch_object_info)(struct transport *transport); + /** * Push the objects and refs. Send the necessary objects, and * then, for any refs where peer_ref is set and diff --git a/transport.c b/transport.c index fc144f0aedabd9..93426805318e36 100644 --- a/transport.c +++ b/transport.c @@ -9,6 +9,7 @@ #include "hook.h" #include "pkt-line.h" #include "fetch-pack.h" +#include "fetch-object-info.h" #include "remote.h" #include "connect.h" #include "send-pack.h" @@ -432,6 +433,48 @@ static int get_bundle_uri(struct transport *transport) transport->bundles, stateless_rpc); } +static int fetch_object_info_via_pack(struct transport *transport) +{ + int ret = 0; + struct git_transport_data *data = transport->data; + struct packet_reader reader; + struct object_info_args args = { 0 }; + + args.server_options = transport->server_options; + args.oids = transport->smart_options->object_info_oids; + args.object_info_options = transport->smart_options->object_info_options; + string_list_sort(args.object_info_options); + + connect_setup(transport, 0); + packet_reader_init(&reader, data->fd[0], NULL, 0, + PACKET_READ_CHOMP_NEWLINE | + PACKET_READ_GENTLE_ON_EOF | + PACKET_READ_DIE_ON_ERR_PACKET); + + data->version = discover_version(&reader); + transport->hash_algo = reader.hash_algo; + + ret = fetch_object_info(data->version, &args, &reader, + data->options.object_info_data, + transport->stateless_rpc, data->fd[1]); + + close(data->fd[0]); + if (data->fd[1] >= 0) + close(data->fd[1]); + if (finish_connect(data->conn)) + ret = -1; + data->conn = NULL; + + return ret; +} + +int transport_fetch_object_info(struct transport *transport) +{ + if (!transport->vtable->fetch_object_info) + die(_("remote does not support object-info")); + return transport->vtable->fetch_object_info(transport); +} + static int fetch_refs_via_pack(struct transport *transport, int nr_heads, struct ref **to_fetch) { @@ -1004,6 +1047,7 @@ static struct transport_vtable taken_over_vtable = { .get_refs_list = get_refs_via_connect, .get_bundle_uri = get_bundle_uri, .fetch_refs = fetch_refs_via_pack, + .fetch_object_info = fetch_object_info_via_pack, .push_refs = git_transport_push, .disconnect = disconnect_git }; @@ -1169,6 +1213,7 @@ static struct transport_vtable builtin_smart_vtable = { .get_refs_list = get_refs_via_connect, .get_bundle_uri = get_bundle_uri, .fetch_refs = fetch_refs_via_pack, + .fetch_object_info = fetch_object_info_via_pack, .push_refs = git_transport_push, .connect = connect_git, .disconnect = disconnect_git diff --git a/transport.h b/transport.h index 7e5867cffaaa4a..a7869d18e020fb 100644 --- a/transport.h +++ b/transport.h @@ -55,6 +55,10 @@ struct git_transport_options { * common commits to this oidset instead of fetching any packfiles. */ struct oidset *acked_commits; + + struct oid_array *object_info_oids; + struct object_info *object_info_data; + struct string_list *object_info_options; }; enum transport_family { @@ -309,6 +313,11 @@ int transport_get_remote_bundle_uri(struct transport *transport); const struct git_hash_algo *transport_get_hash_algo(struct transport *transport); int transport_fetch_refs(struct transport *transport, struct ref *refs); +/* + * Fetch the object info from remote + */ +int transport_fetch_object_info(struct transport *transport); + /* * If this flag is set, unlocking will avoid to call non-async-signal-safe * functions. This will necessarily leave behind some data structures which From 0ae93f56ecd792d227149161b57f292a1c909d0c Mon Sep 17 00:00:00 2001 From: Eric Ju Date: Fri, 24 Jul 2026 12:54:23 +0200 Subject: [PATCH 45/47] cat-file: add remote-object-info to batch-command Since the info command in cat-file --batch-command prints object info for a given object, it is natural to add another command in cat-file --batch-command to print object info for a given object from a remote. Add remote-object-info command to cat-file --batch-command. While info takes object ids one at a time, this creates overhead when making requests to a server. So remote-object-info instead can take multiple object ids at once. The cat-file --batch-command command is generally implemented in the following manner: - Receive and parse input from user - Call respective function attached to command - Get object info, print object info In --buffer mode, this changes to: - Receive and parse input from user - Store respective function attached to command in a queue - After flush, loop through commands in queue - Call respective function attached to command - Get object info, print object info Notice how the getting and printing of object info is accomplished one at a time. As described above, this creates a problem for making requests to a server. Therefore, remote-object-info is implemented in the following manner: - Receive and parse input from user If command is remote-object-info: - Get object info from remote - Loop through and print each object info Else: - Call respective function attached to command - Parse input, get object info, print object info And finally for --buffer mode remote-object-info: - Receive and parse input from user - Store respective function attached to command in a queue - After flush, loop through commands in queue: If command is remote-object-info: - Get object info from remote - Loop through and print each object info Else: - Call respective function attached to command - Get object info, print object info To summarize, remote-object-info gets object info from the remote and then loops through the object info passed in, printing the info. In order for remote-object-info to avoid remote communication overhead in the non-buffer mode, the objects are passed in as such: remote-object-info ... rather than remote-object-info remote-object-info ... remote-object-info Placeholders in the format are validated against an allow-list of the atoms the remote path supports: "objectname" and "objectsize". Unsupported atoms expand to an empty string, honoring how for-each-ref handles known but inapplicable atoms. Without this, atoms like %(objecttype) would mark data->info.typep and because the server only sends size, type_name() would later crash. As extra safety, even outside of the remote path, initialize expand_data's type to OBJ_BAD and handle type_name() returning NULL. Helped-by: Jonathan Tan Helped-by: Christian Couder Mentored-by: Karthik Nayak Mentored-by: Chandra Pratap Signed-off-by: Calvin Wan Signed-off-by: Eric Ju [pablo: added the atom allow-list validation] Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- Documentation/git-cat-file.adoc | 23 +- builtin/cat-file.c | 181 ++++++- object-file.c | 10 + odb.h | 3 + t/meson.build | 1 + t/t1017-cat-file-remote-object-info.sh | 719 +++++++++++++++++++++++++ 6 files changed, 930 insertions(+), 7 deletions(-) create mode 100755 t/t1017-cat-file-remote-object-info.sh diff --git a/Documentation/git-cat-file.adoc b/Documentation/git-cat-file.adoc index 86b9181599317e..ac3b528c6f00f6 100644 --- a/Documentation/git-cat-file.adoc +++ b/Documentation/git-cat-file.adoc @@ -169,6 +169,13 @@ info :: Print object info for object reference ``. This corresponds to the output of `--batch-check`. +remote-object-info ...:: + Print object info for object references `` at specified + `` without downloading objects from the remote. + Raise an error when the `object-info` capability is not supported by the remote. + Raise an error when no object references are provided. + This command may be combined with `--buffer`. + flush:: Used with `--buffer` to execute all preceding commands that were issued since the beginning or since the last flush was issued. When `--buffer` @@ -301,7 +308,8 @@ one per line, and print information based on the command given. With `--batch-command`, the `info` command followed by an object will print information about the object the same way `--batch-check` would, and the `contents` command followed by an object prints contents in the same way -`--batch` would. +`--batch` would. The `remote-object-info` command followed by a remote and +object IDs prints object info from the remote without downloading the objects. You can specify the information shown for each object by using a custom ``. The `` is copied literally to stdout for each @@ -340,8 +348,15 @@ newline. The available atoms are: after that first run of whitespace (i.e., the "rest" of the line) are output in place of the `%(rest)` atom. +The command `remote-object-info` only supports the `%(objectname)` and +`%(objectsize)` placeholders. See `CAVEATS` below for more information. + If no format is specified, the default format is `%(objectname) -%(objecttype) %(objectsize)`. +%(objecttype) %(objectsize)`, except for `remote-object-info` commands which +use `%(objectname) %(objectsize)` because `%(objecttype)` is not supported yet. + +WARNING: When "%(objecttype)" is supported, the default format WILL be unified, +so DO NOT RELY on the current default format to stay the same!!! If `--batch` is specified, or if `--batch-command` is used with the `contents` command, the object information is followed by the object contents (consisting @@ -438,6 +453,10 @@ scripting purposes. CAVEATS ------- +Note that only `%(objectname)` and `%(objectsize)` are currently +supported by the `remote-object-info` command. Using any other placeholder in +the format string will return an empty string in its position. + Note that the sizes of objects on disk are reported accurately, but care should be taken in drawing conclusions about which refs or objects are responsible for disk usage. The size of a packed non-delta object may be diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 03afc44c5ed482..8994b04d15692c 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -29,6 +29,22 @@ #include "promisor-remote.h" #include "mailmap.h" #include "write-or-die.h" +#include "alias.h" +#include "remote.h" +#include "transport.h" + +/* + * Maximum length for a remote URL. While no universal standard exists, + * 8K is assumed to be a reasonable limit. + */ +#define MAX_REMOTE_URL_LEN (8 * 1024) + +/* Maximum number of objects allowed in a single remote-object-info request. */ +#define MAX_ALLOWED_OBJ_LIMIT 10000 + +/* Maximum input size permitted for the remote-object-info command. */ +#define MAX_REMOTE_OBJ_INFO_LINE \ + (MAX_REMOTE_URL_LEN + MAX_ALLOWED_OBJ_LIMIT * (GIT_MAX_HEXSZ + 1)) enum batch_mode { BATCH_MODE_CONTENTS, @@ -317,8 +333,19 @@ struct expand_data { * optimized out. */ unsigned skip_object_info : 1; + + /* + * Flags about when an object info is being fetched from remote. + */ + unsigned is_remote:1; +}; + +#define EXPAND_DATA_INIT { .mode = S_IFINVALID, .type = OBJ_BAD } + +static const char *remote_object_info_atoms[] = { + "objectname", + "objectsize", }; -#define EXPAND_DATA_INIT { .mode = S_IFINVALID } static int is_atom(const char *atom, const char *s, int slen) { @@ -329,14 +356,31 @@ static int is_atom(const char *atom, const char *s, int slen) static int expand_atom(struct strbuf *sb, const char *atom, int len, struct expand_data *data) { + if (data->is_remote) { + size_t i, allowed_nr = ARRAY_SIZE(remote_object_info_atoms); + for (i = 0; i < allowed_nr; i++) + if (is_atom(remote_object_info_atoms[i], atom, len)) + break; + + /* + * On remote, skip unsupported atoms returning an empty sb, + * honoring how for-each-ref handles known but inapplicable + * atoms (e.g. %(tagger)). + */ + if (i == allowed_nr) + return 1; + } + if (is_atom("objectname", atom, len)) { if (!data->mark_query) strbuf_add_oid_hex(sb, &data->oid); } else if (is_atom("objecttype", atom, len)) { - if (data->mark_query) + if (data->mark_query) { data->info.typep = &data->type; - else - strbuf_addstr(sb, type_name(data->type)); + } else { + const char *t = type_name(data->type); + strbuf_addstr(sb, t ? t : ""); + } } else if (is_atom("objectsize", atom, len)) { if (data->mark_query) data->info.sizep = &data->size; @@ -636,6 +680,65 @@ static void batch_one_object(const char *obj_name, object_context_release(&ctx); } +static int get_remote_info(int argc, + const char **argv, + struct object_info **remote_object_info, + struct oid_array *object_info_oids) +{ + int retval = 0; + struct remote *remote = NULL; + struct object_id oid; + struct string_list object_info_options = STRING_LIST_INIT_NODUP; + struct transport *gtransport; + + remote = remote_get(argv[0]); + if (!remote) + die(_("must supply valid remote when using remote-object-info")); + + oid_array_clear(object_info_oids); + for (size_t i = 1; i < argc; i++) { + if (get_oid_hex(argv[i], &oid)) { + size_t len = strlen(argv[i]); + + if (len < the_hash_algo->hexsz && len >= 4) { + size_t j; + for (j = 0; j < len; j++) + if (!isxdigit(argv[i][j])) + break; + if (j == len) + die(_("remote-object-info does not support " + "short oids, %d characters required"), + (int)the_hash_algo->hexsz); + } + die(_("not a valid object name '%s'"), argv[i]); + } + oid_array_append(object_info_oids, &oid); + } + + if (!object_info_oids->nr) + die(_("remote-object-info requires objects")); + + gtransport = transport_get(remote, NULL); + + if (!gtransport->smart_options) { + retval = -1; + goto cleanup; + } + + CALLOC_ARRAY(*remote_object_info, object_info_oids->nr); + gtransport->smart_options->object_info_oids = object_info_oids; + + string_list_append(&object_info_options, "size"); + + gtransport->smart_options->object_info_options = &object_info_options; + gtransport->smart_options->object_info_data = *remote_object_info; + retval = transport_fetch_object_info(gtransport); +cleanup: + string_list_clear(&object_info_options, 0); + transport_disconnect(gtransport); + return retval; +} + struct object_cb_data { struct batch_options *opt; struct expand_data *expand; @@ -717,6 +820,73 @@ static void parse_cmd_mailmap(struct batch_options *opt UNUSED, load_mailmap(); } +static void parse_cmd_remote_object_info(struct batch_options *opt, + const char *line, struct strbuf *output, + struct expand_data *data) +{ + int count; + const char **argv; + char *line_to_split; + struct object_info *remote_object_info = NULL; + struct oid_array object_info_oids = OID_ARRAY_INIT; + const char *saved_format = opt->format; + + if (strlen(line) >= MAX_REMOTE_OBJ_INFO_LINE) + die(_("remote-object-info command too long")); + /* + * TODO: Use the default format once %(objecttype) is supported. + */ + if (!opt->format) + opt->format = "%(objectname) %(objectsize)"; + + line_to_split = xstrdup(line); + count = split_cmdline(line_to_split, &argv); + if (count < 0) + die(_("remote-object-info: failed to parse command line: %s"), + split_cmdline_strerror(count)); + if (count - 1 > MAX_ALLOWED_OBJ_LIMIT) + die(_("remote-object-info supports at most %d objects"), + MAX_ALLOWED_OBJ_LIMIT); + + if (get_remote_info(count, argv, &remote_object_info, + &object_info_oids)) + die(_("failed to get object info from the remote: %s"), argv[0]); + + data->skip_object_info = 1; + for (size_t i = 0; i < object_info_oids.nr; i++) { + data->oid = object_info_oids.oid[i]; + + if (remote_object_info[i].unrecognized) { + report_object_status(opt, oid_to_hex(&data->oid), + &data->oid, "missing"); + continue; + } + + if (remote_object_info[i].sizep) { + /* + * When reaching here, it means remote-object-info can retrieve + * information from server without downloading them. + */ + data->size = *remote_object_info[i].sizep; + opt->batch_mode = BATCH_MODE_INFO; + data->is_remote = 1; + batch_object_write(argv[i + 1], output, opt, data, NULL, 0); + data->is_remote = 0; + } else { + report_object_status(opt, oid_to_hex(&data->oid), &data->oid, "missing"); + } + } + data->skip_object_info = 0; + opt->format = saved_format; + + for (size_t i = 0; i < object_info_oids.nr; i++) + free_object_info_contents(&remote_object_info[i]); + free(line_to_split); + free(argv); + free(remote_object_info); + oid_array_clear(&object_info_oids); +} + static void dispatch_calls(struct batch_options *opt, struct strbuf *output, struct expand_data *data, @@ -747,9 +917,10 @@ static const struct parse_cmd { unsigned takes_args; } commands[] = { { "contents", parse_cmd_contents, 1 }, - { "info", parse_cmd_info, 1 }, { "flush", NULL, 0 }, + { "info", parse_cmd_info, 1 }, { "mailmap", parse_cmd_mailmap, 1 }, + { "remote-object-info", parse_cmd_remote_object_info, 1 }, }; static void batch_objects_command(struct batch_options *opt, diff --git a/object-file.c b/object-file.c index 93602f8c50a858..121416c37c6709 100644 --- a/object-file.c +++ b/object-file.c @@ -1698,3 +1698,13 @@ struct odb_transaction *odb_transaction_files_begin(struct odb_source *source) return &transaction->base; } + +void free_object_info_contents(struct object_info *object_info) +{ + if (!object_info) + return; + free(object_info->typep); + free(object_info->sizep); + free(object_info->disk_sizep); + free(object_info->delta_base_oid); +} diff --git a/odb.h b/odb.h index 88a37febbfa479..92fa414e2c66d6 100644 --- a/odb.h +++ b/odb.h @@ -623,4 +623,7 @@ void parse_alternates(const char *string, const char *relative_base, struct strvec *out); +/* Free pointers inside of object_info, but not object_info itself */ +void free_object_info_contents(struct object_info *object_info); + #endif /* ODB_H */ diff --git a/t/meson.build b/t/meson.build index 8ae6ab6c5fe1e2..10241e3dccb785 100644 --- a/t/meson.build +++ b/t/meson.build @@ -171,6 +171,7 @@ integration_tests = [ 't1014-read-tree-confusing.sh', 't1015-read-index-unmerged.sh', 't1016-compatObjectFormat.sh', + 't1017-cat-file-remote-object-info.sh', 't1020-subdirectory.sh', 't1022-read-tree-partial-clone.sh', 't1050-large.sh', diff --git a/t/t1017-cat-file-remote-object-info.sh b/t/t1017-cat-file-remote-object-info.sh new file mode 100755 index 00000000000000..edc20394d81b0a --- /dev/null +++ b/t/t1017-cat-file-remote-object-info.sh @@ -0,0 +1,719 @@ +#!/bin/sh + +test_description='git cat-file --batch-command with remote-object-info command' + +. ./test-lib.sh +. "$TEST_DIRECTORY"/lib-cat-file.sh + +hello_content="Hello World" +hello_size=$(strlen "$hello_content") +hello_oid=$(echo_without_newline "$hello_content" | git hash-object --stdin) +hello_short_oid=$(git rev-parse --short "$hello_oid") + +unstored_content="Hello Git" +unstored_oid=$(echo_without_newline "$unstored_content" | git hash-object --stdin) + +# This is how we get 13: +# 13 = + + + , where +# file mode is 100644, which is 6 characters; +# file name is hello, which is 5 characters +# a space is 1 character and a null is 1 character +tree_size=$(($(test_oid rawsz) + 13)) + +commit_message="Initial commit" + +# This is how we get 137: +# 137 = + + + +# + + +# + + +# + +# +# An easier way to calculate is: 1. use `git cat-file commit | wc -c`, +# to get 177, 2. then deduct 40 hex characters to get 137 +commit_size=$(($(test_oid hexsz) + 137)) + +tag_header_without_oid="type blob +tag hellotag +tagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>" +tag_header_without_timestamp="object $hello_oid +$tag_header_without_oid" +tag_description="This is a tag" +tag_content="$tag_header_without_timestamp 0 +0000 + +$tag_description" + +tag_oid=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w) +tag_size=$(strlen "$tag_content") + +set_transport_variables () { + hello_oid=$(echo_without_newline "$hello_content" | git hash-object --stdin) + tree_oid=$(git -C "$1" write-tree) + commit_oid=$(echo_without_newline "$commit_message" | git -C "$1" commit-tree $tree_oid) + tag_oid=$(echo_without_newline "$tag_content" | git -C "$1" hash-object -t tag --stdin -w) + tag_size=$(strlen "$tag_content") +} + +# This section tests --batch-command with remote-object-info command +# Since "%(objecttype)" is currently not supported by the command remote-object-info , +# the filters are set to "%(objectname) %(objectsize)" in some test cases. + +# Test --batch-command remote-object-info with 'git://' transport with +# transfer.advertiseobjectinfo set to true, i.e. server has object-info capability +. "$TEST_DIRECTORY"/lib-git-daemon.sh +start_git_daemon --export-all +daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent + +test_expect_success 'create repo to be served by git-daemon' ' + git init "$daemon_parent" && + echo_without_newline "$hello_content" > $daemon_parent/hello && + git -C "$daemon_parent" update-index --add hello && + git -C "$daemon_parent" config transfer.advertiseobjectinfo true && + git clone "$GIT_DAEMON_URL/parent" -n "$daemon_parent/daemon_client_empty" +' + +test_expect_success 'batch-command remote-object-info git://' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid + remote-object-info "$GIT_DAEMON_URL/parent" $tree_oid + remote-object-info "$GIT_DAEMON_URL/parent" $commit_oid + remote-object-info "$GIT_DAEMON_URL/parent" $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info git:// multiple sha1 per line' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid $tree_oid $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info git:// default filter' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid $tree_oid + remote-object-info "$GIT_DAEMON_URL/parent" $commit_oid $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info does not change the default format of info' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + local_content="local object" && + local_oid=$(echo_without_newline "$local_content" | git hash-object -w --stdin) && + local_size=$(strlen "$local_content") && + + echo "$local_oid blob $local_size" >expect && + echo "$hello_oid $hello_size" >>expect && + echo "$local_oid blob $local_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + info $local_oid + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid + info $local_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info git://' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid $tree_oid + remote-object-info "$GIT_DAEMON_URL/parent" $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command -Z remote-object-info git:// default filter' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + printf "%s\0" "$hello_oid $hello_size" >expect && + printf "%s\0" "$tree_oid $tree_size" >>expect && + printf "%s\0" "$commit_oid $commit_size" >>expect && + printf "%s\0" "$tag_oid $tag_size" >>expect && + + printf "%s\0" "$hello_oid missing" >>expect && + printf "%s\0" "$tree_oid missing" >>expect && + printf "%s\0" "$commit_oid missing" >>expect && + printf "%s\0" "$tag_oid missing" >>expect && + + batch_input="remote-object-info $GIT_DAEMON_URL/parent $hello_oid $tree_oid +remote-object-info $GIT_DAEMON_URL/parent $commit_oid $tag_oid +info $hello_oid +info $tree_oid +info $commit_oid +info $tag_oid +" && + echo_without_newline_nul "$batch_input" >commands_null_delimited && + + git cat-file --batch-command -Z < commands_null_delimited >actual && + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info does not support short oids' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + test_must_fail git cat-file --batch-command 2>err <<-EOF && + remote-object-info $GIT_DAEMON_URL/parent $hello_short_oid + EOF + test_grep "does not support short oids" err + ) +' + +test_expect_success 'remote-object-info does not die on missing oid like info' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + git cat-file --batch-command >local <<-EOF && + info $unstored_oid + EOF + git cat-file --batch-command >remote <<-EOF && + remote-object-info $GIT_DAEMON_URL/parent $unstored_oid + EOF + test_cmp local remote + ) +' + +# This tests depends on %(objecttype) not being supported yet, once supported +# it needs to be updated. +test_expect_success 'unsupported placeholder on remote returns empty string' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + echo "" >expect && + git cat-file --batch-command="%(objecttype)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid + EOF + test_cmp expect actual + ) +' + +# Test --batch-command remote-object-info with 'git://' and +# transfer.advertiseobjectinfo set to false, i.e. server does not have object-info capability +test_expect_success 'batch-command remote-object-info git:// fails when transfer.advertiseobjectinfo=false' ' + ( + git -C "$daemon_parent" config transfer.advertiseobjectinfo false && + set_transport_variables "$daemon_parent" && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info $GIT_DAEMON_URL/parent $hello_oid $tree_oid $commit_oid $tag_oid + EOF + test_grep "object-info capability is not enabled on the server" err && + + # revert server state back + git -C "$daemon_parent" config transfer.advertiseobjectinfo true + + ) +' + +stop_git_daemon + +# Test --batch-command remote-object-info with 'file://' transport with +# transfer.advertiseobjectinfo set to true, i.e. server has object-info capability +# shellcheck disable=SC2016 +test_expect_success 'create repo to be served by file:// transport' ' + git init server && + git -C server config protocol.version 2 && + git -C server config transfer.advertiseobjectinfo true && + echo_without_newline "$hello_content" > server/hello && + git -C server update-index --add hello && + git clone -n "file://$(pwd)/server" file_client_empty +' + +test_expect_success 'batch-command remote-object-info file://' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_oid + remote-object-info "file://${server_path}" $tree_oid + remote-object-info "file://${server_path}" $commit_oid + remote-object-info "file://${server_path}" $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info file:// multiple sha1 per line' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_oid $tree_oid $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info file://' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_oid $tree_oid + remote-object-info "file://${server_path}" $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info file:// default filter' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_oid $tree_oid + remote-object-info "file://${server_path}" $commit_oid $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command -Z remote-object-info file:// default filter' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + printf "%s\0" "$hello_oid $hello_size" >expect && + printf "%s\0" "$tree_oid $tree_size" >>expect && + printf "%s\0" "$commit_oid $commit_size" >>expect && + printf "%s\0" "$tag_oid $tag_size" >>expect && + + printf "%s\0" "$hello_oid missing" >>expect && + printf "%s\0" "$tree_oid missing" >>expect && + printf "%s\0" "$commit_oid missing" >>expect && + printf "%s\0" "$tag_oid missing" >>expect && + + batch_input="remote-object-info \"file://${server_path}\" $hello_oid $tree_oid +remote-object-info \"file://${server_path}\" $commit_oid $tag_oid +info $hello_oid +info $tree_oid +info $commit_oid +info $tag_oid +" && + echo_without_newline_nul "$batch_input" >commands_null_delimited && + + git cat-file --batch-command -Z < commands_null_delimited >actual && + test_cmp expect actual + ) +' + +# Test --batch-command remote-object-info with 'file://' and +# transfer.advertiseobjectinfo set to false, i.e. server does not have object-info capability +test_expect_success 'batch-command remote-object-info file:// fails when transfer.advertiseobjectinfo=false' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + git -C "${server_path}" config transfer.advertiseobjectinfo false && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "file://${server_path}" $hello_oid $tree_oid $commit_oid $tag_oid + EOF + test_grep "object-info capability is not enabled on the server" err && + + # revert server state back + git -C "${server_path}" config transfer.advertiseobjectinfo true + ) +' + +# Test --batch-command remote-object-info with 'http://' transport with +# transfer.advertiseobjectinfo set to true, i.e. server has object-info capability + +. "$TEST_DIRECTORY"/lib-httpd.sh +start_httpd + +test_expect_success 'create repo to be served by http:// transport' ' + git init "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config http.receivepack true && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config transfer.advertiseobjectinfo true && + echo_without_newline "$hello_content" > $HTTPD_DOCUMENT_ROOT_PATH/http_parent/hello && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" update-index --add hello && + git clone "$HTTPD_URL/smart/http_parent" -n "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" +' + +test_expect_success 'batch-command remote-object-info http://' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid + remote-object-info "$HTTPD_URL/smart/http_parent" $tree_oid + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_oid + remote-object-info "$HTTPD_URL/smart/http_parent" $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info http:// one line' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid $tree_oid $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info http://' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid $tree_oid + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info http:// default filter' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid $tree_oid + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_oid $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command -Z remote-object-info http:// default filter' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + printf "%s\0" "$hello_oid $hello_size" >expect && + printf "%s\0" "$tree_oid $tree_size" >>expect && + printf "%s\0" "$commit_oid $commit_size" >>expect && + printf "%s\0" "$tag_oid $tag_size" >>expect && + + batch_input="remote-object-info $HTTPD_URL/smart/http_parent $hello_oid $tree_oid +remote-object-info $HTTPD_URL/smart/http_parent $commit_oid $tag_oid +" && + echo_without_newline_nul "$batch_input" >commands_null_delimited && + + git cat-file --batch-command -Z < commands_null_delimited >actual && + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info fails on unsupported filter option (objectsize:disk)' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + echo "$hello_oid " >expect && + + git cat-file --batch-command="%(objectname) %(objectsize:disk)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info fails on unsupported filter option (deltabase)' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + echo "" >expect && + + git cat-file --batch-command="%(deltabase)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info fails on server with legacy protocol' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git -c protocol.version=0 cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid + EOF + test_grep "object-info requires protocol v2" err + ) +' + +test_expect_success 'remote-object-info fails on server with legacy protocol with default filter' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git -c protocol.version=0 cat-file --batch-command 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid + EOF + test_grep "object-info requires protocol v2" err + ) +' + +test_expect_success 'remote-object-info fails on malformed OID' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + malformed_object_id="this_id_is_not_valid" && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $malformed_object_id + EOF + test_grep "not a valid object name '$malformed_object_id'" err + ) +' + +test_expect_success 'remote-object-info fails on malformed OID with default filter' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + malformed_object_id="this_id_is_not_valid" && + + test_must_fail git cat-file --batch-command 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $malformed_object_id + EOF + test_grep "not a valid object name '$malformed_object_id'" err + ) +' + +test_expect_success 'remote-object-info fails on not providing OID' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" + EOF + test_grep "remote-object-info requires objects" err + ) +' + + +# Test --batch-command remote-object-info with 'http://' transport and +# transfer.advertiseobjectinfo set to false, i.e. server does not have object-info capability +test_expect_success 'batch-command remote-object-info http:// fails when transfer.advertiseobjectinfo=false ' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config transfer.advertiseobjectinfo false && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid $tree_oid $commit_oid $tag_oid + EOF + test_grep "object-info capability is not enabled on the server" err && + + # revert server state back + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config transfer.advertiseobjectinfo true + ) +' + +# DO NOT add non-httpd-specific tests here, because the last part of this +# test script is only executed when httpd is available and enabled. + +test_done From dd1968bf84def5453886ab7320c44c18f4d03005 Mon Sep 17 00:00:00 2001 From: Pablo Sabater Date: Fri, 24 Jul 2026 12:54:24 +0200 Subject: [PATCH 46/47] cat-file: make remote-object-info allow-list adapt to the server The static allow-list in expand_atom() is hardcoded to allow only "objectname" and "objectsize" for remote queries. This works because, up to this point, servers will either support object-info with name and size or they do not support them at all. As object-info gains new capabilities, we cannot expect different servers with different Git versions to have the same object-info capabilities. Therefore, the client needs to adapt its allow-list to what the server advertises. The client now: 1. Requests the protocol option that the placeholder refers to (i.e. "size" for "%(objectsize)"). 2. Drops any requested option that the server does not advertise in fetch_object_info(). 3. Maps the remaining advertised options back to their placeholders and populates remote_allowed_atoms. 4. Uses remote_allowed_atoms in expand_atom(), preserving the previous behavior for supported placeholders. For example, if the client requests "%(objectsize) %(objecttype)" and the server only supports 'size', then the client only requests 'size'. The server returns the size (i.e "42") "%(objectsize)" is expanded normally while "%(objecttype)" expands to an empty string: "42 " Note that the empty string expansion is only for known but unsupported placeholders. "%(objectcolor)" which doesn't exist would die(). This honors what for-each-ref does for known but inapplicable atoms (placeholders). Move object_info_options out of get_remote_info() so the caller which has data can select what options will be requested instead of requesting always size. Move batch_object_write() out so output is always produced. If there are no supported attributes, the output is a blank line. Include "type" in the object_info_options even though the client does not yet know how to parse the server's "type" capability. As a result, "type" is always filtered out, allowing the tests to verify that known but unsupported placeholders expand to an empty string. Since the filter removes options by swapping with the last element, the list is no longer kept sorted. Drop the pre-sort in fetch_object_info_via_pack() and use the unsorted string_list lookup for the response header. This has no effect in performance as the list can only be two entries long ('size' and 'type'). Mentored-by: Karthik Nayak Mentored-by: Chandra Pratap Signed-off-by: Pablo Sabater Signed-off-by: Junio C Hamano --- builtin/cat-file.c | 96 +++++++++++++++++--------- fetch-object-info.c | 20 +++++- fetch-object-info.h | 3 + t/t1017-cat-file-remote-object-info.sh | 28 ++++++++ transport.c | 1 - 5 files changed, 113 insertions(+), 35 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 8994b04d15692c..44a32a1df48efd 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -338,15 +338,20 @@ struct expand_data { * Flags about when an object info is being fetched from remote. */ unsigned is_remote:1; -}; - -#define EXPAND_DATA_INIT { .mode = S_IFINVALID, .type = OBJ_BAD } -static const char *remote_object_info_atoms[] = { - "objectname", - "objectsize", + /* + * List of atoms (i.e. "objectsize") that the server supports. Built + * from the server's object-info advertised capabilities. + */ + struct string_list remote_allowed_atoms; }; +#define EXPAND_DATA_INIT { \ + .mode = S_IFINVALID, \ + .type = OBJ_BAD, \ + .remote_allowed_atoms = STRING_LIST_INIT_NODUP, \ +} + static int is_atom(const char *atom, const char *s, int slen) { int alen = strlen(atom); @@ -357,17 +362,12 @@ static int expand_atom(struct strbuf *sb, const char *atom, int len, struct expand_data *data) { if (data->is_remote) { - size_t i, allowed_nr = ARRAY_SIZE(remote_object_info_atoms); - for (i = 0; i < allowed_nr; i++) - if (is_atom(remote_object_info_atoms[i], atom, len)) + size_t i; + for (i = 0; i < data->remote_allowed_atoms.nr; i++) + if (is_atom(data->remote_allowed_atoms.items[i].string, + atom, len)) break; - - /* - * On remote, skip unsupported atoms returning an empty sb, - * honoring how for-each-ref handles known but inapplicable - * atoms (e.g. %(tagger)). - */ - if (i == allowed_nr) + if (i == data->remote_allowed_atoms.nr) return 1; } @@ -683,12 +683,12 @@ static void batch_one_object(const char *obj_name, static int get_remote_info(int argc, const char **argv, struct object_info **remote_object_info, - struct oid_array *object_info_oids) + struct oid_array *object_info_oids, + struct string_list *object_info_options) { int retval = 0; struct remote *remote = NULL; struct object_id oid; - struct string_list object_info_options = STRING_LIST_INIT_NODUP; struct transport *gtransport; remote = remote_get(argv[0]); @@ -728,13 +728,10 @@ static int get_remote_info(int argc, CALLOC_ARRAY(*remote_object_info, object_info_oids->nr); gtransport->smart_options->object_info_oids = object_info_oids; - string_list_append(&object_info_options, "size"); - - gtransport->smart_options->object_info_options = &object_info_options; + gtransport->smart_options->object_info_options = object_info_options; gtransport->smart_options->object_info_data = *remote_object_info; retval = transport_fetch_object_info(gtransport); cleanup: - string_list_clear(&object_info_options, 0); transport_disconnect(gtransport); return retval; } @@ -820,6 +817,21 @@ static void parse_cmd_mailmap(struct batch_options *opt UNUSED, load_mailmap(); } +struct protocol_placeholder_entry { + const char *option; + const char *atom; +}; + +static const struct protocol_placeholder_entry remote_atom_map[] = { + {"size", "objectsize"}, + {"type", "objecttype"}, + /* + * Add new protocol options here. Even if the server doesn't support + * them the allow_list will drop them if the server doesn't advertise + * them. + */ +}; + static void parse_cmd_remote_object_info(struct batch_options *opt, const char *line, struct strbuf *output, struct expand_data *data) @@ -829,6 +841,7 @@ static void parse_cmd_remote_object_info(struct batch_options *opt, char *line_to_split; struct object_info *remote_object_info = NULL; struct oid_array object_info_oids = OID_ARRAY_INIT; + struct string_list object_info_options = STRING_LIST_INIT_NODUP; const char *saved_format = opt->format; if (strlen(line) >= MAX_REMOTE_OBJ_INFO_LINE) @@ -848,10 +861,22 @@ static void parse_cmd_remote_object_info(struct batch_options *opt, die(_("remote-object-info supports at most %d objects"), MAX_ALLOWED_OBJ_LIMIT); + if (data->info.sizep) + string_list_append(&object_info_options, "size"); + if (data->info.typep) + string_list_append(&object_info_options, "type"); + if (get_remote_info(count, argv, &remote_object_info, - &object_info_oids)) + &object_info_oids, &object_info_options)) die(_("failed to get object info from the remote: %s"), argv[0]); + string_list_clear(&data->remote_allowed_atoms, 0); + string_list_append(&data->remote_allowed_atoms, "objectname"); + for (size_t i = 0; i < ARRAY_SIZE(remote_atom_map); i++) + if (unsorted_string_list_has_string(&object_info_options, remote_atom_map[i].option)) + string_list_append(&data->remote_allowed_atoms, + remote_atom_map[i].atom); + data->skip_object_info = 1; for (size_t i = 0; i < object_info_oids.nr; i++) { data->oid = object_info_oids.oid[i]; @@ -862,25 +887,29 @@ static void parse_cmd_remote_object_info(struct batch_options *opt, continue; } + /* + * When reaching here, it means remote-object-info can retrieve + * information from server without downloading them. + */ if (remote_object_info[i].sizep) { - /* - * When reaching here, it means remote-object-info can retrieve - * information from server without downloading them. - */ data->size = *remote_object_info[i].sizep; - opt->batch_mode = BATCH_MODE_INFO; - data->is_remote = 1; - batch_object_write(argv[i + 1], output, opt, data, NULL, 0); - data->is_remote = 0; - } else { - report_object_status(opt, oid_to_hex(&data->oid), &data->oid, "missing"); } + + if (remote_object_info[i].typep) { + data->type = *remote_object_info[i].typep; + } + + opt->batch_mode = BATCH_MODE_INFO; + data->is_remote = 1; + batch_object_write(argv[i + 1], output, opt, data, NULL, 0); + data->is_remote = 0; } data->skip_object_info = 0; opt->format = saved_format; for (size_t i = 0; i < object_info_oids.nr; i++) free_object_info_contents(&remote_object_info[i]); + string_list_clear(&object_info_options, 0); free(line_to_split); free(argv); free(remote_object_info); @@ -1200,6 +1229,7 @@ static int batch_objects(struct batch_options *opt) cleanup: strbuf_release(&input); strbuf_release(&output); + string_list_clear(&data.remote_allowed_atoms, 0); cfg->warn_on_object_refname_ambiguity = save_warning; return retval; } diff --git a/fetch-object-info.c b/fetch-object-info.c index 30475a1e873d61..ba7e179c44ee54 100644 --- a/fetch-object-info.c +++ b/fetch-object-info.c @@ -55,6 +55,24 @@ int fetch_object_info(const enum protocol_version version, struct object_info_ar case protocol_v2: if (!server_supports_v2("object-info")) die(_("object-info capability is not enabled on the server")); + /* + * When removing an element from the list it gets swapped by the + * last element, iterate backwards to prevent elements skipping + * evaluation. + * + * object_info_options->nr can be safely casted without overflow + * because the number of options is a small known number (the + * supported placeholders which currently are size and type). + */ + for (int i = (int)args->object_info_options->nr - 1; i >= 0; i--) + if (!server_supports_feature("object-info", + args->object_info_options->items[i].string, 0)) + unsorted_string_list_delete_item(args->object_info_options, i, 0); + + /* + * Even if no options are left, we still send the oid so we get + * at least an existence check. + */ send_object_info_request(fd_out, args); break; case protocol_v1: @@ -71,7 +89,7 @@ int fetch_object_info(const enum protocol_version version, struct object_info_ar return -1; } - if (!string_list_has_string(args->object_info_options, reader->line)) + if (!unsorted_string_list_has_string(args->object_info_options, reader->line)) return -1; if (!strcmp(reader->line, "size")) { diff --git a/fetch-object-info.h b/fetch-object-info.h index 31aad98408a29b..269cebb3f7df48 100644 --- a/fetch-object-info.h +++ b/fetch-object-info.h @@ -14,6 +14,9 @@ struct object_info; /* * Sends git-cat-file object-info command into the request buf and read the * results from packets. + * + * Modifies args->object_info_options, on return it contains only the supported + * options by the server. */ int fetch_object_info(enum protocol_version version, struct object_info_args *args, struct packet_reader *reader, struct object_info *object_info_data, diff --git a/t/t1017-cat-file-remote-object-info.sh b/t/t1017-cat-file-remote-object-info.sh index edc20394d81b0a..116862f9d0b447 100755 --- a/t/t1017-cat-file-remote-object-info.sh +++ b/t/t1017-cat-file-remote-object-info.sh @@ -271,6 +271,34 @@ test_expect_success 'unsupported placeholder on remote returns empty string' ' ) ' +test_expect_success 'requesting only objectname echoes back' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + echo $hello_oid >expect && + git cat-file --batch-command="%(objectname)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'objectname goes through existence check' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + echo "$unstored_oid missing" >expect && + + git cat-file --batch-command="%(objectname)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $unstored_oid + EOF + + test_cmp expect actual + ) +' + # Test --batch-command remote-object-info with 'git://' and # transfer.advertiseobjectinfo set to false, i.e. server does not have object-info capability test_expect_success 'batch-command remote-object-info git:// fails when transfer.advertiseobjectinfo=false' ' diff --git a/transport.c b/transport.c index 93426805318e36..f0a6a455479800 100644 --- a/transport.c +++ b/transport.c @@ -443,7 +443,6 @@ static int fetch_object_info_via_pack(struct transport *transport) args.server_options = transport->server_options; args.oids = transport->smart_options->object_info_oids; args.object_info_options = transport->smart_options->object_info_options; - string_list_sort(args.object_info_options); connect_setup(transport, 0); packet_reader_init(&reader, data->fd[0], NULL, 0, From a97fcc37c2bc6340a8d7ce78dedf227aac4e9aa7 Mon Sep 17 00:00:00 2001 From: Junio C Hamano Date: Thu, 30 Jul 2026 10:31:52 -0700 Subject: [PATCH 47/47] The 9th batch Signed-off-by: Junio C Hamano --- Documentation/RelNotes/2.56.0.adoc | 44 ++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/Documentation/RelNotes/2.56.0.adoc b/Documentation/RelNotes/2.56.0.adoc index cf2da31b1a54a9..6cee8bb8e89134 100644 --- a/Documentation/RelNotes/2.56.0.adoc +++ b/Documentation/RelNotes/2.56.0.adoc @@ -61,6 +61,21 @@ UI, Workflows & Features simplification) by indenting them, preventing them from appearing falsely related to unrelated commits rendered immediately above them. + * Userdiff patterns for Swift have been added, with support for + Swift-specific constructs such as attributes, modifiers, failable + initializers, and generics. + + * Configuration file locking has been updated to retry for a short + period, avoiding failures when multiple processes attempt to update + the configuration simultaneously. + + * The 'remote-object-info' command has been added to 'git cat-file + --batch-command', allowing clients to request object metadata + (currently size) from a remote server via protocol v2 without + downloading the entire object. Format placeholders are dynamically + filtered on the client based on server-advertised capabilities, + returning empty strings for inapplicable or unsupported fields. + Performance, Internal Implementation, Development Support etc. -------------------------------------------------------------- @@ -241,6 +256,30 @@ Performance, Internal Implementation, Development Support etc. implicit dependency on the global 'the_repository' variable in 'copy.c'. + * The tempfile and lockfile APIs have been refactored to stop depending + on the 'the_repository' global variable, and their callers have been + updated to use the repository-aware variants. + + * The 'trust_executable_bit' (coming from the 'core.filemode' + configuration) has been migrated into 'struct repo_config_values' to + tie it to a specific repository instance. + + * The 'excludes_file' and various other global configuration variables + (including 'editor_program', 'pager_program', 'askpass_program', and + 'push_default') have been migrated into the per-repository structure. + + * The 'git stash push' command has been optimized to avoid unnecessary + sparse index expansion when pathspecs are wholly inside the + sparse-checkout cone. Also, a potential out-of-bounds read in the + sparse-index expansion check helper pathspec_needs_expanded_index() + has been fixed by consistently using the parsed, prefixed path. + + * The logic to write loose objects has been refactored and moved from + 'object-file.c' to the loose backend source file 'odb/source-loose.c', + making the loose backend more self-contained. This is achieved by + first refactoring force_object_loose() to use generic ODB write + interfaces instead of loose-backend internals. + Fixes since v2.55 ----------------- @@ -401,3 +440,8 @@ Fixes since v2.55 been corrected. The 'Clone' implementation of 'CryptoHasher' now properly initializes the context before cloning, and its 'Drop' implementation now discards the context to prevent leaks. + + * The object ID shortening and linking in the 'commitdiff' view of + 'gitweb' has been corrected to work even when the index line carries + a trailing file mode. + (merge fda513d6fe tl/gitweb-shorten-hashes-with-modes later to maint).