diff --git a/Documentation/RelNotes/2.56.0.adoc b/Documentation/RelNotes/2.56.0.adoc index cf2da31b1a54a9..6cee8bb8e89134 100644 --- a/Documentation/RelNotes/2.56.0.adoc +++ b/Documentation/RelNotes/2.56.0.adoc @@ -61,6 +61,21 @@ UI, Workflows & Features simplification) by indenting them, preventing them from appearing falsely related to unrelated commits rendered immediately above them. + * Userdiff patterns for Swift have been added, with support for + Swift-specific constructs such as attributes, modifiers, failable + initializers, and generics. + + * Configuration file locking has been updated to retry for a short + period, avoiding failures when multiple processes attempt to update + the configuration simultaneously. + + * The 'remote-object-info' command has been added to 'git cat-file + --batch-command', allowing clients to request object metadata + (currently size) from a remote server via protocol v2 without + downloading the entire object. Format placeholders are dynamically + filtered on the client based on server-advertised capabilities, + returning empty strings for inapplicable or unsupported fields. + Performance, Internal Implementation, Development Support etc. -------------------------------------------------------------- @@ -241,6 +256,30 @@ Performance, Internal Implementation, Development Support etc. implicit dependency on the global 'the_repository' variable in 'copy.c'. + * The tempfile and lockfile APIs have been refactored to stop depending + on the 'the_repository' global variable, and their callers have been + updated to use the repository-aware variants. + + * The 'trust_executable_bit' (coming from the 'core.filemode' + configuration) has been migrated into 'struct repo_config_values' to + tie it to a specific repository instance. + + * The 'excludes_file' and various other global configuration variables + (including 'editor_program', 'pager_program', 'askpass_program', and + 'push_default') have been migrated into the per-repository structure. + + * The 'git stash push' command has been optimized to avoid unnecessary + sparse index expansion when pathspecs are wholly inside the + sparse-checkout cone. Also, a potential out-of-bounds read in the + sparse-index expansion check helper pathspec_needs_expanded_index() + has been fixed by consistently using the parsed, prefixed path. + + * The logic to write loose objects has been refactored and moved from + 'object-file.c' to the loose backend source file 'odb/source-loose.c', + making the loose backend more self-contained. This is achieved by + first refactoring force_object_loose() to use generic ODB write + interfaces instead of loose-backend internals. + Fixes since v2.55 ----------------- @@ -401,3 +440,8 @@ Fixes since v2.55 been corrected. The 'Clone' implementation of 'CryptoHasher' now properly initializes the context before cloning, and its 'Drop' implementation now discards the context to prevent leaks. + + * The object ID shortening and linking in the 'commitdiff' view of + 'gitweb' has been corrected to work even when the index line carries + a trailing file mode. + (merge fda513d6fe tl/gitweb-shorten-hashes-with-modes later to maint). diff --git a/Documentation/config/core.adoc b/Documentation/config/core.adoc index a0ebf03e2eb050..340329edc38143 100644 --- a/Documentation/config/core.adoc +++ b/Documentation/config/core.adoc @@ -589,6 +589,14 @@ core.packedRefsTimeout:: all; -1 means to try indefinitely. Default is 1000 (i.e., retry for 1 second). +core.configLockTimeout:: + The length of time, in milliseconds, to retry when trying to + lock a configuration file for writing. Value 0 means not to + retry at all; -1 means to try indefinitely. Default is 1000 + (i.e., retry for 1 second). This is read from the configuration + that is already on disk before the lock is taken, so it can be + set persistently like any other option. + core.pager:: Text viewer for use by Git commands (e.g., 'less'). The value is meant to be interpreted by the shell. The order of preference diff --git a/Documentation/git-cat-file.adoc b/Documentation/git-cat-file.adoc index 86b9181599317e..ac3b528c6f00f6 100644 --- a/Documentation/git-cat-file.adoc +++ b/Documentation/git-cat-file.adoc @@ -169,6 +169,13 @@ info :: Print object info for object reference ``. This corresponds to the output of `--batch-check`. +remote-object-info ...:: + Print object info for object references `` at specified + `` without downloading objects from the remote. + Raise an error when the `object-info` capability is not supported by the remote. + Raise an error when no object references are provided. + This command may be combined with `--buffer`. + flush:: Used with `--buffer` to execute all preceding commands that were issued since the beginning or since the last flush was issued. When `--buffer` @@ -301,7 +308,8 @@ one per line, and print information based on the command given. With `--batch-command`, the `info` command followed by an object will print information about the object the same way `--batch-check` would, and the `contents` command followed by an object prints contents in the same way -`--batch` would. +`--batch` would. The `remote-object-info` command followed by a remote and +object IDs prints object info from the remote without downloading the objects. You can specify the information shown for each object by using a custom ``. The `` is copied literally to stdout for each @@ -340,8 +348,15 @@ newline. The available atoms are: after that first run of whitespace (i.e., the "rest" of the line) are output in place of the `%(rest)` atom. +The command `remote-object-info` only supports the `%(objectname)` and +`%(objectsize)` placeholders. See `CAVEATS` below for more information. + If no format is specified, the default format is `%(objectname) -%(objecttype) %(objectsize)`. +%(objecttype) %(objectsize)`, except for `remote-object-info` commands which +use `%(objectname) %(objectsize)` because `%(objecttype)` is not supported yet. + +WARNING: When "%(objecttype)" is supported, the default format WILL be unified, +so DO NOT RELY on the current default format to stay the same!!! If `--batch` is specified, or if `--batch-command` is used with the `contents` command, the object information is followed by the object contents (consisting @@ -438,6 +453,10 @@ scripting purposes. CAVEATS ------- +Note that only `%(objectname)` and `%(objectsize)` are currently +supported by the `remote-object-info` command. Using any other placeholder in +the format string will return an empty string in its position. + Note that the sizes of objects on disk are reported accurately, but care should be taken in drawing conclusions about which refs or objects are responsible for disk usage. The size of a packed non-delta object may be diff --git a/Documentation/gitattributes.adoc b/Documentation/gitattributes.adoc index bd76167a45eb71..9fea75f96f928f 100644 --- a/Documentation/gitattributes.adoc +++ b/Documentation/gitattributes.adoc @@ -914,6 +914,8 @@ patterns are available: - `scheme` suitable for source code in most Lisp dialects, including Scheme, Emacs Lisp, Common Lisp, and Clojure. +- `swift` suitable for source code in the Swift language. + - `tex` suitable for source code for LaTeX documents. diff --git a/Documentation/gitprotocol-v2.adoc b/Documentation/gitprotocol-v2.adoc index 2beb70595fc1e5..7bf62014c3917d 100644 --- a/Documentation/gitprotocol-v2.adoc +++ b/Documentation/gitprotocol-v2.adoc @@ -568,21 +568,26 @@ An `object-info` request takes the following arguments: oid Indicates to the server an object which the client wants to obtain - information for. + information for. They must be full OIDs. -The response of `object-info` is a list of the requested object ids -and associated requested information, each separated by a single space. +The response of `object-info` consists of one pkt-line per requested attribute, +echoing the attributes the server will report, followed by one pkt-line per +requested object id with its information, each field separated by a single +space. output = info flush-pkt - info = PKT-LINE(attrs) LF) - *PKT-LINE(obj-info LF) - - attrs = attr | attrs SP attrs + info = *PKT-LINE(attr LF) + *PKT-LINE(obj-info LF) attr = "size" - obj-info = obj-id SP obj-size + obj-size = 1*DIGIT + + obj-info = obj-id [SP [obj-size]] + +If the server does not recognize the OID, the response will be ` SP` +regardless of the number of attributes requested. bundle-uri ~~~~~~~~~~ diff --git a/Makefile b/Makefile index 98e995e4be0967..fac3e8879c8377 100644 --- a/Makefile +++ b/Makefile @@ -1169,6 +1169,7 @@ LIB_OBJS += ewah/ewah_io.o LIB_OBJS += ewah/ewah_rlw.o LIB_OBJS += exec-cmd.o LIB_OBJS += fetch-negotiator.o +LIB_OBJS += fetch-object-info.o LIB_OBJS += fetch-pack.o LIB_OBJS += fmt-merge-msg.o LIB_OBJS += fsck.o diff --git a/apply.c b/apply.c index 5e87ba772e8531..f00b7ba4d3a7e6 100644 --- a/apply.c +++ b/apply.c @@ -47,11 +47,17 @@ struct gitdiff_data { int p_value; }; -static void git_apply_config(void) +static void git_apply_config(struct repository *repo) { - repo_config_get_string(the_repository, "apply.whitespace", &apply_default_whitespace); - repo_config_get_string(the_repository, "apply.ignorewhitespace", &apply_default_ignorewhitespace); - repo_config(the_repository, git_xmerge_config, NULL); + struct repo_config_values *cfg = repo_config_values(repo); + + FREE_AND_NULL(cfg->apply_default_whitespace); + repo_config_get_string(repo, "apply.whitespace", + &cfg->apply_default_whitespace); + FREE_AND_NULL(cfg->apply_default_ignorewhitespace); + repo_config_get_string(repo, "apply.ignorewhitespace", + &cfg->apply_default_ignorewhitespace); + repo_config(repo, git_xmerge_config, NULL); } static int parse_whitespace_option(struct apply_state *state, const char *option) @@ -109,6 +115,8 @@ int init_apply_state(struct apply_state *state, struct repository *repo, const char *prefix) { + struct repo_config_values *cfg = repo_config_values(repo); + memset(state, 0, sizeof(*state)); state->prefix = prefix; state->repo = repo; @@ -126,10 +134,13 @@ int init_apply_state(struct apply_state *state, strset_init(&state->kept_symlinks); strbuf_init(&state->root, 0); - git_apply_config(); - if (apply_default_whitespace && parse_whitespace_option(state, apply_default_whitespace)) + git_apply_config(repo); + + if (cfg->apply_default_whitespace && + parse_whitespace_option(state, cfg->apply_default_whitespace)) return -1; - if (apply_default_ignorewhitespace && parse_ignorewhitespace_option(state, apply_default_ignorewhitespace)) + if (cfg->apply_default_ignorewhitespace && + parse_ignorewhitespace_option(state, cfg->apply_default_ignorewhitespace)) return -1; return 0; } @@ -192,7 +203,8 @@ int check_apply_state(struct apply_state *state, int force_apply) static void set_default_whitespace_mode(struct apply_state *state) { - if (!state->whitespace_option && !apply_default_whitespace) + if (!state->whitespace_option && + !repo_config_values(state->repo)->apply_default_whitespace) state->ws_error_action = (state->apply ? warn_on_ws_error : nowarn_ws_error); } @@ -3893,8 +3905,8 @@ static int check_preimage(struct apply_state *state, if (*ce && !(*ce)->ce_mode) BUG("ce_mode == 0 for path '%s'", old_name); - if (trust_executable_bit || !S_ISREG(st->st_mode)) - st_mode = ce_mode_from_stat(*ce, st->st_mode); + if (repo_trust_executable_bit(state->repo) || !S_ISREG(st->st_mode)) + st_mode = ce_mode_from_stat(state->repo, *ce, st->st_mode); else if (*ce) st_mode = (*ce)->ce_mode; else @@ -4287,7 +4299,8 @@ static int build_fake_ancestor(struct apply_state *state, struct patch *list) } } - hold_lock_file_for_update(&lock, state->fake_ancestor, LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(state->repo, &lock, state->fake_ancestor, + LOCK_DIE_ON_ERROR); res = write_locked_index(&result, &lock, COMMIT_LOCK); discard_index(&result); @@ -4511,7 +4524,7 @@ static int try_create_file(struct apply_state *state, const char *path, return !!mkdir(path, 0777); } - if (has_symlinks && S_ISLNK(mode)) + if (repo_has_symlinks(state->repo) && S_ISLNK(mode)) /* Although buf:size is counted string, it also is NUL * terminated. */ @@ -4945,9 +4958,10 @@ static int apply_patch(struct apply_state *state, state->update_index = (state->check_index || state->ita_only) && state->apply; if (state->update_index && !is_lock_file_locked(&state->lock_file)) { if (state->index_file) - hold_lock_file_for_update(&state->lock_file, - state->index_file, - LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(state->repo, + &state->lock_file, + state->index_file, + LOCK_DIE_ON_ERROR); else repo_hold_locked_index(state->repo, &state->lock_file, LOCK_DIE_ON_ERROR); diff --git a/branch.c b/branch.c index 4f38905bad928d..3a12e2a9ebeeb6 100644 --- a/branch.c +++ b/branch.c @@ -61,7 +61,7 @@ static int find_tracked_branch(struct remote *remote, void *priv) static int should_setup_rebase(const char *origin) { - switch (autorebase) { + switch (repo_config_values(the_repository)->autorebase) { case AUTOREBASE_NEVER: return 0; case AUTOREBASE_LOCAL: diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 1458dd76d683e0..884b6d5ad348b5 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -28,6 +28,22 @@ #include "promisor-remote.h" #include "mailmap.h" #include "write-or-die.h" +#include "alias.h" +#include "remote.h" +#include "transport.h" + +/* + * Maximum length for a remote URL. While no universal standard exists, + * 8K is assumed to be a reasonable limit. + */ +#define MAX_REMOTE_URL_LEN (8 * 1024) + +/* Maximum number of objects allowed in a single remote-object-info request. */ +#define MAX_ALLOWED_OBJ_LIMIT 10000 + +/* Maximum input size permitted for the remote-object-info command. */ +#define MAX_REMOTE_OBJ_INFO_LINE \ + (MAX_REMOTE_URL_LEN + MAX_ALLOWED_OBJ_LIMIT * (GIT_MAX_HEXSZ + 1)) enum batch_mode { BATCH_MODE_CONTENTS, @@ -316,8 +332,24 @@ struct expand_data { * optimized out. */ unsigned skip_object_info : 1; + + /* + * Flags about when an object info is being fetched from remote. + */ + unsigned is_remote:1; + + /* + * List of atoms (i.e. "objectsize") that the server supports. Built + * from the server's object-info advertised capabilities. + */ + struct string_list remote_allowed_atoms; }; -#define EXPAND_DATA_INIT { .mode = S_IFINVALID } + +#define EXPAND_DATA_INIT { \ + .mode = S_IFINVALID, \ + .type = OBJ_BAD, \ + .remote_allowed_atoms = STRING_LIST_INIT_NODUP, \ +} static int is_atom(const char *atom, const char *s, int slen) { @@ -328,14 +360,26 @@ static int is_atom(const char *atom, const char *s, int slen) static int expand_atom(struct strbuf *sb, const char *atom, int len, struct expand_data *data) { + if (data->is_remote) { + size_t i; + for (i = 0; i < data->remote_allowed_atoms.nr; i++) + if (is_atom(data->remote_allowed_atoms.items[i].string, + atom, len)) + break; + if (i == data->remote_allowed_atoms.nr) + return 1; + } + if (is_atom("objectname", atom, len)) { if (!data->mark_query) strbuf_add_oid_hex(sb, &data->oid); } else if (is_atom("objecttype", atom, len)) { - if (data->mark_query) + if (data->mark_query) { data->info.typep = &data->type; - else - strbuf_addstr(sb, type_name(data->type)); + } else { + const char *t = type_name(data->type); + strbuf_addstr(sb, t ? t : ""); + } } else if (is_atom("objectsize", atom, len)) { if (data->mark_query) data->info.sizep = &data->size; @@ -635,6 +679,62 @@ static void batch_one_object(const char *obj_name, object_context_release(&ctx); } +static int get_remote_info(int argc, + const char **argv, + struct object_info **remote_object_info, + struct oid_array *object_info_oids, + struct string_list *object_info_options) +{ + int retval = 0; + struct remote *remote = NULL; + struct object_id oid; + struct transport *gtransport; + + remote = remote_get(argv[0]); + if (!remote) + die(_("must supply valid remote when using remote-object-info")); + + oid_array_clear(object_info_oids); + for (size_t i = 1; i < argc; i++) { + if (get_oid_hex(argv[i], &oid)) { + size_t len = strlen(argv[i]); + + if (len < the_hash_algo->hexsz && len >= 4) { + size_t j; + for (j = 0; j < len; j++) + if (!isxdigit(argv[i][j])) + break; + if (j == len) + die(_("remote-object-info does not support " + "short oids, %d characters required"), + (int)the_hash_algo->hexsz); + } + die(_("not a valid object name '%s'"), argv[i]); + } + oid_array_append(object_info_oids, &oid); + } + + if (!object_info_oids->nr) + die(_("remote-object-info requires objects")); + + gtransport = transport_get(remote, NULL); + + if (!gtransport->smart_options) { + retval = -1; + goto cleanup; + } + + CALLOC_ARRAY(*remote_object_info, object_info_oids->nr); + gtransport->smart_options->object_info_oids = object_info_oids; + + gtransport->smart_options->object_info_options = object_info_options; + gtransport->smart_options->object_info_data = *remote_object_info; + retval = transport_fetch_object_info(gtransport); +cleanup: + transport_disconnect(gtransport); + return retval; +} + struct object_cb_data { struct batch_options *opt; struct expand_data *expand; @@ -716,18 +816,115 @@ static void parse_cmd_mailmap(struct batch_options *opt UNUSED, load_mailmap(); } +struct protocol_placeholder_entry { + const char *option; + const char *atom; +}; + +static const struct protocol_placeholder_entry remote_atom_map[] = { + {"size", "objectsize"}, + {"type", "objecttype"}, + /* + * Add new protocol options here. Even if the server doesn't support + * them the allow_list will drop them if the server doesn't advertise + * them. + */ +}; + +static void parse_cmd_remote_object_info(struct batch_options *opt, + const char *line, struct strbuf *output, + struct expand_data *data) +{ + int count; + const char **argv; + char *line_to_split; + struct object_info *remote_object_info = NULL; + struct oid_array object_info_oids = OID_ARRAY_INIT; + struct string_list object_info_options = STRING_LIST_INIT_NODUP; + const char *saved_format = opt->format; + + if (strlen(line) >= MAX_REMOTE_OBJ_INFO_LINE) + die(_("remote-object-info command too long")); + /* + * TODO: Use the default format once %(objecttype) is supported. + */ + if (!opt->format) + opt->format = "%(objectname) %(objectsize)"; + + line_to_split = xstrdup(line); + count = split_cmdline(line_to_split, &argv); + if (count < 0) + die(_("remote-object-info: failed to parse command line: %s"), + split_cmdline_strerror(count)); + if (count - 1 > MAX_ALLOWED_OBJ_LIMIT) + die(_("remote-object-info supports at most %d objects"), + MAX_ALLOWED_OBJ_LIMIT); + + if (data->info.sizep) + string_list_append(&object_info_options, "size"); + if (data->info.typep) + string_list_append(&object_info_options, "type"); + + if (get_remote_info(count, argv, &remote_object_info, + &object_info_oids, &object_info_options)) + die(_("failed to get object info from the remote: %s"), argv[0]); + + string_list_clear(&data->remote_allowed_atoms, 0); + string_list_append(&data->remote_allowed_atoms, "objectname"); + for (size_t i = 0; i < ARRAY_SIZE(remote_atom_map); i++) + if (unsorted_string_list_has_string(&object_info_options, remote_atom_map[i].option)) + string_list_append(&data->remote_allowed_atoms, + remote_atom_map[i].atom); + + data->skip_object_info = 1; + for (size_t i = 0; i < object_info_oids.nr; i++) { + data->oid = object_info_oids.oid[i]; + + if (remote_object_info[i].unrecognized) { + report_object_status(opt, oid_to_hex(&data->oid), + &data->oid, "missing"); + continue; + } + + /* + * When reaching here, it means remote-object-info can retrieve + * information from server without downloading them. + */ + if (remote_object_info[i].sizep) { + data->size = *remote_object_info[i].sizep; + } + + if (remote_object_info[i].typep) { + data->type = *remote_object_info[i].typep; + } + + opt->batch_mode = BATCH_MODE_INFO; + data->is_remote = 1; + batch_object_write(argv[i + 1], output, opt, data, NULL, 0); + data->is_remote = 0; + } + data->skip_object_info = 0; + opt->format = saved_format; + + for (size_t i = 0; i < object_info_oids.nr; i++) + free_object_info_contents(&remote_object_info[i]); + string_list_clear(&object_info_options, 0); + free(line_to_split); + free(argv); + free(remote_object_info); + oid_array_clear(&object_info_oids); +} + static void dispatch_calls(struct batch_options *opt, struct strbuf *output, struct expand_data *data, struct queued_cmd *cmd, - int nr) + size_t nr) { - int i; - if (!opt->buffer_output) die(_("flush is only for --buffer mode")); - for (i = 0; i < nr; i++) + for (size_t i = 0; i < nr; i++) cmd[i].fn(opt, cmd[i].line, output, data); fflush(stdout); @@ -735,9 +932,7 @@ static void dispatch_calls(struct batch_options *opt, static void free_cmds(struct queued_cmd *cmd, size_t *nr) { - size_t i; - - for (i = 0; i < *nr; i++) + for (size_t i = 0; i < *nr; i++) FREE_AND_NULL(cmd[i].line); *nr = 0; @@ -750,9 +945,10 @@ static const struct parse_cmd { unsigned takes_args; } commands[] = { { "contents", parse_cmd_contents, 1 }, - { "info", parse_cmd_info, 1 }, { "flush", NULL, 0 }, + { "info", parse_cmd_info, 1 }, { "mailmap", parse_cmd_mailmap, 1 }, + { "remote-object-info", parse_cmd_remote_object_info, 1 }, }; static void batch_objects_command(struct batch_options *opt, @@ -764,7 +960,6 @@ static void batch_objects_command(struct batch_options *opt, size_t alloc = 0, nr = 0; while (strbuf_getdelim_strip_crlf(&input, stdin, opt->input_delim) != EOF) { - int i; const struct parse_cmd *cmd = NULL; const char *p = NULL, *cmd_end; struct queued_cmd call = {0}; @@ -774,7 +969,7 @@ static void batch_objects_command(struct batch_options *opt, if (isspace(*input.buf)) die(_("whitespace before command: '%s'"), input.buf); - for (i = 0; i < ARRAY_SIZE(commands); i++) { + for (size_t i = 0; i < ARRAY_SIZE(commands); i++) { if (!skip_prefix(input.buf, commands[i].name, &cmd_end)) continue; @@ -972,6 +1167,7 @@ static int batch_objects(struct batch_options *opt) cleanup: strbuf_release(&input); strbuf_release(&output); + string_list_clear(&data.remote_allowed_atoms, 0); cfg->warn_on_object_refname_ambiguity = save_warning; return retval; } diff --git a/builtin/difftool.c b/builtin/difftool.c index 5e7777fbe48daa..bc7b2ea44396bb 100644 --- a/builtin/difftool.c +++ b/builtin/difftool.c @@ -636,7 +636,7 @@ static int run_dir_diff(struct repository *repo, struct lock_file lock = LOCK_INIT; strbuf_reset(&buf); strbuf_addf(&buf, "%s/wtindex", tmpdir.buf); - if (hold_lock_file_for_update(&lock, buf.buf, 0) < 0 || + if (repo_hold_lock_file_for_update(repo, &lock, buf.buf, 0) < 0 || write_locked_index(&wtindex, &lock, COMMIT_LOCK)) { ret = error("could not write %s", buf.buf); goto finish; diff --git a/builtin/gc.c b/builtin/gc.c index 46999a99abff42..49c8474fade8ed 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -1790,7 +1790,7 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts, struct repository *r = the_repository; char *lock_path = xstrfmt("%s/maintenance", r->objects->sources->path); - if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) { + if (repo_hold_lock_file_for_update(r, &lk, lock_path, LOCK_NO_DEREF) < 0) { /* * Another maintenance command is running. * diff --git a/builtin/history.c b/builtin/history.c index d28c1f08bb66ea..000155ad9c24d5 100644 --- a/builtin/history.c +++ b/builtin/history.c @@ -795,7 +795,7 @@ static int write_ondisk_index(struct repository *repo, prime_cache_tree(repo, &index, tree); - if (hold_lock_file_for_update(&lock, path, 0) < 0) { + if (repo_hold_lock_file_for_update(repo, &lock, path, 0) < 0) { ret = error_errno(_("unable to acquire index lock")); goto out; } diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 3673b14b89b275..1ec5b6f206366e 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -32,6 +32,7 @@ #include "list.h" #include "packfile.h" #include "object-file.h" +#include "object-file-convert.h" #include "odb.h" #include "odb/streaming.h" #include "replace-object.h" @@ -4621,6 +4622,51 @@ static int loosened_object_can_be_discarded(const struct object_id *oid, return 1; } +static int force_object_loose(struct odb_source *source, + const struct object_id *oid, + const time_t *mtime) +{ + struct odb_source_files *files = odb_source_files_downcast(source); + const struct git_hash_algo *compat = source->odb->repo->compat_hash_algo; + struct object_info oi = OBJECT_INFO_INIT; + struct object_id compat_oid, *compat_oid_p = NULL; + enum object_type type; + void *buf = NULL; + size_t len; + int ret; + + for (struct odb_source *s = source->odb->sources; s; s = s->next) { + struct odb_source_files *files = odb_source_files_downcast(s); + if (!odb_source_read_object_info(&files->loose->base, oid, NULL, 0)) + return 0; + } + + oi.typep = &type; + oi.sizep = &len; + oi.contentp = &buf; + if (odb_read_object_info_extended(source->odb, oid, &oi, 0)) { + ret = error(_("cannot read object for %s"), oid_to_hex(oid)); + goto out; + } + + if (compat) { + if (repo_oid_to_algop(source->odb->repo, oid, compat, &compat_oid)) { + ret = error(_("cannot map object %s to %s"), + oid_to_hex(oid), compat->name); + goto out; + } + + compat_oid_p = &compat_oid; + } + + ret = odb_source_write_object(&files->loose->base, buf, len, type, oid, + compat_oid_p, mtime, 0); + +out: + free(buf); + return ret; +} + static void loosen_unused_packed_objects(void) { struct packed_git *p; @@ -4641,7 +4687,7 @@ static void loosen_unused_packed_objects(void) !has_sha1_pack_kept_or_nonlocal(&oid) && !loosened_object_can_be_discarded(&oid, p->mtime)) { if (force_object_loose(the_repository->objects->sources, - &oid, p->mtime)) + &oid, &p->mtime)) die(_("unable to force loose object")); loosened_objects_nr++; } diff --git a/builtin/push.c b/builtin/push.c index 8ccdb07c404874..2377b5af554bda 100644 --- a/builtin/push.c +++ b/builtin/push.c @@ -74,6 +74,7 @@ static void refspec_append_mapped(struct refspec *refspec, const char *ref, struct remote *remote, struct ref *matched) { const char *branch_name; + struct repo_config_values *cfg = repo_config_values(the_repository); if (remote->push.nr) { struct refspec_item query = { @@ -89,7 +90,7 @@ static void refspec_append_mapped(struct refspec *refspec, const char *ref, } } - if (push_default == PUSH_DEFAULT_UPSTREAM && + if (cfg->push_default == PUSH_DEFAULT_UPSTREAM && skip_prefix(matched->name, "refs/heads/", &branch_name)) { struct branch *branch = branch_get(branch_name); if (branch->merge_nr == 1 && branch->merge[0]->src) { @@ -161,7 +162,7 @@ static NORETURN void die_push_simple(struct branch *branch, * Don't show advice for people who explicitly set * push.default. */ - if (push_default == PUSH_DEFAULT_UNSPECIFIED) + if (cfg->push_default == PUSH_DEFAULT_UNSPECIFIED) advice_pushdefault_maybe = _("\n" "To choose either option permanently, " "see push.default in 'git help config'.\n"); @@ -232,8 +233,9 @@ static void setup_default_push_refspecs(int *flags, struct remote *remote) struct branch *branch; const char *dst; int same_remote; + struct repo_config_values *cfg = repo_config_values(the_repository); - switch (push_default) { + switch (cfg->push_default) { case PUSH_DEFAULT_MATCHING: refspec_append(&rs, ":"); return; @@ -253,7 +255,7 @@ static void setup_default_push_refspecs(int *flags, struct remote *remote) dst = branch->refname; same_remote = !strcmp(remote->name, remote_for_branch(branch, NULL)); - switch (push_default) { + switch (cfg->push_default) { default: case PUSH_DEFAULT_UNSPECIFIED: case PUSH_DEFAULT_SIMPLE: diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 0863d0fb460cf8..cb4a037b770291 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -341,7 +341,8 @@ static int write_patterns_and_update(struct repository *repo, if (safe_create_leading_directories(repo, sparse_filename)) die(_("failed to create directory for sparse-checkout file")); - hold_lock_file_for_update(&lk, sparse_filename, LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(repo, &lk, sparse_filename, + LOCK_DIE_ON_ERROR); result = update_working_directory(repo, pl); if (result) { diff --git a/builtin/stash.c b/builtin/stash.c index c4809f299a313b..72c52571f8c06c 100644 --- a/builtin/stash.c +++ b/builtin/stash.c @@ -1702,8 +1702,8 @@ static int do_push_stash(const struct pathspec *ps, const char *stash_msg, int q if (!include_untracked && ps->nr) { char *ps_matched = xcalloc(ps->nr, 1); - /* TODO: audit for interaction with sparse-index. */ - ensure_full_index(the_repository->index); + if (pathspec_needs_expanded_index(the_repository->index, ps)) + ensure_full_index(the_repository->index); for (size_t i = 0; i < the_repository->index->cache_nr; i++) ce_path_match(the_repository->index, the_repository->index->cache[i], ps, ps_matched); diff --git a/builtin/update-index.c b/builtin/update-index.c index 4c4b39a1575a38..241abd4332dcf9 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -294,7 +294,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len ce->ce_flags = create_ce_flags(0); ce->ce_namelen = len; fill_stat_cache_info(the_repository->index, ce, st); - ce->ce_mode = ce_mode_from_stat(old, st->st_mode); + ce->ce_mode = ce_mode_from_stat(the_repository, old, st->st_mode); if (index_path(the_repository->index, &ce->oid, path, st, info_only ? 0 : INDEX_WRITE_OBJECT)) { diff --git a/bundle.c b/bundle.c index fd2db2c837df60..b64716f252b78f 100644 --- a/bundle.c +++ b/bundle.c @@ -519,8 +519,8 @@ int create_bundle(struct repository *r, const char *path, if (bundle_to_stdout) bundle_fd = 1; else - bundle_fd = hold_lock_file_for_update(&lock, path, - LOCK_DIE_ON_ERROR); + bundle_fd = repo_hold_lock_file_for_update(r, &lock, path, + LOCK_DIE_ON_ERROR); if (version == -1) version = min_version; diff --git a/combine-diff.c b/combine-diff.c index fb72174918786a..717d5376888f8e 100644 --- a/combine-diff.c +++ b/combine-diff.c @@ -1080,7 +1080,7 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent, /* if symlinks don't work, assume symlink if all parents * are symlinks */ - is_file = has_symlinks; + is_file = repo_has_symlinks(rev->repo); for (i = 0; !is_file && i < num_parent; i++) is_file = !S_ISLNK(elem->parent[i].mode); if (!is_file) diff --git a/commit-graph.c b/commit-graph.c index ce2e020759ca25..49e8f639305212 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -2127,8 +2127,8 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx) if (ctx->split) { char *lock_name = get_commit_graph_chain_filename(ctx->odb_source); - hold_lock_file_for_update_mode(&lk, lock_name, - LOCK_DIE_ON_ERROR, 0444); + repo_hold_lock_file_for_update_mode(ctx->r, &lk, lock_name, + LOCK_DIE_ON_ERROR, 0444); free(lock_name); graph_layer = mks_tempfile_m(ctx->graph_name, 0444); @@ -2146,8 +2146,9 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx) f = hashfd(ctx->r->hash_algo, get_tempfile_fd(graph_layer), get_tempfile_path(graph_layer)); } else { - hold_lock_file_for_update_mode(&lk, ctx->graph_name, - LOCK_DIE_ON_ERROR, 0444); + repo_hold_lock_file_for_update_mode(ctx->r, &lk, + ctx->graph_name, + LOCK_DIE_ON_ERROR, 0444); f = hashfd(ctx->r->hash_algo, get_lock_file_fd(&lk), get_lock_file_path(&lk)); } diff --git a/compat/mingw.c b/compat/mingw.c index 3eca3a7f2e87b2..e0fbd2c66de23d 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -7,6 +7,7 @@ #include "config.h" #include "dir.h" #include "environment.h" +#include "repository.h" #include "gettext.h" #include "run-command.h" #include "strbuf.h" @@ -1044,7 +1045,7 @@ int mingw_chdir(const char *dirname) if (xutftowcs_path(wdirname, dirname) < 0) return -1; - if (has_symlinks) { + if (repo_has_symlinks(the_repository)) { HANDLE hnd = CreateFileW(wdirname, 0, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL); @@ -2914,7 +2915,7 @@ int symlink(const char *target, const char *link) int len; /* fail if symlinks are disabled or API is not supported (WinXP) */ - if (!has_symlinks) { + if (!repo_has_symlinks(the_repository)) { errno = ENOSYS; return -1; } @@ -3184,15 +3185,23 @@ static void setup_windows_environment(void) if (!tmp && (tmp = getenv("USERPROFILE"))) setenv("HOME", tmp, 1); } +} +int mingw_platform_has_symlinks(void) +{ + static int has_symlinks = -1; /* * Change 'core.symlinks' default to false, unless native symlinks are * enabled in MSys2 (via 'MSYS=winsymlinks:nativestrict'). Thus we can * run the test suite (which doesn't obey config files) with or without * symlink support. */ - if (!(tmp = getenv("MSYS")) || !strstr(tmp, "winsymlinks:nativestrict")) - has_symlinks = 0; + if (has_symlinks < 0) { + const char *tmp = getenv("MSYS"); + has_symlinks = (tmp && strstr(tmp, "winsymlinks:nativestrict")) ? 1 : 0; + } + + return has_symlinks; } static void get_current_user_sid(PSID *sid, HANDLE *linked_token) diff --git a/compat/mingw.h b/compat/mingw.h index 444daedfa52469..df02aeb632d8c3 100644 --- a/compat/mingw.h +++ b/compat/mingw.h @@ -208,6 +208,9 @@ void open_in_gdb(void); */ int err_win_to_posix(DWORD winerr); +int mingw_platform_has_symlinks(void); +#define platform_has_symlinks() mingw_platform_has_symlinks() + #ifndef NO_UNIX_SOCKETS int mingw_have_unix_sockets(void); #undef have_unix_sockets diff --git a/config.c b/config.c index 9d6d7872d76c10..1bdd702e7a3969 100644 --- a/config.c +++ b/config.c @@ -2954,6 +2954,24 @@ char *git_config_prepare_comment_string(const char *comment) return prepared; } +/* + * How long to retry acquiring config.lock when another process holds + * it. Default matches core.packedRefsTimeout; override via + * core.configLockTimeout. + */ +static long config_lock_timeout_ms(struct repository *r) +{ + static int configured; + static int timeout_ms = 1000; + + if (!configured) { + repo_config_get_int(r, "core.configlocktimeout", &timeout_ms); + configured = 1; + } + + return timeout_ms; +} + static void validate_comment_string(const char *comment) { size_t leading_blanks; @@ -3037,7 +3055,8 @@ int repo_config_set_multivar_in_file_gently(struct repository *r, * The lock serves a purpose in addition to locking: the new * contents of .git/config will be written into it. */ - fd = hold_lock_file_for_update(&lock, config_filename, 0); + fd = repo_hold_lock_file_for_update_timeout(r, &lock, config_filename, 0, + config_lock_timeout_ms(r)); if (fd < 0) { error_errno(_("could not lock config file %s"), config_filename); ret = CONFIG_NO_LOCK; @@ -3382,7 +3401,9 @@ static int repo_config_copy_or_rename_section_in_file( if (!config_filename) config_filename = filename_buf = repo_git_path(r, "config"); - out_fd = hold_lock_file_for_update(&lock, config_filename, 0); + out_fd = repo_hold_lock_file_for_update_timeout(r, &lock, + config_filename, 0, + config_lock_timeout_ms(r)); if (out_fd < 0) { ret = error(_("could not lock config file %s"), config_filename); goto out; diff --git a/connect.c b/connect.c index 1d74c1eda21f93..feb0420b0f7e3e 100644 --- a/connect.c +++ b/connect.c @@ -709,6 +709,40 @@ int server_supports(const char *feature) return !!server_feature_value(feature, NULL); } +void write_command_and_capabilities(struct strbuf *req_buf, const char *command, + const struct string_list *server_options) +{ + const char *hash_name; + int advertise_sid = 0; + + repo_config_get_bool(the_repository, "transfer.advertisesid", &advertise_sid); + + ensure_server_supports_v2(command); + packet_buf_write(req_buf, "command=%s", command); + if (server_supports_v2("agent")) + packet_buf_write(req_buf, "agent=%s", git_user_agent_sanitized()); + if (advertise_sid && server_supports_v2("session-id")) + packet_buf_write(req_buf, "session-id=%s", trace2_session_id()); + if (server_options && server_options->nr) { + ensure_server_supports_v2("server-option"); + for (size_t i = 0; i < server_options->nr; i++) + packet_buf_write(req_buf, "server-option=%s", + server_options->items[i].string); + } + + if (server_feature_v2("object-format", &hash_name)) { + const unsigned int hash_algo = hash_algo_by_name(hash_name); + if (hash_algo_by_ptr(the_hash_algo) != hash_algo) + die(_("mismatched algorithms: client %s; server %s"), + the_hash_algo->name, hash_name); + packet_buf_write(req_buf, "object-format=%s", the_hash_algo->name); + } else if (hash_algo_by_ptr(the_hash_algo) != GIT_HASH_SHA1_LEGACY) { + die(_("the server does not support algorithm '%s'"), + the_hash_algo->name); + } + packet_buf_delim(req_buf); +} + static const char *url_scheme_name(enum url_scheme scheme) { switch (scheme) { diff --git a/connect.h b/connect.h index aa482a37fb4da4..957e5fe2b9eccb 100644 --- a/connect.h +++ b/connect.h @@ -34,4 +34,12 @@ void check_stateless_delimiter(int stateless_rpc, struct packet_reader *reader, const char *error); +struct string_list; +/* + * Write a protocol v2 command request, along with the capability + * advertisements, into req_buf. + */ +void write_command_and_capabilities(struct strbuf *req_buf, const char *command, + const struct string_list *server_options); + #endif diff --git a/diff-lib.c b/diff-lib.c index ae91027a024eec..46cae637ecda83 100644 --- a/diff-lib.c +++ b/diff-lib.c @@ -160,7 +160,7 @@ void run_diff_files(struct rev_info *revs, unsigned int option) changed = check_removed(ce, &st); if (!changed) - wt_mode = ce_mode_from_stat(ce, st.st_mode); + wt_mode = ce_mode_from_stat(revs->repo, ce, st.st_mode); else { if (changed < 0) { perror(ce->name); @@ -193,7 +193,7 @@ void run_diff_files(struct rev_info *revs, unsigned int option) num_compare_stages++; oidcpy(&dpath->parent[stage - 2].oid, &nce->oid); - dpath->parent[stage-2].mode = ce_mode_from_stat(nce, mode); + dpath->parent[stage-2].mode = ce_mode_from_stat(revs->repo, nce, mode); dpath->parent[stage-2].status = DIFF_STATUS_MODIFIED; } @@ -262,7 +262,7 @@ void run_diff_files(struct rev_info *revs, unsigned int option) continue; } else if (revs->diffopt.ita_invisible_in_index && ce_intent_to_add(ce)) { - newmode = ce_mode_from_stat(ce, st.st_mode); + newmode = ce_mode_from_stat(revs->repo, ce, st.st_mode); diff_addremove(&revs->diffopt, '+', newmode, null_oid(the_hash_algo), 0, ce->name, 0); continue; @@ -270,7 +270,7 @@ void run_diff_files(struct rev_info *revs, unsigned int option) changed = match_stat_with_submodule(&revs->diffopt, ce, &st, ce_option, &dirty_submodule); - newmode = ce_mode_from_stat(ce, st.st_mode); + newmode = ce_mode_from_stat(revs->repo, ce, st.st_mode); } if (!changed && !dirty_submodule) { @@ -338,7 +338,7 @@ static int get_stat_data(const struct cache_entry *ce, changed = match_stat_with_submodule(diffopt, ce, &st, 0, dirty_submodule); if (changed) { - mode = ce_mode_from_stat(ce, st.st_mode); + mode = ce_mode_from_stat(diffopt->repo, ce, st.st_mode); oid = null_oid(the_hash_algo); } } diff --git a/dir.c b/dir.c index 1094d7be3c16ec..95d8a1cce90f77 100644 --- a/dir.c +++ b/dir.c @@ -3481,11 +3481,11 @@ static GIT_PATH_FUNC(git_path_info_exclude, "info/exclude") void setup_standard_excludes(struct dir_struct *dir) { + const char *excludes_file = repo_excludes_file(the_repository); + dir->exclude_per_dir = ".gitignore"; /* core.excludesfile defaulting to $XDG_CONFIG_HOME/git/ignore */ - if (!excludes_file) - excludes_file = xdg_config_home("ignore"); if (excludes_file && !access_or_warn(excludes_file, R_OK, 0)) add_patterns_from_file_1(dir, excludes_file, dir->untracked ? &dir->internal.ss_excludes_file : NULL); diff --git a/editor.c b/editor.c index fd174e6a034f1c..0d1cb8768db1a0 100644 --- a/editor.c +++ b/editor.c @@ -29,8 +29,8 @@ const char *git_editor(void) const char *editor = getenv("GIT_EDITOR"); int terminal_is_dumb = is_terminal_dumb(); - if (!editor && editor_program) - editor = editor_program; + if (!editor) + editor = repo_config_values(the_repository)->editor_program; if (!editor && !terminal_is_dumb) editor = getenv("VISUAL"); if (!editor) diff --git a/entry.c b/entry.c index 6b79884e3246d6..1c4f0f44070ea3 100644 --- a/entry.c +++ b/entry.c @@ -319,7 +319,8 @@ static int write_entry(struct cache_entry *ce, char *path, struct conv_attrs *ca * We can't make a real symlink; write out a regular file entry * with the symlink destination as its contents. */ - if (!has_symlinks || to_tempfile) + if (!repo_has_symlinks(state->istate && state->istate->repo ? + state->istate->repo : the_repository) || to_tempfile) goto write_file_entry; ret = symlink(new_blob, path); diff --git a/environment.c b/environment.c index c663113e8a6dcc..76ee65e62b4823 100644 --- a/environment.c +++ b/environment.c @@ -41,31 +41,21 @@ static int pack_compression_seen; static int zlib_compression_seen; -int trust_executable_bit = 1; -int has_symlinks = 1; int minimum_abbrev = 4, default_abbrev = -1; int assume_unchanged; char *git_commit_encoding; char *git_log_output_encoding; -char *apply_default_whitespace; -char *apply_default_ignorewhitespace; int fsync_object_files = -1; int use_fsync = -1; enum fsync_method fsync_method = FSYNC_METHOD_DEFAULT; enum fsync_component fsync_components = FSYNC_COMPONENTS_DEFAULT; -char *editor_program; -char *askpass_program; -char *excludes_file; enum auto_crlf auto_crlf = AUTO_CRLF_FALSE; enum eol core_eol = EOL_UNSET; int global_conv_flags_eol = CONV_EOL_RNDTRP_WARN; char *check_roundtrip_encoding; -enum rebase_setup_type autorebase = AUTOREBASE_NEVER; -enum push_default_type push_default = PUSH_DEFAULT_UNSPECIFIED; #ifndef OBJECT_CREATION_MODE #define OBJECT_CREATION_MODE OBJECT_CREATION_USES_HARDLINKS #endif -enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE; int grafts_keep_true_parents; unsigned long pack_size_limit_cfg; @@ -148,6 +138,30 @@ int repo_ignore_case(struct repository *repo) 0; } +int repo_trust_executable_bit(struct repository *repo) +{ + return repo->initialized + ? repo_config_values(repo)->trust_executable_bit + : 1; +} + +int repo_has_symlinks(struct repository *repo) +{ + return repo->initialized + ? repo_config_values(repo)->has_symlinks + : platform_has_symlinks(); +} + +const char *repo_excludes_file(struct repository *repo) +{ + struct repo_config_values *cfg = repo_config_values(repo); + + if (!cfg->excludes_file) + cfg->excludes_file = xdg_config_home("ignore"); + + return cfg->excludes_file; +} + int have_git_dir(void) { return startup_info->have_repository @@ -311,7 +325,7 @@ int git_default_core_config(const char *var, const char *value, /* This needs a better name */ if (!strcmp(var, "core.filemode")) { - trust_executable_bit = git_config_bool(var, value); + cfg->trust_executable_bit = git_config_bool(var, value); return 0; } if (!strcmp(var, "core.trustctime")) { @@ -336,7 +350,8 @@ int git_default_core_config(const char *var, const char *value, } if (!strcmp(var, "core.symlinks")) { - has_symlinks = git_config_bool(var, value); + struct repo_config_values *cfg = repo_config_values(the_repository); + cfg->has_symlinks = git_config_bool(var, value); return 0; } @@ -442,8 +457,8 @@ int git_default_core_config(const char *var, const char *value, } if (!strcmp(var, "core.editor")) { - FREE_AND_NULL(editor_program); - return git_config_string(&editor_program, var, value); + FREE_AND_NULL(cfg->editor_program); + return git_config_string(&cfg->editor_program, var, value); } if (!strcmp(var, "core.commentchar") || @@ -470,13 +485,13 @@ int git_default_core_config(const char *var, const char *value, } if (!strcmp(var, "core.askpass")) { - FREE_AND_NULL(askpass_program); - return git_config_string(&askpass_program, var, value); + FREE_AND_NULL(cfg->askpass_program); + return git_config_string(&cfg->askpass_program, var, value); } if (!strcmp(var, "core.excludesfile")) { - FREE_AND_NULL(excludes_file); - return git_config_pathname(&excludes_file, var, value); + FREE_AND_NULL(cfg->excludes_file); + return git_config_pathname(&cfg->excludes_file, var, value); } if (!strcmp(var, "core.whitespace")) { @@ -523,9 +538,9 @@ int git_default_core_config(const char *var, const char *value, if (!value) return config_error_nonbool(var); if (!strcmp(value, "rename")) - object_creation_mode = OBJECT_CREATION_USES_RENAMES; + cfg->object_creation_mode = OBJECT_CREATION_USES_RENAMES; else if (!strcmp(value, "link")) - object_creation_mode = OBJECT_CREATION_USES_HARDLINKS; + cfg->object_creation_mode = OBJECT_CREATION_USES_HARDLINKS; else die(_("invalid mode for object creation: %s"), value); return 0; @@ -611,13 +626,13 @@ static int git_default_branch_config(const char *var, const char *value) if (!value) return config_error_nonbool(var); else if (!strcmp(value, "never")) - autorebase = AUTOREBASE_NEVER; + cfg->autorebase = AUTOREBASE_NEVER; else if (!strcmp(value, "local")) - autorebase = AUTOREBASE_LOCAL; + cfg->autorebase = AUTOREBASE_LOCAL; else if (!strcmp(value, "remote")) - autorebase = AUTOREBASE_REMOTE; + cfg->autorebase = AUTOREBASE_REMOTE; else if (!strcmp(value, "always")) - autorebase = AUTOREBASE_ALWAYS; + cfg->autorebase = AUTOREBASE_ALWAYS; else return error(_("malformed value for %s"), var); return 0; @@ -629,21 +644,23 @@ static int git_default_branch_config(const char *var, const char *value) static int git_default_push_config(const char *var, const char *value) { + struct repo_config_values *cfg = repo_config_values(the_repository); + if (!strcmp(var, "push.default")) { if (!value) return config_error_nonbool(var); else if (!strcmp(value, "nothing")) - push_default = PUSH_DEFAULT_NOTHING; + cfg->push_default = PUSH_DEFAULT_NOTHING; else if (!strcmp(value, "matching")) - push_default = PUSH_DEFAULT_MATCHING; + cfg->push_default = PUSH_DEFAULT_MATCHING; else if (!strcmp(value, "simple")) - push_default = PUSH_DEFAULT_SIMPLE; + cfg->push_default = PUSH_DEFAULT_SIMPLE; else if (!strcmp(value, "upstream")) - push_default = PUSH_DEFAULT_UPSTREAM; + cfg->push_default = PUSH_DEFAULT_UPSTREAM; else if (!strcmp(value, "tracking")) /* deprecated */ - push_default = PUSH_DEFAULT_UPSTREAM; + cfg->push_default = PUSH_DEFAULT_UPSTREAM; else if (!strcmp(value, "current")) - push_default = PUSH_DEFAULT_CURRENT; + cfg->push_default = PUSH_DEFAULT_CURRENT; else { error(_("malformed value for %s: %s"), var, value); return error(_("must be one of nothing, matching, simple, " @@ -729,10 +746,21 @@ int git_default_config(const char *var, const char *value, void repo_config_values_init(struct repo_config_values *cfg) { cfg->attributes_file = NULL; + cfg->excludes_file = NULL; + cfg->editor_program = NULL; + cfg->pager_program = NULL; + cfg->askpass_program = NULL; + cfg->apply_default_whitespace = NULL; + cfg->apply_default_ignorewhitespace = NULL; + cfg->push_default = PUSH_DEFAULT_UNSPECIFIED; + cfg->autorebase = AUTOREBASE_NEVER; + cfg->object_creation_mode = OBJECT_CREATION_MODE; cfg->apply_sparse_checkout = 0; cfg->protect_hfs = PROTECT_HFS_DEFAULT; cfg->protect_ntfs = PROTECT_NTFS_DEFAULT; cfg->ignore_case = 0; + cfg->trust_executable_bit = 1; + cfg->has_symlinks = platform_has_symlinks(); cfg->branch_track = BRANCH_TRACK_REMOTE; cfg->trust_ctime = 1; cfg->check_stat = 1; @@ -743,3 +771,14 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->sparse_expect_files_outside_of_patterns = 0; cfg->warn_on_object_refname_ambiguity = 1; } + +void repo_config_values_clear(struct repo_config_values *cfg) +{ + FREE_AND_NULL(cfg->attributes_file); + FREE_AND_NULL(cfg->excludes_file); + FREE_AND_NULL(cfg->editor_program); + FREE_AND_NULL(cfg->pager_program); + FREE_AND_NULL(cfg->askpass_program); + FREE_AND_NULL(cfg->apply_default_whitespace); + FREE_AND_NULL(cfg->apply_default_ignorewhitespace); +} diff --git a/environment.h b/environment.h index acfb670be107a0..e7ec5b0437342d 100644 --- a/environment.h +++ b/environment.h @@ -87,9 +87,45 @@ extern const char * const local_repo_env[]; struct strvec; struct repository; + +/* + * NEEDSWORK: It would be better if these definitions could be moved to + * other more specific files, but care is needed to avoid circular + * inclusion issues. + */ +enum push_default_type { + PUSH_DEFAULT_NOTHING = 0, + PUSH_DEFAULT_MATCHING, + PUSH_DEFAULT_SIMPLE, + PUSH_DEFAULT_UPSTREAM, + PUSH_DEFAULT_CURRENT, + PUSH_DEFAULT_UNSPECIFIED +}; + +enum rebase_setup_type { + AUTOREBASE_NEVER = 0, + AUTOREBASE_LOCAL, + AUTOREBASE_REMOTE, + AUTOREBASE_ALWAYS +}; + +enum object_creation_mode { + OBJECT_CREATION_USES_HARDLINKS = 0, + OBJECT_CREATION_USES_RENAMES = 1 +}; + struct repo_config_values { /* section "core" config values */ char *attributes_file; + char *excludes_file; + char *editor_program; + char *pager_program; + char *askpass_program; + char *apply_default_whitespace; + char *apply_default_ignorewhitespace; + enum push_default_type push_default; + enum rebase_setup_type autorebase; + enum object_creation_mode object_creation_mode; int apply_sparse_checkout; int trust_ctime; int check_stat; @@ -101,6 +137,8 @@ struct repo_config_values { int protect_hfs; int protect_ntfs; int ignore_case; + int trust_executable_bit; + int has_symlinks; /* section "sparse" config values */ int sparse_expect_files_outside_of_patterns; @@ -151,10 +189,25 @@ int repo_protect_ntfs(struct repository *repo); */ int repo_ignore_case(struct repository *repo); +int repo_trust_executable_bit(struct repository *repo); + +int repo_has_symlinks(struct repository *repo); + +const char *repo_excludes_file(struct repository *repo); + void repo_config_values_init(struct repo_config_values *cfg); int is_bare_repository(struct repository *repo); +/* + * Frees memory allocated for dynamically loaded configuration values + * inside `repo_config_values`. + * + * As dynamically allocated variables are migrated into this struct, + * their FREE_AND_NULL() calls should be appended here. + */ +void repo_config_values_clear(struct repo_config_values *cfg); + /* * TODO: All the below state either explicitly or implicitly relies on * `the_repository`. We should eventually get rid of these and make the @@ -178,38 +231,10 @@ int is_bare_repository(struct repository *repo); int have_git_dir(void); /* Environment bits from configuration mechanism */ -extern int trust_executable_bit; -extern int has_symlinks; extern int minimum_abbrev, default_abbrev; extern int assume_unchanged; -extern char *apply_default_whitespace; -extern char *apply_default_ignorewhitespace; extern unsigned long pack_size_limit_cfg; -enum rebase_setup_type { - AUTOREBASE_NEVER = 0, - AUTOREBASE_LOCAL, - AUTOREBASE_REMOTE, - AUTOREBASE_ALWAYS -}; -extern enum rebase_setup_type autorebase; - -enum push_default_type { - PUSH_DEFAULT_NOTHING = 0, - PUSH_DEFAULT_MATCHING, - PUSH_DEFAULT_SIMPLE, - PUSH_DEFAULT_UPSTREAM, - PUSH_DEFAULT_CURRENT, - PUSH_DEFAULT_UNSPECIFIED -}; -extern enum push_default_type push_default; - -enum object_creation_mode { - OBJECT_CREATION_USES_HARDLINKS = 0, - OBJECT_CREATION_USES_RENAMES = 1 -}; -extern enum object_creation_mode object_creation_mode; - extern int grafts_keep_true_parents; const char *get_log_output_encoding(void); @@ -218,10 +243,6 @@ const char *get_commit_output_encoding(void); extern char *git_commit_encoding; extern char *git_log_output_encoding; -extern char *editor_program; -extern char *askpass_program; -extern char *excludes_file; - /* * The character that begins a commented line in user-editable file * that is subject to stripspace. diff --git a/fetch-object-info.c b/fetch-object-info.c new file mode 100644 index 00000000000000..ba7e179c44ee54 --- /dev/null +++ b/fetch-object-info.c @@ -0,0 +1,156 @@ +#include "git-compat-util.h" +#include "gettext.h" +#include "hex.h" +#include "pkt-line.h" +#include "connect.h" +#include "oid-array.h" +#include "odb.h" +#include "fetch-object-info.h" +#include "string-list.h" + +/* Sends object-info command and its arguments into the request buffer. */ +static void send_object_info_request(const int fd_out, struct object_info_args *args) +{ + struct strbuf req_buf = STRBUF_INIT; + + write_command_and_capabilities(&req_buf, "object-info", args->server_options); + + if (unsorted_string_list_has_string(args->object_info_options, "size")) + packet_buf_write(&req_buf, "size"); + else if (args->object_info_options->nr) + BUG("only size should be in object_info_options"); + + if (args->oids) + for (size_t i = 0; i < args->oids->nr; i++) + packet_buf_write(&req_buf, "oid %s", oid_to_hex(&args->oids->oid[i])); + + packet_buf_flush(&req_buf); + if (write_in_full(fd_out, req_buf.buf, req_buf.len) < 0) + die_errno(_("unable to write request to remote")); + + strbuf_release(&req_buf); +} + +static int parse_object_size(const char *s, size_t *res) +{ + uintmax_t uim; + + if (!s[0] || s[strspn(s, "0123456789")]) + return -1; + errno = 0; + uim = strtoumax(s, NULL, 10); + if (errno || uim > SIZE_MAX) + return -1; + *res = uim; + return 0; +} + +int fetch_object_info(const enum protocol_version version, struct object_info_args *args, + struct packet_reader *reader, struct object_info *object_info_data, + const int stateless_rpc, const int fd_out) +{ + int size_index = -1; + + switch (version) { + case protocol_v2: + if (!server_supports_v2("object-info")) + die(_("object-info capability is not enabled on the server")); + /* + * When removing an element from the list it gets swapped by the + * last element, iterate backwards to prevent elements skipping + * evaluation. + * + * object_info_options->nr can be safely casted without overflow + * because the number of options is a small known number (the + * supported placeholders which currently are size and type). + */ + for (int i = (int)args->object_info_options->nr - 1; i >= 0; i--) + if (!server_supports_feature("object-info", + args->object_info_options->items[i].string, 0)) + unsorted_string_list_delete_item(args->object_info_options, i, 0); + + /* + * Even if no options are left, we still send the oid so we get + * at least an existence check. + */ + send_object_info_request(fd_out, args); + break; + case protocol_v1: + case protocol_v0: + die(_("object-info requires protocol v2")); + case protocol_unknown_version: + BUG("unknown protocol version"); + } + + for (size_t i = 0; i < args->object_info_options->nr; i++) { + if (packet_reader_read(reader) != PACKET_READ_NORMAL) { + check_stateless_delimiter(stateless_rpc, reader, + "stateless delimiter expected"); + return -1; + } + + if (!unsorted_string_list_has_string(args->object_info_options, reader->line)) + return -1; + + if (!strcmp(reader->line, "size")) { + /* + * i is the number of supported options which currently + * is only size. No risk of overflow. + */ + size_index = (int)i; + for (size_t j = 0; j < args->oids->nr; j++) + object_info_data[j].sizep = + xcalloc(1, sizeof(*object_info_data[j].sizep)); + } else { + BUG("only size is supported"); + } + } + + for (size_t i = 0; + packet_reader_read(reader) == PACKET_READ_NORMAL && + i < args->oids->nr; + i++) { + struct string_list object_info_values = STRING_LIST_INIT_DUP; + + string_list_split(&object_info_values, reader->line, " ", -1); + + if (strcmp(object_info_values.items[0].string, + oid_to_hex(&args->oids->oid[i]))) + die(_("object-info: expected OID: %s, got %s"), + oid_to_hex(&args->oids->oid[i]), + object_info_values.items[0].string); + + /* + * If the response is two elements but the second one is an + * empty string, that means that the OID is unrecognized by the + * server. + */ + if (object_info_values.nr >= 2 && + !strcmp(object_info_values.items[1].string, "")) { + object_info_data[i].unrecognized = 1; + string_list_clear(&object_info_values, 0); + continue; + } + + /* + * Because we filter the options to be only the supported by + * the server we expect the server to answer with the same + * number of attributes requested. + */ + if (args->object_info_options->nr + 1 != object_info_values.nr) + die("object-info: unexpected number of attributes: %s", + reader->line); + + if (size_index >= 0 && + parse_object_size(object_info_values.items[size_index + 1].string, + object_info_data[i].sizep)) + die("object-info: ref %s has invalid size %s", + object_info_values.items[0].string, + object_info_values.items[size_index + 1].string); + + string_list_clear(&object_info_values, 0); + } + check_stateless_delimiter(stateless_rpc, reader, "stateless delimiter expected"); + + return 0; +} diff --git a/fetch-object-info.h b/fetch-object-info.h new file mode 100644 index 00000000000000..269cebb3f7df48 --- /dev/null +++ b/fetch-object-info.h @@ -0,0 +1,25 @@ +#ifndef FETCH_OBJECT_INFO_H +#define FETCH_OBJECT_INFO_H + +#include "pkt-line.h" +#include "protocol.h" + +struct object_info_args { + struct string_list *object_info_options; + const struct string_list *server_options; + struct oid_array *oids; +}; + +struct object_info; +/* + * Sends git-cat-file object-info command into the request buf and read the + * results from packets. + * + * Modifies args->object_info_options, on return it contains only the supported + * options by the server. + */ +int fetch_object_info(enum protocol_version version, struct object_info_args *args, + struct packet_reader *reader, struct object_info *object_info_data, + int stateless_rpc, int fd_out); + +#endif /* FETCH_OBJECT_INFO_H */ diff --git a/fetch-pack.c b/fetch-pack.c index 29c41132ee0495..922a9b25812c68 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -49,7 +49,6 @@ static int fetch_fsck_objects = -1; static int transfer_fsck_objects = -1; static int agent_supported; static int server_supports_filtering; -static int advertise_sid; static struct shallow_lock shallow_lock; static const char *alternate_shallow_file; static struct strbuf fsck_msg_types = STRBUF_INIT; @@ -363,6 +362,9 @@ static int find_common(struct fetch_negotiator *negotiator, size_t state_len = 0; struct packet_reader reader; struct oidset negotiation_include_oids = OIDSET_INIT; + int advertise_sid = 0; + + repo_config_get_bool(the_repository, "transfer.advertisesid", &advertise_sid); if (args->stateless_rpc && multi_ack == 1) die(_("the option '%s' requires '%s'"), "--stateless-rpc", "multi_ack_detailed"); @@ -414,7 +416,7 @@ static int find_common(struct fetch_negotiator *negotiator, if (deepen_not_ok) strbuf_addstr(&c, " deepen-not"); if (agent_supported) strbuf_addf(&c, " agent=%s", git_user_agent_sanitized()); - if (advertise_sid) + if (advertise_sid && server_supports("session-id")) strbuf_addf(&c, " session-id=%s", trace2_session_id()); if (args->filter_options.choice) strbuf_addstr(&c, " filter"); @@ -1160,9 +1162,6 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args, (int)agent_len, agent_feature); } - if (!server_supports("session-id")) - advertise_sid = 0; - if (server_supports("shallow")) print_verbose(args, _("Server supports %s"), "shallow"); else if (args->depth > 0 || is_repository_shallow(r)) @@ -1376,38 +1375,6 @@ static int add_haves(struct fetch_negotiator *negotiator, return haves_added; } -static void write_fetch_command_and_capabilities(struct strbuf *req_buf, - const struct string_list *server_options) -{ - const char *hash_name; - - ensure_server_supports_v2("fetch"); - packet_buf_write(req_buf, "command=fetch"); - if (server_supports_v2("agent")) - packet_buf_write(req_buf, "agent=%s", git_user_agent_sanitized()); - if (advertise_sid && server_supports_v2("session-id")) - packet_buf_write(req_buf, "session-id=%s", trace2_session_id()); - if (server_options && server_options->nr) { - int i; - ensure_server_supports_v2("server-option"); - for (i = 0; i < server_options->nr; i++) - packet_buf_write(req_buf, "server-option=%s", - server_options->items[i].string); - } - - if (server_feature_v2("object-format", &hash_name)) { - int hash_algo = hash_algo_by_name(hash_name); - if (hash_algo_by_ptr(the_hash_algo) != hash_algo) - die(_("mismatched algorithms: client %s; server %s"), - the_hash_algo->name, hash_name); - packet_buf_write(req_buf, "object-format=%s", the_hash_algo->name); - } else if (hash_algo_by_ptr(the_hash_algo) != GIT_HASH_SHA1_LEGACY) { - die(_("the server does not support algorithm '%s'"), - the_hash_algo->name); - } - packet_buf_delim(req_buf); -} - static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, struct fetch_pack_args *args, const struct ref *wants, struct oidset *common, @@ -1419,7 +1386,7 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, int done_sent = 0; struct strbuf req_buf = STRBUF_INIT; - write_fetch_command_and_capabilities(&req_buf, args->server_options); + write_command_and_capabilities(&req_buf, "fetch", args->server_options); if (args->use_thin_pack) packet_buf_write(&req_buf, "thin-pack"); @@ -1768,18 +1735,18 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, reader.me = "fetch-pack"; } + /* v2 supports these by default */ + allow_unadvertised_object_request |= ALLOW_REACHABLE_SHA1; + use_sideband = 2; + if (args->depth > 0 || args->deepen_since || args->deepen_not) + args->deepen = 1; + while (state != FETCH_DONE) { switch (state) { case FETCH_CHECK_LOCAL: sort_ref_list(&ref, ref_compare_name); QSORT(sought, nr_sought, cmp_ref_by_name); - /* v2 supports these by default */ - allow_unadvertised_object_request |= ALLOW_REACHABLE_SHA1; - use_sideband = 2; - if (args->depth > 0 || args->deepen_since || args->deepen_not) - args->deepen = 1; - /* Filter 'ref' by 'sought' and those that aren't local */ mark_complete_and_common_ref(negotiator, args, &ref); filter_refs(args, &ref, sought, nr_sought); @@ -1999,7 +1966,6 @@ static void fetch_pack_config(void) repo_config_get_bool(the_repository, "repack.usedeltabaseoffset", &prefer_ofs_delta); repo_config_get_bool(the_repository, "fetch.fsckobjects", &fetch_fsck_objects); repo_config_get_bool(the_repository, "transfer.fsckobjects", &transfer_fsck_objects); - repo_config_get_bool(the_repository, "transfer.advertisesid", &advertise_sid); if (!uri_protocols.nr) { char *str; @@ -2287,7 +2253,7 @@ void negotiate_using_fetch(const struct oid_array *negotiation_restrict_tips, the_repository, "%d", negotiation_round); strbuf_reset(&req_buf); - write_fetch_command_and_capabilities(&req_buf, server_options); + write_command_and_capabilities(&req_buf, "fetch", server_options); packet_buf_write(&req_buf, "wait-for-done"); diff --git a/git-compat-util.h b/git-compat-util.h index 88097764078538..a0f901ce793ed6 100644 --- a/git-compat-util.h +++ b/git-compat-util.h @@ -245,6 +245,10 @@ static inline int git_is_dir_sep(int c) #define is_dir_sep git_is_dir_sep #endif +#ifndef platform_has_symlinks +#define platform_has_symlinks() 1 +#endif + #ifndef offset_1st_component static inline int git_offset_1st_component(const char *path) { diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl index fde804593b6ff3..8c2d9b8eefe4db 100755 --- a/gitweb/gitweb.perl +++ b/gitweb/gitweb.perl @@ -2339,12 +2339,14 @@ sub format_extended_diff_header_line { $line .= $cgi->a({-href=>$to->{'href'}, -class=>"path"}, esc_path($to->{'file'})); } - # match single - if ($line =~ m/\s(\d{6})$/) { - $line .= ' (' . - file_type_long($1) . - ')'; + + # Temporarily remove a trailing so an index line ends with its + # object IDs and can be shortened below. + my $mode; + if ($line =~ s/\s(\d{6})$//) { + $mode = $1; } + # match if ($line =~ oid_nlen_prefix_infix_regex($sha1_len, "index ", ",") | $line =~ oid_nlen_prefix_infix_regex($sha256_len, "index ", ",")) { @@ -2388,6 +2390,12 @@ sub format_extended_diff_header_line { my ($from_id, $to_id) = ($diffinfo->{'from_id'}, $diffinfo->{'to_id'}); $line =~ s!$from_id\.\.$to_id!$from_link..$to_link!; } + if (defined $mode) { + $line .= " $mode" . + ' (' . + file_type_long($mode) . + ')'; + } return $line . "
\n"; } diff --git a/lockfile.c b/lockfile.c index 7add2f136abd85..100f60377174ef 100644 --- a/lockfile.c +++ b/lockfile.c @@ -2,11 +2,14 @@ * Copyright (c) 2005, Junio C Hamano */ +#define USE_THE_REPOSITORY_VARIABLE + #include "git-compat-util.h" #include "abspath.h" #include "gettext.h" #include "lockfile.h" #include "parse.h" +#include "repository.h" #include "strbuf.h" #include "wrapper.h" @@ -162,8 +165,8 @@ static int read_lock_pid(const char *pid_path, uintmax_t *pid_out) } /* Make sure errno contains a meaningful value on error */ -static int lock_file(struct lock_file *lk, const char *path, int flags, - int mode) +static int lock_file(struct repository *r, struct lock_file *lk, + const char *path, int flags, int mode) { struct strbuf base_path = STRBUF_INIT; struct strbuf lock_path = STRBUF_INIT; @@ -176,7 +179,7 @@ static int lock_file(struct lock_file *lk, const char *path, int flags, get_lock_path(&lock_path, base_path.buf); get_pid_path(&pid_path, base_path.buf); - lk->tempfile = create_tempfile_mode(lock_path.buf, mode); + lk->tempfile = repo_create_tempfile_mode(r, lock_path.buf, mode); if (lk->tempfile) lk->pid_tempfile = create_lock_pid_file(pid_path.buf, mode); @@ -200,8 +203,9 @@ static int lock_file(struct lock_file *lk, const char *path, int flags, * timeout_ms milliseconds. If timeout_ms is 0, try locking the file * exactly once. If timeout_ms is -1, try indefinitely. */ -static int lock_file_timeout(struct lock_file *lk, const char *path, - int flags, long timeout_ms, int mode) +static int lock_file_timeout(struct repository *r, struct lock_file *lk, + const char *path, int flags, long timeout_ms, + int mode) { int n = 1; int multiplier = 1; @@ -209,7 +213,7 @@ static int lock_file_timeout(struct lock_file *lk, const char *path, static int random_initialized = 0; if (timeout_ms == 0) - return lock_file(lk, path, flags, mode); + return lock_file(r, lk, path, flags, mode); if (!random_initialized) { srand((unsigned int)getpid()); @@ -223,7 +227,7 @@ static int lock_file_timeout(struct lock_file *lk, const char *path, long backoff_ms, wait_ms; int fd; - fd = lock_file(lk, path, flags, mode); + fd = lock_file(r, lk, path, flags, mode); if (fd >= 0) return fd; /* success */ @@ -308,7 +312,17 @@ int hold_lock_file_for_update_timeout_mode(struct lock_file *lk, const char *path, int flags, long timeout_ms, int mode) { - int fd = lock_file_timeout(lk, path, flags, timeout_ms, mode); + return repo_hold_lock_file_for_update_timeout_mode(the_repository, + lk, path, flags, + timeout_ms, mode); +} + +int repo_hold_lock_file_for_update_timeout_mode(struct repository *r, + struct lock_file *lk, + const char *path, int flags, + long timeout_ms, int mode) +{ + int fd = lock_file_timeout(r, lk, path, flags, timeout_ms, mode); if (fd < 0) { if (flags & LOCK_DIE_ON_ERROR) unable_to_lock_die(path, errno); diff --git a/lockfile.h b/lockfile.h index e7233f28dea5c7..1667612674b52a 100644 --- a/lockfile.h +++ b/lockfile.h @@ -189,6 +189,11 @@ int hold_lock_file_for_update_timeout_mode( struct lock_file *lk, const char *path, int flags, long timeout_ms, int mode); +int repo_hold_lock_file_for_update_timeout_mode(struct repository *r, + struct lock_file *lk, + const char *path, int flags, + long timeout_ms, int mode); + static inline int hold_lock_file_for_update_timeout( struct lock_file *lk, const char *path, int flags, long timeout_ms) @@ -197,6 +202,16 @@ static inline int hold_lock_file_for_update_timeout( timeout_ms, 0666); } +static inline int repo_hold_lock_file_for_update_timeout(struct repository *r, + struct lock_file *lk, + const char *path, + int flags, + long timeout_ms) +{ + return repo_hold_lock_file_for_update_timeout_mode(r, lk, path, flags, + timeout_ms, 0666); +} + /* * Attempt to create a lockfile for the file at `path` and return a * file descriptor for writing to it, or -1 on error. The flags @@ -208,6 +223,13 @@ static inline int hold_lock_file_for_update( return hold_lock_file_for_update_timeout(lk, path, flags, 0); } +static inline int repo_hold_lock_file_for_update(struct repository *r, + struct lock_file *lk, + const char *path, int flags) +{ + return repo_hold_lock_file_for_update_timeout(r, lk, path, flags, 0); +} + static inline int hold_lock_file_for_update_mode( struct lock_file *lk, const char *path, int flags, int mode) @@ -215,6 +237,15 @@ static inline int hold_lock_file_for_update_mode( return hold_lock_file_for_update_timeout_mode(lk, path, flags, 0, mode); } +static inline int repo_hold_lock_file_for_update_mode(struct repository *r, + struct lock_file *lk, + const char *path, + int flags, int mode) +{ + return repo_hold_lock_file_for_update_timeout_mode(r, lk, path, flags, + 0, mode); +} + /* * Return a nonzero value iff `lk` is currently locked. */ diff --git a/loose.c b/loose.c index bf01d3e42def34..e97c3bb50af867 100644 --- a/loose.c +++ b/loose.c @@ -137,7 +137,8 @@ int repo_write_loose_object_map(struct repository *repo) return 0; repo_common_path_replace(repo, &path, "objects/loose-object-idx"); - fd = hold_lock_file_for_update_timeout(&lock, path.buf, LOCK_DIE_ON_ERROR, -1); + fd = repo_hold_lock_file_for_update_timeout(repo, &lock, path.buf, + LOCK_DIE_ON_ERROR, -1); iter = kh_begin(map); if (write_in_full(fd, loose_object_header, strlen(loose_object_header)) < 0) goto errout; @@ -179,7 +180,8 @@ static int write_one_object(struct odb_source_loose *loose, struct strbuf buf = STRBUF_INIT, path = STRBUF_INIT; strbuf_addf(&path, "%s/loose-object-idx", loose->base.path); - hold_lock_file_for_update_timeout(&lock, path.buf, LOCK_DIE_ON_ERROR, -1); + repo_hold_lock_file_for_update_timeout(loose->base.odb->repo, &lock, + path.buf, LOCK_DIE_ON_ERROR, -1); fd = open(path.buf, O_WRONLY | O_CREAT | O_APPEND, 0666); if (fd < 0) diff --git a/meson.build b/meson.build index f7c40ea079b797..7073d5844d2531 100644 --- a/meson.build +++ b/meson.build @@ -374,6 +374,7 @@ libgit_sources = [ 'ewah/ewah_rlw.c', 'exec-cmd.c', 'fetch-negotiator.c', + 'fetch-object-info.c', 'fetch-pack.c', 'fmt-merge-msg.c', 'fsck.c', diff --git a/midx-write.c b/midx-write.c index 8c1837f6df4671..580724d21a677f 100644 --- a/midx-write.c +++ b/midx-write.c @@ -1627,8 +1627,8 @@ static int write_midx_internal(struct write_midx_opts *opts) struct strbuf lock_name = STRBUF_INIT; get_midx_chain_filename(opts->source, &lock_name); - hold_lock_file_for_update(&lk, lock_name.buf, - LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(r, &lk, lock_name.buf, + LOCK_DIE_ON_ERROR); strbuf_release(&lock_name); } @@ -1647,7 +1647,8 @@ static int write_midx_internal(struct write_midx_opts *opts) f = hashfd(r->hash_algo, get_tempfile_fd(incr), get_tempfile_path(incr)); } else { - hold_lock_file_for_update(&lk, midx_name.buf, LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(r, &lk, midx_name.buf, + LOCK_DIE_ON_ERROR); f = hashfd(r->hash_algo, get_lock_file_fd(&lk), get_lock_file_path(&lk)); } diff --git a/object-file.c b/object-file.c index 7ff2b730ac0ead..ec35c318bc9fe7 100644 --- a/object-file.c +++ b/object-file.c @@ -67,9 +67,17 @@ const char *odb_loose_path(struct odb_source_loose *loose, } /* Returns 1 if we have successfully freshened the file, 0 otherwise. */ -static int freshen_file(const char *fn) +static int freshen_file(const char *fn, const time_t *mtime) { - return !utime(fn, NULL); + struct utimbuf times, *timesp = NULL; + + if (mtime) { + times.actime = *mtime; + times.modtime = *mtime; + timesp = × + } + + return !utime(fn, timesp); } /* @@ -79,11 +87,12 @@ static int freshen_file(const char *fn) * either does not exist on disk, or has a stale mtime and may be subject to * pruning). */ -int check_and_freshen_file(const char *fn, int freshen) +int check_and_freshen_file(const char *fn, int freshen, + const time_t *mtime) { if (access(fn, F_OK)) return 0; - if (freshen && !freshen_file(fn)) + if (freshen && !freshen_file(fn, mtime)) return 0; return 1; } @@ -313,31 +322,6 @@ int parse_loose_header(const char *hdr, struct object_info *oi) return 0; } -static void hash_object_body(const struct git_hash_algo *algo, struct git_hash_ctx *c, - const void *buf, size_t len, - struct object_id *oid, - char *hdr, size_t *hdrlen) -{ - git_hash_init(c, algo); - git_hash_update(c, hdr, *hdrlen); - git_hash_update(c, buf, len); - git_hash_final_oid(oid, c); -} - -void write_object_file_prepare(const struct git_hash_algo *algo, - const void *buf, size_t len, - enum object_type type, struct object_id *oid, - char *hdr, size_t *hdrlen) -{ - struct git_hash_ctx c; - - /* Generate the header */ - *hdrlen = format_object_header(hdr, *hdrlen, type, len); - - /* Hash (function pointers) computation */ - hash_object_body(algo, &c, buf, len, oid, hdr, hdrlen); -} - #define CHECK_COLLISION_DEST_VANISHED -2 static int check_collision(const char *source, const char *dest) @@ -409,11 +393,12 @@ int finalize_object_file_flags(struct repository *repo, { unsigned retries = 0; int ret; + struct repo_config_values *cfg = repo_config_values(repo); retry: ret = 0; - if (object_creation_mode == OBJECT_CREATION_USES_RENAMES) + if (cfg->object_creation_mode == OBJECT_CREATION_USES_RENAMES) goto try_rename; else if (link(tmpfile, filename)) ret = errno; @@ -473,10 +458,16 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf, size_t len, enum object_type type, struct object_id *oid) { + struct git_hash_ctx c; char hdr[MAX_HEADER_LEN]; - size_t hdrlen = sizeof(hdr); + int hdrlen; + + hdrlen = format_object_header(hdr, sizeof(hdr), type, len); - write_object_file_prepare(algo, buf, len, type, oid, hdr, &hdrlen); + git_hash_init(&c, algo); + git_hash_update(&c, hdr, hdrlen); + git_hash_update(&c, buf, len); + git_hash_final_oid(oid, &c); } struct transaction_packfile { @@ -498,7 +489,7 @@ struct odb_transaction_files { const char *prefix; }; -static int odb_transaction_files_prepare(struct odb_transaction *base) +int odb_transaction_files_prepare(struct odb_transaction *base) { struct odb_transaction_files *transaction = container_of_or_null(base, struct odb_transaction_files, base); @@ -521,8 +512,8 @@ static int odb_transaction_files_prepare(struct odb_transaction *base) return 0; } -static void odb_transaction_files_fsync(struct odb_transaction *base, - int fd, const char *filename) +void odb_transaction_files_fsync(struct odb_transaction *base, + int fd, const char *filename) { struct odb_transaction_files *transaction = container_of_or_null(base, struct odb_transaction_files, base); @@ -546,397 +537,6 @@ static void odb_transaction_files_fsync(struct odb_transaction *base, } } -/* Finalize a file on disk, and close it. */ -static void close_loose_object(struct odb_source_loose *loose, - int fd, const char *filename) -{ - if (loose->base.will_destroy) - goto out; - - if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) - odb_transaction_files_fsync(loose->base.odb->transaction, fd, filename); - else if (fsync_object_files > 0) - fsync_or_die(fd, filename); - else - fsync_component_or_die(FSYNC_COMPONENT_LOOSE_OBJECT, fd, - filename); - -out: - if (close(fd) != 0) - die_errno(_("error when closing loose object file")); -} - -/* Size of directory component, including the ending '/' */ -static inline int directory_size(const char *filename) -{ - const char *s = strrchr(filename, '/'); - if (!s) - return 0; - return s - filename + 1; -} - -/* - * This creates a temporary file in the same directory as the final - * 'filename' - * - * We want to avoid cross-directory filename renames, because those - * can have problems on various filesystems (FAT, NFS, Coda). - */ -static int create_tmpfile(struct repository *repo, - struct strbuf *tmp, const char *filename) -{ - int fd, dirlen = directory_size(filename); - - strbuf_reset(tmp); - strbuf_add(tmp, filename, dirlen); - strbuf_addstr(tmp, "tmp_obj_XXXXXX"); - fd = git_mkstemp_mode(tmp->buf, 0444); - if (fd < 0 && dirlen && errno == ENOENT) { - /* - * Make sure the directory exists; note that the contents - * of the buffer are undefined after mkstemp returns an - * error, so we have to rewrite the whole buffer from - * scratch. - */ - strbuf_reset(tmp); - strbuf_add(tmp, filename, dirlen - 1); - if (mkdir(tmp->buf, 0777) && errno != EEXIST) - return -1; - if (adjust_shared_perm(repo, tmp->buf)) - return -1; - - /* Try again */ - strbuf_addstr(tmp, "/tmp_obj_XXXXXX"); - fd = git_mkstemp_mode(tmp->buf, 0444); - } - return fd; -} - -/** - * Common steps for loose object writers to start writing loose - * objects: - * - * - Create tmpfile for the loose object. - * - Setup zlib stream for compression. - * - Start to feed header to zlib stream. - * - * Returns a "fd", which should later be provided to - * end_loose_object_common(). - */ -static int start_loose_object_common(struct odb_source_loose *loose, - struct strbuf *tmp_file, - const char *filename, unsigned flags, - git_zstream *stream, - unsigned char *buf, size_t buflen, - struct git_hash_ctx *c, struct git_hash_ctx *compat_c, - char *hdr, int hdrlen) -{ - const struct git_hash_algo *algo = loose->base.odb->repo->hash_algo; - const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; - int fd; - struct repo_config_values *cfg = repo_config_values(the_repository); - - fd = create_tmpfile(loose->base.odb->repo, tmp_file, filename); - if (fd < 0) { - if (flags & ODB_WRITE_OBJECT_SILENT) - return -1; - else if (errno == EACCES) - return error(_("insufficient permission for adding " - "an object to repository database %s"), - loose->base.path); - else - return error_errno( - _("unable to create temporary file")); - } - - /* Setup zlib stream for compression */ - git_deflate_init(stream, cfg->zlib_compression_level); - stream->next_out = buf; - stream->avail_out = buflen; - git_hash_init(c, algo); - if (compat && compat_c) - git_hash_init(compat_c, compat); - - /* Start to feed header to zlib stream */ - stream->next_in = (unsigned char *)hdr; - stream->avail_in = hdrlen; - while (git_deflate(stream, 0) == Z_OK) - ; /* nothing */ - git_hash_update(c, hdr, hdrlen); - if (compat && compat_c) - git_hash_update(compat_c, hdr, hdrlen); - - return fd; -} - -/** - * Common steps for the inner git_deflate() loop for writing loose - * objects. Returns what git_deflate() returns. - */ -static int write_loose_object_common(struct odb_source_loose *loose, - struct git_hash_ctx *c, struct git_hash_ctx *compat_c, - git_zstream *stream, const int flush, - unsigned char *in0, const int fd, - unsigned char *compressed, - const size_t compressed_len) -{ - const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; - int ret; - - ret = git_deflate(stream, flush ? Z_FINISH : 0); - git_hash_update(c, in0, stream->next_in - in0); - if (compat && compat_c) - git_hash_update(compat_c, in0, stream->next_in - in0); - if (write_in_full(fd, compressed, stream->next_out - compressed) < 0) - die_errno(_("unable to write loose object file")); - stream->next_out = compressed; - stream->avail_out = compressed_len; - - return ret; -} - -/** - * Common steps for loose object writers to end writing loose objects: - * - * - End the compression of zlib stream. - * - Get the calculated oid to "oid". - */ -static int end_loose_object_common(struct odb_source_loose *loose, - struct git_hash_ctx *c, struct git_hash_ctx *compat_c, - git_zstream *stream, struct object_id *oid, - struct object_id *compat_oid) -{ - const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; - int ret; - - ret = git_deflate_end_gently(stream); - if (ret != Z_OK) - return ret; - git_hash_final_oid(oid, c); - if (compat && compat_c) - git_hash_final_oid(compat_oid, compat_c); - - return Z_OK; -} - -int write_loose_object(struct odb_source_loose *loose, - const struct object_id *oid, char *hdr, - int hdrlen, const void *buf, unsigned long len, - time_t mtime, unsigned flags) -{ - int fd, ret; - unsigned char compressed[4096]; - git_zstream stream; - struct git_hash_ctx c; - struct object_id parano_oid; - static struct strbuf tmp_file = STRBUF_INIT; - static struct strbuf filename = STRBUF_INIT; - - if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) - odb_transaction_files_prepare(loose->base.odb->transaction); - - odb_loose_path(loose, &filename, oid); - - fd = start_loose_object_common(loose, &tmp_file, filename.buf, flags, - &stream, compressed, sizeof(compressed), - &c, NULL, hdr, hdrlen); - if (fd < 0) - return -1; - - /* Then the data itself.. */ - stream.next_in = (void *)buf; - stream.avail_in = len; - do { - unsigned char *in0 = stream.next_in; - - ret = write_loose_object_common(loose, &c, NULL, &stream, 1, in0, fd, - compressed, sizeof(compressed)); - } while (ret == Z_OK); - - if (ret != Z_STREAM_END) - die(_("unable to deflate new object %s (%d)"), oid_to_hex(oid), - ret); - ret = end_loose_object_common(loose, &c, NULL, &stream, ¶no_oid, NULL); - if (ret != Z_OK) - die(_("deflateEnd on object %s failed (%d)"), oid_to_hex(oid), - ret); - if (!oideq(oid, ¶no_oid)) - die(_("confused by unstable object source data for %s"), - oid_to_hex(oid)); - - close_loose_object(loose, fd, tmp_file.buf); - - if (mtime) { - struct utimbuf utb; - utb.actime = mtime; - utb.modtime = mtime; - if (utime(tmp_file.buf, &utb) < 0 && - !(flags & ODB_WRITE_OBJECT_SILENT)) - warning_errno(_("failed utime() on %s"), tmp_file.buf); - } - - return finalize_object_file_flags(loose->base.odb->repo, tmp_file.buf, filename.buf, - FOF_SKIP_COLLISION_CHECK); -} - -int odb_source_loose_write_stream(struct odb_source_loose *loose, - struct odb_write_stream *in_stream, size_t len, - struct object_id *oid) -{ - const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; - struct object_id compat_oid; - int fd, ret, err = 0, flush = 0; - unsigned char compressed[4096]; - git_zstream stream; - struct git_hash_ctx c, compat_c; - struct strbuf tmp_file = STRBUF_INIT; - struct strbuf filename = STRBUF_INIT; - unsigned char buf[8192]; - int dirlen; - char hdr[MAX_HEADER_LEN]; - int hdrlen; - - if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) - odb_transaction_files_prepare(loose->base.odb->transaction); - - /* Since oid is not determined, save tmp file to odb path. */ - strbuf_addf(&filename, "%s/", loose->base.path); - hdrlen = format_object_header(hdr, sizeof(hdr), OBJ_BLOB, len); - - /* - * Common steps for write_loose_object and stream_loose_object to - * start writing loose objects: - * - * - Create tmpfile for the loose object. - * - Setup zlib stream for compression. - * - Start to feed header to zlib stream. - */ - fd = start_loose_object_common(loose, &tmp_file, filename.buf, 0, - &stream, compressed, sizeof(compressed), - &c, &compat_c, hdr, hdrlen); - if (fd < 0) { - err = -1; - goto cleanup; - } - - /* Then the data itself.. */ - do { - unsigned char *in0 = stream.next_in; - - if (!stream.avail_in && !in_stream->is_finished) { - ssize_t read_len = odb_write_stream_read(in_stream, buf, - sizeof(buf)); - if (read_len < 0) { - close(fd); - err = -1; - goto cleanup; - } - - stream.avail_in = read_len; - stream.next_in = buf; - in0 = buf; - /* All data has been read. */ - if (in_stream->is_finished) - flush = 1; - } - ret = write_loose_object_common(loose, &c, &compat_c, &stream, flush, in0, fd, - compressed, sizeof(compressed)); - /* - * Unlike write_loose_object(), we do not have the entire - * buffer. If we get Z_BUF_ERROR due to too few input bytes, - * then we'll replenish them in the next input_stream->read() - * call when we loop. - */ - } while (ret == Z_OK || ret == Z_BUF_ERROR); - - if (stream.total_in != len + hdrlen) - die(_("write stream object %"PRIuMAX" != %"PRIuMAX), (uintmax_t)stream.total_in, - (uintmax_t)len + hdrlen); - - /* - * Common steps for write_loose_object and stream_loose_object to - * end writing loose object: - * - * - End the compression of zlib stream. - * - Get the calculated oid. - */ - if (ret != Z_STREAM_END) - die(_("unable to stream deflate new object (%d)"), ret); - ret = end_loose_object_common(loose, &c, &compat_c, &stream, oid, &compat_oid); - if (ret != Z_OK) - die(_("deflateEnd on stream object failed (%d)"), ret); - close_loose_object(loose, fd, tmp_file.buf); - - if (odb_freshen_object(loose->base.odb, oid)) { - unlink_or_warn(tmp_file.buf); - goto cleanup; - } - odb_loose_path(loose, &filename, oid); - - /* We finally know the object path, and create the missing dir. */ - dirlen = directory_size(filename.buf); - if (dirlen) { - struct strbuf dir = STRBUF_INIT; - strbuf_add(&dir, filename.buf, dirlen); - - if (safe_create_dir_in_gitdir(loose->base.odb->repo, dir.buf) && - errno != EEXIST) { - err = error_errno(_("unable to create directory %s"), dir.buf); - strbuf_release(&dir); - goto cleanup; - } - strbuf_release(&dir); - } - - err = finalize_object_file_flags(loose->base.odb->repo, tmp_file.buf, filename.buf, - FOF_SKIP_COLLISION_CHECK); - if (!err && compat) - err = repo_add_loose_object_map(loose, oid, &compat_oid); -cleanup: - strbuf_release(&tmp_file); - strbuf_release(&filename); - return err; -} - -int force_object_loose(struct odb_source *source, - const struct object_id *oid, time_t mtime) -{ - struct odb_source_files *files = odb_source_files_downcast(source); - const struct git_hash_algo *compat = source->odb->repo->compat_hash_algo; - void *buf; - size_t len; - struct object_info oi = OBJECT_INFO_INIT; - struct object_id compat_oid; - enum object_type type; - char hdr[MAX_HEADER_LEN]; - int hdrlen; - int ret; - - for (struct odb_source *s = source->odb->sources; s; s = s->next) { - struct odb_source_files *files = odb_source_files_downcast(s); - if (!odb_source_read_object_info(&files->loose->base, oid, NULL, 0)) - return 0; - } - - oi.typep = &type; - oi.sizep = &len; - oi.contentp = &buf; - if (odb_read_object_info_extended(source->odb, oid, &oi, 0)) - return error(_("cannot read object for %s"), oid_to_hex(oid)); - if (compat) { - if (repo_oid_to_algop(source->odb->repo, oid, compat, &compat_oid)) - return error(_("cannot map object %s to %s"), - oid_to_hex(oid), compat->name); - } - hdrlen = format_object_header(hdr, sizeof(hdr), type, len); - ret = write_loose_object(files->loose, oid, hdr, hdrlen, buf, len, mtime, 0); - if (!ret && compat) - ret = repo_add_loose_object_map(files->loose, oid, &compat_oid); - free(buf); - - return ret; -} - /* * We can't use the normal fsck_error_function() for index_mem(), * because we don't yet have a valid oid for it to report. Instead, @@ -1740,3 +1340,13 @@ int odb_transaction_files_begin(struct odb_source *source, return 0; } + +void free_object_info_contents(struct object_info *object_info) +{ + if (!object_info) + return; + free(object_info->typep); + free(object_info->sizep); + free(object_info->disk_sizep); + free(object_info->delta_base_oid); +} diff --git a/object-file.h b/object-file.h index d04ffa6493ff17..805f2cfa289661 100644 --- a/object-file.h +++ b/object-file.h @@ -24,20 +24,6 @@ int index_path(struct index_state *istate, struct object_id *oid, const char *pa struct object_info; struct odb_source; -/* - * Write the given stream into the loose object source. The only difference - * from the generic implementation of this function is that we don't perform an - * object existence check here. - * - * TODO: We should stop exposing this function altogether and move it into - * "odb/source-loose.c". This requires a couple of refactorings though to make - * `force_object_loose()` generic and is thus postponed to a later point in - * time. - */ -int odb_source_loose_write_stream(struct odb_source_loose *source, - struct odb_write_stream *stream, size_t len, - struct object_id *oid); - /* * Put in `buf` the name of the file in the local object database that * would be used to store a loose object with the specified oid. @@ -98,9 +84,6 @@ int for_each_file_in_obj_subdir(unsigned int subdir_nr, int format_object_header(char *str, size_t size, enum object_type type, size_t objsize); -int force_object_loose(struct odb_source *source, - const struct object_id *oid, time_t mtime); - /** * With in-core object data in "buf", rehash it to make sure the * object name actually matches "oid" to detect object corruption. @@ -134,17 +117,10 @@ int finalize_object_file_flags(struct repository *repo, void hash_object_file(const struct git_hash_algo *algo, const void *buf, size_t len, enum object_type type, struct object_id *oid); -void write_object_file_prepare(const struct git_hash_algo *algo, - const void *buf, size_t len, - enum object_type type, struct object_id *oid, - char *hdr, size_t *hdrlen); -int write_loose_object(struct odb_source_loose *loose, - const struct object_id *oid, char *hdr, - int hdrlen, const void *buf, unsigned long len, - time_t mtime, unsigned flags); /* Helper to check and "touch" a file */ -int check_and_freshen_file(const char *fn, int freshen); +int check_and_freshen_file(const char *fn, int freshen, + const time_t *mtime); /* * Open the loose object at path, check its hash, and return the contents, @@ -201,4 +177,8 @@ int odb_transaction_files_begin(struct odb_source *source, struct odb_transaction **out, enum odb_transaction_flags flags); +int odb_transaction_files_prepare(struct odb_transaction *base); +void odb_transaction_files_fsync(struct odb_transaction *base, + int fd, const char *filename); + #endif /* OBJECT_FILE_H */ diff --git a/odb.c b/odb.c index cf6e7938c01e56..dabd481f57dbc4 100644 --- a/odb.c +++ b/odb.c @@ -738,7 +738,7 @@ int odb_pretend_object(struct object_database *odb, return 0; return odb_source_write_object(odb->inmemory_objects, - buf, len, type, oid, NULL, 0); + buf, len, type, oid, NULL, NULL, 0); } void *odb_read_object(struct object_database *odb, @@ -829,7 +829,7 @@ int odb_freshen_object(struct object_database *odb, struct odb_source *source; odb_prepare_alternates(odb); for (source = odb->sources; source; source = source->next) - if (odb_source_freshen_object(source, oid)) + if (odb_source_freshen_object(source, oid, NULL)) return 1; return 0; } @@ -989,11 +989,42 @@ int odb_write_object_ext(struct object_database *odb, const void *buf, unsigned long len, enum object_type type, struct object_id *oid, - struct object_id *compat_oid, + const struct object_id *compat_oid_in, enum odb_write_object_flags flags) { + const struct git_hash_algo *compat = odb->repo->compat_hash_algo; + struct object_id compat_oid, *compat_oid_p = NULL; + + hash_object_file(odb->repo->hash_algo, buf, len, type, oid); + + /* + * We can skip the write in case we already have the object available. + * In that case, we only freshen its mtime. + */ + if (odb_freshen_object(odb, oid)) + return 0; + + if (compat) { + const struct git_hash_algo *algo = odb->repo->hash_algo; + + if (compat_oid_in) { + oidcpy(&compat_oid, compat_oid_in); + } else if (type == OBJ_BLOB) { + hash_object_file(compat, buf, len, type, &compat_oid); + } else { + struct strbuf converted = STRBUF_INIT; + convert_object_file(odb->repo, &converted, algo, compat, + buf, len, type, 0); + hash_object_file(compat, converted.buf, converted.len, + type, &compat_oid); + strbuf_release(&converted); + } + + compat_oid_p = &compat_oid; + } + return odb_source_write_object(odb->sources, buf, len, type, - oid, compat_oid, flags); + oid, compat_oid_p, NULL, flags); } int odb_write_object_stream(struct object_database *odb, diff --git a/odb.h b/odb.h index 7995bed97bc36e..cbc2f9ced42338 100644 --- a/odb.h +++ b/odb.h @@ -340,6 +340,12 @@ struct object_info { * or multiple times in the same source. */ struct odb_source_info *source_infop; + + /* + * object-info protocol specific. Set by the protocol when the remote + * does not recognize the requested object. + */ + unsigned int unrecognized:1; }; /* @@ -597,9 +603,11 @@ enum odb_write_object_flags { /* * Write an object into the object database. The object is being written into - * the local alternate of the repository. If provided, the converted object ID - * as well as the compatibility object ID are written to the respective - * pointers. + * the local alternate of the repository. If provided, the object ID of the + * final object is written into `oid`. + * + * If the caller provides a `compat_oid`, then this compatibility object hash + * will be stored instead of computing the compatibility hash ad-hoc. * * Returns 0 on success, a negative error code otherwise. */ @@ -607,7 +615,7 @@ int odb_write_object_ext(struct object_database *odb, const void *buf, unsigned long len, enum object_type type, struct object_id *oid, - struct object_id *compat_oid, + const struct object_id *compat_oid, enum odb_write_object_flags flags); static inline int odb_write_object(struct object_database *odb, @@ -629,4 +637,7 @@ void parse_alternates(const char *string, const char *relative_base, struct strvec *out); +/* Free pointers inside of object_info, but not object_info itself */ +void free_object_info_contents(struct object_info *object_info); + #endif /* ODB_H */ diff --git a/odb/source-files.c b/odb/source-files.c index 413875851135a5..5e086d266fac4f 100644 --- a/odb/source-files.c +++ b/odb/source-files.c @@ -150,11 +150,12 @@ static int odb_source_files_find_abbrev_len(struct odb_source *source, } static int odb_source_files_freshen_object(struct odb_source *source, - const struct object_id *oid) + const struct object_id *oid, + const time_t *mtime) { struct odb_source_files *files = odb_source_files_downcast(source); - if (odb_source_freshen_object(&files->packed->base, oid) || - odb_source_freshen_object(&files->loose->base, oid)) + if (odb_source_freshen_object(&files->packed->base, oid, mtime) || + odb_source_freshen_object(&files->loose->base, oid, mtime)) return 1; return 0; } @@ -162,13 +163,14 @@ static int odb_source_files_freshen_object(struct odb_source *source, static int odb_source_files_write_object(struct odb_source *source, const void *buf, size_t len, enum object_type type, - struct object_id *oid, - struct object_id *compat_oid, + const struct object_id *oid, + const struct object_id *compat_oid, + const time_t *mtime, enum odb_write_object_flags flags) { struct odb_source_files *files = odb_source_files_downcast(source); return odb_source_write_object(&files->loose->base, buf, len, type, - oid, compat_oid, flags); + oid, compat_oid, mtime, flags); } static int odb_source_files_write_object_stream(struct odb_source *source, @@ -215,7 +217,8 @@ static int odb_source_files_write_alternate(struct odb_source *source, int found = 0; int ret; - hold_lock_file_for_update(&lock, path, LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(source->odb->repo, &lock, path, + LOCK_DIE_ON_ERROR); out = fdopen_lock_file(&lock, "w"); if (!out) { ret = error_errno(_("unable to fdopen alternates lockfile")); diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c index e47bfd8fccabf0..3e71611b8e0071 100644 --- a/odb/source-inmemory.c +++ b/odb/source-inmemory.c @@ -230,15 +230,14 @@ static int odb_source_inmemory_count_objects(struct odb_source *source, static int odb_source_inmemory_write_object(struct odb_source *source, const void *buf, size_t len, enum object_type type, - struct object_id *oid, - struct object_id *compat_oid UNUSED, + const struct object_id *oid, + const struct object_id *compat_oid UNUSED, + const time_t *mtime UNUSED, enum odb_write_object_flags flags UNUSED) { struct odb_source_inmemory *inmemory = odb_source_inmemory_downcast(source); struct inmemory_object *object; - hash_object_file(source->odb->repo->hash_algo, buf, len, type, oid); - if (!inmemory->objects) { CALLOC_ARRAY(inmemory->objects, 1); oidtree_init(inmemory->objects); @@ -285,8 +284,10 @@ static int odb_source_inmemory_write_object_stream(struct odb_source *source, goto out; } + hash_object_file(source->odb->repo->hash_algo, data, total_read, OBJ_BLOB, oid); + ret = odb_source_inmemory_write_object(source, data, len, OBJ_BLOB, oid, - NULL, 0); + NULL, NULL, 0); if (ret < 0) goto out; @@ -296,7 +297,8 @@ static int odb_source_inmemory_write_object_stream(struct odb_source *source, } static int odb_source_inmemory_freshen_object(struct odb_source *source, - const struct object_id *oid) + const struct object_id *oid, + const time_t *mtime UNUSED) { struct odb_source_inmemory *inmemory = odb_source_inmemory_downcast(source); if (find_cached_object(inmemory, oid)) diff --git a/odb/source-loose.c b/odb/source-loose.c index 3f7d04a56e36ce..ef0e9192777c4a 100644 --- a/odb/source-loose.c +++ b/odb/source-loose.c @@ -11,8 +11,11 @@ #include "odb/source-loose.h" #include "odb/streaming.h" #include "oidtree.h" +#include "path.h" #include "repository.h" #include "strbuf.h" +#include "tempfile.h" +#include "write-or-die.h" static int append_loose_object(const struct object_id *oid, const char *path UNUSED, @@ -574,53 +577,270 @@ static int odb_source_loose_count_objects(struct odb_source *source, } static int odb_source_loose_freshen_object(struct odb_source *source, - const struct object_id *oid) + const struct object_id *oid, + const time_t *mtime) { struct odb_source_loose *loose = odb_source_loose_downcast(source); static struct strbuf path = STRBUF_INIT; odb_loose_path(loose, &path, oid); - return !!check_and_freshen_file(path.buf, 1); + return !!check_and_freshen_file(path.buf, 1, mtime); +} + +/* Finalize a file on disk, and close it. */ +static void close_loose_object(struct odb_source_loose *loose, + int fd, const char *filename) +{ + if (loose->base.will_destroy) + goto out; + + if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) + odb_transaction_files_fsync(loose->base.odb->transaction, fd, filename); + else if (fsync_object_files > 0) + fsync_or_die(fd, filename); + else + fsync_component_or_die(FSYNC_COMPONENT_LOOSE_OBJECT, fd, + filename); + +out: + if (close(fd) != 0) + die_errno(_("error when closing loose object file")); +} + +/* Size of directory component, including the ending '/' */ +static inline int directory_size(const char *filename) +{ + const char *s = strrchr(filename, '/'); + if (!s) + return 0; + return s - filename + 1; +} + +/* + * This creates a temporary file in the same directory as the final + * 'filename' + * + * We want to avoid cross-directory filename renames, because those + * can have problems on various filesystems (FAT, NFS, Coda). + */ +static int create_tmpfile(struct repository *repo, + struct strbuf *tmp, const char *filename) +{ + int fd, dirlen = directory_size(filename); + + strbuf_reset(tmp); + strbuf_add(tmp, filename, dirlen); + strbuf_addstr(tmp, "tmp_obj_XXXXXX"); + fd = git_mkstemp_mode(tmp->buf, 0444); + if (fd < 0 && dirlen && errno == ENOENT) { + /* + * Make sure the directory exists; note that the contents + * of the buffer are undefined after mkstemp returns an + * error, so we have to rewrite the whole buffer from + * scratch. + */ + strbuf_reset(tmp); + strbuf_add(tmp, filename, dirlen - 1); + if (mkdir(tmp->buf, 0777) && errno != EEXIST) + return -1; + if (adjust_shared_perm(repo, tmp->buf)) + return -1; + + /* Try again */ + strbuf_addstr(tmp, "/tmp_obj_XXXXXX"); + fd = git_mkstemp_mode(tmp->buf, 0444); + } + return fd; +} + +/** + * Common steps for loose object writers to start writing loose + * objects: + * + * - Create tmpfile for the loose object. + * - Setup zlib stream for compression. + * - Start to feed header to zlib stream. + * + * Returns a "fd", which should later be provided to + * end_loose_object_common(). + */ +static int start_loose_object_common(struct odb_source_loose *loose, + struct strbuf *tmp_file, + const char *filename, unsigned flags, + git_zstream *stream, + unsigned char *buf, size_t buflen, + struct git_hash_ctx *c, struct git_hash_ctx *compat_c, + char *hdr, int hdrlen) +{ + const struct git_hash_algo *algo = loose->base.odb->repo->hash_algo; + const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; + int fd; + struct repo_config_values *cfg = repo_config_values(loose->base.odb->repo); + + fd = create_tmpfile(loose->base.odb->repo, tmp_file, filename); + if (fd < 0) { + if (flags & ODB_WRITE_OBJECT_SILENT) + return -1; + else if (errno == EACCES) + return error(_("insufficient permission for adding " + "an object to repository database %s"), + loose->base.path); + else + return error_errno( + _("unable to create temporary file")); + } + + /* Setup zlib stream for compression */ + git_deflate_init(stream, cfg->zlib_compression_level); + stream->next_out = buf; + stream->avail_out = buflen; + git_hash_init(c, algo); + if (compat && compat_c) + git_hash_init(compat_c, compat); + + /* Start to feed header to zlib stream */ + stream->next_in = (unsigned char *)hdr; + stream->avail_in = hdrlen; + while (git_deflate(stream, 0) == Z_OK) + ; /* nothing */ + git_hash_update(c, hdr, hdrlen); + if (compat && compat_c) + git_hash_update(compat_c, hdr, hdrlen); + + return fd; +} + +/** + * Common steps for the inner git_deflate() loop for writing loose + * objects. Returns what git_deflate() returns. + */ +static int write_loose_object_common(struct odb_source_loose *loose, + struct git_hash_ctx *c, struct git_hash_ctx *compat_c, + git_zstream *stream, const int flush, + unsigned char *in0, const int fd, + unsigned char *compressed, + const size_t compressed_len) +{ + const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; + int ret; + + ret = git_deflate(stream, flush ? Z_FINISH : 0); + git_hash_update(c, in0, stream->next_in - in0); + if (compat && compat_c) + git_hash_update(compat_c, in0, stream->next_in - in0); + if (write_in_full(fd, compressed, stream->next_out - compressed) < 0) + die_errno(_("unable to write loose object file")); + stream->next_out = compressed; + stream->avail_out = compressed_len; + + return ret; +} + +/** + * Common steps for loose object writers to end writing loose objects: + * + * - End the compression of zlib stream. + * - Get the calculated oid to "oid". + */ +static int end_loose_object_common(struct odb_source_loose *loose, + struct git_hash_ctx *c, struct git_hash_ctx *compat_c, + git_zstream *stream, struct object_id *oid, + struct object_id *compat_oid) +{ + const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; + int ret; + + ret = git_deflate_end_gently(stream); + if (ret != Z_OK) + return ret; + git_hash_final_oid(oid, c); + if (compat && compat_c) + git_hash_final_oid(compat_oid, compat_c); + + return Z_OK; +} + +static int write_loose_object(struct odb_source_loose *loose, + const struct object_id *oid, char *hdr, + int hdrlen, const void *buf, unsigned long len, + const time_t *mtime, unsigned flags) +{ + int fd, ret; + unsigned char compressed[4096]; + git_zstream stream; + struct git_hash_ctx c; + struct object_id parano_oid; + static struct strbuf tmp_file = STRBUF_INIT; + static struct strbuf filename = STRBUF_INIT; + + if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) + odb_transaction_files_prepare(loose->base.odb->transaction); + + odb_loose_path(loose, &filename, oid); + + fd = start_loose_object_common(loose, &tmp_file, filename.buf, flags, + &stream, compressed, sizeof(compressed), + &c, NULL, hdr, hdrlen); + if (fd < 0) + return -1; + + /* Then the data itself.. */ + stream.next_in = (void *)buf; + stream.avail_in = len; + do { + unsigned char *in0 = stream.next_in; + + ret = write_loose_object_common(loose, &c, NULL, &stream, 1, in0, fd, + compressed, sizeof(compressed)); + } while (ret == Z_OK); + + if (ret != Z_STREAM_END) + die(_("unable to deflate new object %s (%d)"), oid_to_hex(oid), + ret); + ret = end_loose_object_common(loose, &c, NULL, &stream, ¶no_oid, NULL); + if (ret != Z_OK) + die(_("deflateEnd on object %s failed (%d)"), oid_to_hex(oid), + ret); + if (!oideq(oid, ¶no_oid)) + die(_("confused by unstable object source data for %s"), + oid_to_hex(oid)); + + close_loose_object(loose, fd, tmp_file.buf); + + if (mtime) { + struct utimbuf utb = { + .actime = *mtime, + .modtime = *mtime, + }; + + if (utime(tmp_file.buf, &utb) < 0 && + !(flags & ODB_WRITE_OBJECT_SILENT)) + warning_errno(_("failed utime() on %s"), tmp_file.buf); + } + + return finalize_object_file_flags(loose->base.odb->repo, tmp_file.buf, filename.buf, + FOF_SKIP_COLLISION_CHECK); } static int odb_source_loose_write_object(struct odb_source *source, const void *buf, size_t len, - enum object_type type, struct object_id *oid, - struct object_id *compat_oid_in, + enum object_type type, + const struct object_id *oid, + const struct object_id *compat_oid, + const time_t *mtime, enum odb_write_object_flags flags) { struct odb_source_loose *loose = odb_source_loose_downcast(source); - const struct git_hash_algo *algo = source->odb->repo->hash_algo; - const struct git_hash_algo *compat = source->odb->repo->compat_hash_algo; - struct object_id compat_oid; char hdr[MAX_HEADER_LEN]; - size_t hdrlen = sizeof(hdr); - - /* Generate compat_oid */ - if (compat) { - if (compat_oid_in) - oidcpy(&compat_oid, compat_oid_in); - else if (type == OBJ_BLOB) - hash_object_file(compat, buf, len, type, &compat_oid); - else { - struct strbuf converted = STRBUF_INIT; - convert_object_file(source->odb->repo, &converted, algo, compat, - buf, len, type, 0); - hash_object_file(compat, converted.buf, converted.len, - type, &compat_oid); - strbuf_release(&converted); - } - } + int hdrlen; - /* Normally if we have it in the pack then we do not bother writing - * it out into .git/objects/??/?{38} file. - */ - write_object_file_prepare(algo, buf, len, type, oid, hdr, &hdrlen); - if (odb_freshen_object(source->odb, oid)) - return 0; - if (write_loose_object(loose, oid, hdr, hdrlen, buf, len, 0, flags)) + hdrlen = format_object_header(hdr, sizeof(hdr), type, len); + + if (write_loose_object(loose, oid, hdr, hdrlen, buf, len, mtime, flags)) return -1; - if (compat) - return repo_add_loose_object_map(loose, oid, &compat_oid); + + if (compat_oid) + return repo_add_loose_object_map(loose, oid, compat_oid); + return 0; } @@ -629,12 +849,120 @@ static int odb_source_loose_write_object_stream(struct odb_source *source, size_t len, struct object_id *oid) { + struct odb_source_loose *loose = odb_source_loose_downcast(source); + const struct git_hash_algo *compat = loose->base.odb->repo->compat_hash_algo; + struct object_id compat_oid; + int fd, ret, err = 0, flush = 0; + unsigned char compressed[4096]; + git_zstream stream; + struct git_hash_ctx c, compat_c; + struct strbuf tmp_file = STRBUF_INIT; + struct strbuf filename = STRBUF_INIT; + unsigned char buf[8192]; + int dirlen; + char hdr[MAX_HEADER_LEN]; + int hdrlen; + + if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT)) + odb_transaction_files_prepare(loose->base.odb->transaction); + + /* Since oid is not determined, save tmp file to odb path. */ + strbuf_addf(&filename, "%s/", loose->base.path); + hdrlen = format_object_header(hdr, sizeof(hdr), OBJ_BLOB, len); + /* - * TODO: the implementation should be moved here, see the comment on - * the called function in "object-file.h". + * Common steps for write_loose_object and stream_loose_object to + * start writing loose objects: + * + * - Create tmpfile for the loose object. + * - Setup zlib stream for compression. + * - Start to feed header to zlib stream. */ - struct odb_source_loose *loose = odb_source_loose_downcast(source); - return odb_source_loose_write_stream(loose, in_stream, len, oid); + fd = start_loose_object_common(loose, &tmp_file, filename.buf, 0, + &stream, compressed, sizeof(compressed), + &c, &compat_c, hdr, hdrlen); + if (fd < 0) { + err = -1; + goto cleanup; + } + + /* Then the data itself.. */ + do { + unsigned char *in0 = stream.next_in; + + if (!stream.avail_in && !in_stream->is_finished) { + ssize_t read_len = odb_write_stream_read(in_stream, buf, + sizeof(buf)); + if (read_len < 0) { + close(fd); + err = -1; + goto cleanup; + } + + stream.avail_in = read_len; + stream.next_in = buf; + in0 = buf; + /* All data has been read. */ + if (in_stream->is_finished) + flush = 1; + } + ret = write_loose_object_common(loose, &c, &compat_c, &stream, flush, in0, fd, + compressed, sizeof(compressed)); + /* + * Unlike write_loose_object(), we do not have the entire + * buffer. If we get Z_BUF_ERROR due to too few input bytes, + * then we'll replenish them in the next input_stream->read() + * call when we loop. + */ + } while (ret == Z_OK || ret == Z_BUF_ERROR); + + if (stream.total_in != len + hdrlen) + die(_("write stream object %"PRIuMAX" != %"PRIuMAX), (uintmax_t)stream.total_in, + (uintmax_t)len + hdrlen); + + /* + * Common steps for write_loose_object and stream_loose_object to + * end writing loose object: + * + * - End the compression of zlib stream. + * - Get the calculated oid. + */ + if (ret != Z_STREAM_END) + die(_("unable to stream deflate new object (%d)"), ret); + ret = end_loose_object_common(loose, &c, &compat_c, &stream, oid, &compat_oid); + if (ret != Z_OK) + die(_("deflateEnd on stream object failed (%d)"), ret); + close_loose_object(loose, fd, tmp_file.buf); + + if (odb_freshen_object(loose->base.odb, oid)) { + unlink_or_warn(tmp_file.buf); + goto cleanup; + } + odb_loose_path(loose, &filename, oid); + + /* We finally know the object path, and create the missing dir. */ + dirlen = directory_size(filename.buf); + if (dirlen) { + struct strbuf dir = STRBUF_INIT; + strbuf_add(&dir, filename.buf, dirlen); + + if (safe_create_dir_in_gitdir(loose->base.odb->repo, dir.buf) && + errno != EEXIST) { + err = error_errno(_("unable to create directory %s"), dir.buf); + strbuf_release(&dir); + goto cleanup; + } + strbuf_release(&dir); + } + + err = finalize_object_file_flags(loose->base.odb->repo, tmp_file.buf, filename.buf, + FOF_SKIP_COLLISION_CHECK); + if (!err && compat) + err = repo_add_loose_object_map(loose, oid, &compat_oid); +cleanup: + strbuf_release(&tmp_file); + strbuf_release(&filename); + return err; } static int odb_source_loose_begin_transaction(struct odb_source *source UNUSED, diff --git a/odb/source-packed.c b/odb/source-packed.c index d00aec5a452c58..0890704e76879b 100644 --- a/odb/source-packed.c +++ b/odb/source-packed.c @@ -570,18 +570,26 @@ static int odb_source_packed_find_abbrev_len(struct odb_source *source, } static int odb_source_packed_freshen_object(struct odb_source *source, - const struct object_id *oid) + const struct object_id *oid, + const time_t *mtime) { struct odb_source_packed *packed = odb_source_packed_downcast(source); + struct utimbuf times, *timesp = NULL; struct pack_entry e; + if (mtime) { + times.actime = *mtime; + times.modtime = *mtime; + timesp = × + } + if (!find_pack_entry(packed, oid, &e)) return 0; if (e.p->is_cruft) return 0; if (e.p->freshened) return 1; - if (utime(e.p->pack_name, NULL)) + if (utime(e.p->pack_name, timesp)) return 0; e.p->freshened = 1; @@ -592,8 +600,9 @@ static int odb_source_packed_write_object(struct odb_source *source UNUSED, const void *buf UNUSED, size_t len UNUSED, enum object_type type UNUSED, - struct object_id *oid UNUSED, - struct object_id *compat_oid UNUSED, + const struct object_id *oid UNUSED, + const struct object_id *compat_oid UNUSED, + const time_t *mtime UNUSED, unsigned flags UNUSED) { return error("packed backend cannot write objects"); diff --git a/odb/source.h b/odb/source.h index cd63dba91f4e2f..fc04dd5cda8800 100644 --- a/odb/source.h +++ b/odb/source.h @@ -190,7 +190,8 @@ struct odb_source { * has been freshened. */ int (*freshen_object)(struct odb_source *source, - const struct object_id *oid); + const struct object_id *oid, + const time_t *mtime); /* * This callback is expected to persist the given object into the @@ -206,8 +207,9 @@ struct odb_source { int (*write_object)(struct odb_source *source, const void *buf, size_t len, enum object_type type, - struct object_id *oid, - struct object_id *compat_oid, + const struct object_id *oid, + const struct object_id *compat_oid, + const time_t *mtime, enum odb_write_object_flags flags); /* @@ -403,9 +405,10 @@ static inline int odb_source_find_abbrev_len(struct odb_source *source, * not exist. */ static inline int odb_source_freshen_object(struct odb_source *source, - const struct object_id *oid) + const struct object_id *oid, + const time_t *mtime) { - return source->freshen_object(source, oid); + return source->freshen_object(source, oid, mtime); } /* @@ -416,12 +419,13 @@ static inline int odb_source_freshen_object(struct odb_source *source, static inline int odb_source_write_object(struct odb_source *source, const void *buf, unsigned long len, enum object_type type, - struct object_id *oid, - struct object_id *compat_oid, + const struct object_id *oid, + const struct object_id *compat_oid, + const time_t *mtime, enum odb_write_object_flags flags) { return source->write_object(source, buf, len, type, oid, - compat_oid, flags); + compat_oid, mtime, flags); } /* diff --git a/pager.c b/pager.c index 35b210e0484f90..543ef129366a19 100644 --- a/pager.c +++ b/pager.c @@ -5,6 +5,8 @@ #include "run-command.h" #include "sigchain.h" #include "alias.h" +#include "repository.h" +#include "environment.h" int pager_use_color = 1; @@ -13,7 +15,6 @@ int pager_use_color = 1; #endif static struct child_process pager_process; -static char *pager_program; static int old_fd1 = -1, old_fd2 = -1; /* Is the value coming back from term_columns() just a guess? */ @@ -75,10 +76,17 @@ static void wait_for_pager_signal(int signo) static int core_pager_config(const char *var, const char *value, const struct config_context *ctx UNUSED, - void *data UNUSED) + void *data) { - if (!strcmp(var, "core.pager")) - return git_config_string(&pager_program, var, value); + struct repository *r = data; + + if (!strcmp(var, "core.pager")) { + struct repo_config_values *cfg = repo_config_values(r); + + FREE_AND_NULL(cfg->pager_program); + return git_config_string(&cfg->pager_program, var, value); + } + return 0; } @@ -91,10 +99,12 @@ const char *git_pager(struct repository *r, int stdout_is_tty) pager = getenv("GIT_PAGER"); if (!pager) { - if (!pager_program) + struct repo_config_values *cfg = repo_config_values(r); + + if (!cfg->pager_program) read_early_config(r, - core_pager_config, NULL); - pager = pager_program; + core_pager_config, r); + pager = cfg->pager_program; } if (!pager) pager = getenv("PAGER"); @@ -302,7 +312,11 @@ int check_pager_config(struct repository *r, const char *cmd) read_early_config(r, pager_command_config, &data); - if (data.value) - pager_program = data.value; + if (data.value) { + struct repo_config_values *cfg = repo_config_values(r); + + free(cfg->pager_program); + cfg->pager_program = data.value; + } return data.want; } diff --git a/pathspec.c b/pathspec.c index f78b22709ccb67..281858f21f9c59 100644 --- a/pathspec.c +++ b/pathspec.c @@ -847,9 +847,9 @@ int pathspec_needs_expanded_index(struct index_state *istate, * - not-in-cone/bar*: may need expanded index * - **.c: may need expanded index */ - if (strspn(item.original + item.nowildcard_len, "*") == + if (strspn(item.match + item.nowildcard_len, "*") == (unsigned int)(item.len - item.nowildcard_len) && - path_in_cone_mode_sparse_checkout(item.original, istate)) + path_in_cone_mode_sparse_checkout(item.match, istate)) continue; for (pos = 0; pos < istate->cache_nr; pos++) { @@ -865,7 +865,7 @@ int pathspec_needs_expanded_index(struct index_state *istate, */ if ((unsigned int)item.nowildcard_len > ce_namelen(ce) && - !strncmp(item.original, ce->name, + !strncmp(item.match, ce->name, ce_namelen(ce))) { res = 1; break; @@ -876,13 +876,13 @@ int pathspec_needs_expanded_index(struct index_state *istate, * directory and the pathspec does not match the whole * directory, need to expand the index. */ - if (!strncmp(item.original, ce->name, item.nowildcard_len) && - wildmatch(item.original, ce->name, 0)) { + if (!strncmp(item.match, ce->name, item.nowildcard_len) && + wildmatch(item.match, ce->name, 0)) { res = 1; break; } } - } else if (!path_in_cone_mode_sparse_checkout(item.original, istate) && + } else if (!path_in_cone_mode_sparse_checkout(item.match, istate) && !matches_skip_worktree(pathspec, i, &skip_worktree_seen)) res = 1; diff --git a/prompt.c b/prompt.c index 706fba2a5030c7..d8d74c7e379dbe 100644 --- a/prompt.c +++ b/prompt.c @@ -3,6 +3,7 @@ #include "git-compat-util.h" #include "parse.h" #include "environment.h" +#include "repository.h" #include "run-command.h" #include "strbuf.h" #include "prompt.h" @@ -51,7 +52,7 @@ char *git_prompt(const char *prompt, int flags) askpass = getenv("GIT_ASKPASS"); if (!askpass) - askpass = askpass_program; + askpass = repo_config_values(the_repository)->askpass_program; if (!askpass) askpass = getenv("SSH_ASKPASS"); if (askpass && *askpass) diff --git a/protocol-caps.c b/protocol-caps.c index 8858ea4489d714..02261be14d817a 100644 --- a/protocol-caps.c +++ b/protocol-caps.c @@ -30,6 +30,32 @@ static int parse_oid(const char *line, struct string_list *oid_str_list) return 1; } +/* + * odb_read_object_info_extended() wrapper. Similar to odb_read_object_info() + * but uses the flags: + * + * - OBJECT_INFO_SKIP_FETCH_OBJECT so a server won't fetch an object when a + * object-info request asks for an OID that it doesn't have. + * + * - OBJECT_INFO_QUICK to avoid re-scanning packs when the object is not found. + */ +static enum object_type get_object_info(struct object_database *odb, + const struct object_id *oid, + size_t *sizep) +{ + enum object_type type; + struct object_info oi = OBJECT_INFO_INIT; + + oi.typep = &type; + oi.sizep = sizep; + if (odb_read_object_info_extended(odb, oid, &oi, + OBJECT_INFO_LOOKUP_REPLACE | + OBJECT_INFO_SKIP_FETCH_OBJECT | + OBJECT_INFO_QUICK) < 0) + return OBJ_BAD; + return type; +} + /* * Validates and send requested info back to the client. Any errors detected * are returned as they are detected. @@ -62,15 +88,22 @@ static void send_info(struct repository *r, struct packet_writer *writer, strbuf_addstr(&send_buffer, oid_str); + /* + * Check the existence of the object first. + * If an object is not recognized by the server append SP to + * the response. + */ + if (get_object_info(r->objects, &oid, &object_size) <= OBJ_NONE) { + strbuf_addstr(&send_buffer, " "); + goto write; + } + if (info->size) { - if (odb_read_object_info(r->objects, &oid, &object_size) < 0) { - strbuf_addstr(&send_buffer, " "); - } else { - strbuf_addf(&send_buffer, " %"PRIuMAX, - (uintmax_t)object_size); - } + strbuf_addf(&send_buffer, " %"PRIuMAX, + (uintmax_t)object_size); } +write: packet_writer_write(writer, "%s", send_buffer.buf); strbuf_reset(&send_buffer); } diff --git a/read-cache.c b/read-cache.c index 38b55323dd739a..6c449f393d8d4d 100644 --- a/read-cache.c +++ b/read-cache.c @@ -205,13 +205,11 @@ void fill_stat_cache_info(struct index_state *istate, struct cache_entry *ce, st static unsigned int st_mode_from_ce(const struct cache_entry *ce) { - extern int trust_executable_bit, has_symlinks; - switch (ce->ce_mode & S_IFMT) { case S_IFLNK: - return has_symlinks ? S_IFLNK : (S_IFREG | 0644); + return repo_has_symlinks(the_repository) ? S_IFLNK : (S_IFREG | 0644); case S_IFREG: - return (ce->ce_mode & (trust_executable_bit ? 0755 : 0644)) | S_IFREG; + return (ce->ce_mode & (repo_trust_executable_bit(the_repository) ? 0755 : 0644)) | S_IFREG; case S_IFGITLINK: return S_IFDIR | 0755; case S_IFDIR: @@ -321,13 +319,13 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st) /* We consider only the owner x bit to be relevant for * "mode changes" */ - if (trust_executable_bit && + if (repo_trust_executable_bit(the_repository) && (0100 & (ce->ce_mode ^ st->st_mode))) changed |= MODE_CHANGED; break; case S_IFLNK: if (!S_ISLNK(st->st_mode) && - (has_symlinks || !S_ISREG(st->st_mode))) + (repo_has_symlinks(the_repository) || !S_ISREG(st->st_mode))) changed |= TYPE_CHANGED; break; case S_IFGITLINK: @@ -742,7 +740,8 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st, ce->ce_flags |= CE_INTENT_TO_ADD; - if (trust_executable_bit && has_symlinks) { + if (repo_trust_executable_bit(istate->repo) && + repo_has_symlinks(istate->repo)) { ce->ce_mode = create_ce_mode(st_mode); } else { /* If there is an existing entry, pick the mode bits and type @@ -752,7 +751,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st, int pos = index_name_pos_also_unmerged(istate, path, namelen); ent = (0 <= pos) ? istate->cache[pos] : NULL; - ce->ce_mode = ce_mode_from_stat(ent, st_mode); + ce->ce_mode = ce_mode_from_stat(istate->repo, ent, st_mode); } /* When core.ignorecase=true, determine if a directory of the same name but differing @@ -2342,7 +2341,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist) */ static void freshen_shared_index(const char *shared_index, int warn) { - if (!check_and_freshen_file(shared_index, 1) && warn) + if (!check_and_freshen_file(shared_index, 1, NULL) && warn) warning(_("could not freshen shared index '%s'"), shared_index); } diff --git a/read-cache.h b/read-cache.h index 043da1f1aae706..ab9d40aa81f95e 100644 --- a/read-cache.h +++ b/read-cache.h @@ -4,15 +4,23 @@ #include "read-cache-ll.h" #include "object.h" #include "pathspec.h" +#include "environment.h" -static inline unsigned int ce_mode_from_stat(const struct cache_entry *ce, +/* + * Determine the appropriate index mode for a file based on its stat() + * information and the existing cache entry (if any). + * + * This function handles degradation for filesystems that lack + * symlink support or reliable executable bits. + */ +static inline unsigned int ce_mode_from_stat(struct repository *repo, + const struct cache_entry *ce, unsigned int mode) { - extern int trust_executable_bit, has_symlinks; - if (!has_symlinks && S_ISREG(mode) && + if (S_ISREG(mode) && !repo_has_symlinks(repo) && ce && S_ISLNK(ce->ce_mode)) return ce->ce_mode; - if (!trust_executable_bit && S_ISREG(mode)) { + if (S_ISREG(mode) && !repo_trust_executable_bit(repo)) { if (ce && S_ISREG(ce->ce_mode)) return ce->ce_mode; return create_ce_mode(0666); diff --git a/refs/files-backend.c b/refs/files-backend.c index 66212b2e2daddc..1cc20aa486c263 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -844,7 +844,7 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs, goto error_return; } - if (hold_lock_file_for_update_timeout( + if (repo_hold_lock_file_for_update_timeout(refs->base.repo, &lock->lk, ref_file.buf, LOCK_NO_DEREF, get_files_ref_lock_timeout_ms(transaction->ref_store->repo)) < 0) { int myerr = errno; @@ -1253,8 +1253,8 @@ struct create_reflock_cb { static int create_reflock(const char *path, void *cb) { struct create_reflock_cb *data = cb; - return hold_lock_file_for_update_timeout( - data->lk, path, LOCK_NO_DEREF, + return repo_hold_lock_file_for_update_timeout( + data->repo, data->lk, path, LOCK_NO_DEREF, get_files_ref_lock_timeout_ms(data->repo)) < 0 ? -1 : 0; } @@ -3584,7 +3584,9 @@ static int files_reflog_expire(struct ref_store *ref_store, * work we need, including cleaning up if the program * exits unexpectedly. */ - if (hold_lock_file_for_update(&reflog_lock, log_file, 0) < 0) { + if (repo_hold_lock_file_for_update(ref_store->repo, + &reflog_lock, log_file, + 0) < 0) { struct strbuf err = STRBUF_INIT; unable_to_lock_message(log_file, errno, &err); error("%s", err.buf); diff --git a/refs/packed-backend.c b/refs/packed-backend.c index c5d96793fa5841..b9b04b70105f9a 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -1252,10 +1252,9 @@ int packed_refs_lock(struct ref_store *ref_store, int flags, struct strbuf *err) * don't write new content to it, but rather to a separate * tempfile. */ - if (hold_lock_file_for_update_timeout( - &refs->lock, - refs->path, - flags, refs->timeout_value) < 0) { + if (repo_hold_lock_file_for_update_timeout(ref_store->repo, &refs->lock, + refs->path, flags, + refs->timeout_value) < 0) { unable_to_lock_message(refs->path, errno, err); return -1; } @@ -1399,7 +1398,7 @@ static enum ref_transaction_error write_with_updates(struct packed_ref_store *re packed_refs_path = get_locked_file_path(&refs->lock); strbuf_addf(&sb, "%s.new", packed_refs_path); free(packed_refs_path); - refs->tempfile = create_tempfile(sb.buf); + refs->tempfile = repo_create_tempfile(refs->base.repo, sb.buf); if (!refs->tempfile) { strbuf_addf(err, "unable to create file %s: %s", sb.buf, strerror(errno)); diff --git a/refs/packed-backend.h b/refs/packed-backend.h index 1db48e801d63d0..8a7b3238254d3d 100644 --- a/refs/packed-backend.h +++ b/refs/packed-backend.h @@ -21,7 +21,7 @@ struct ref_store *packed_ref_store_init(struct repository *repo, /* * Lock the packed-refs file for writing. Flags is passed to - * hold_lock_file_for_update(). Return 0 on success. On errors, write + * repo_hold_lock_file_for_update(). Return 0 on success. On errors, write * an error message to `err` and return a nonzero value. */ int packed_refs_lock(struct ref_store *ref_store, int flags, struct strbuf *err); diff --git a/remote.c b/remote.c index f98278ee86b738..b55fc6e18a487f 100644 --- a/remote.c +++ b/remote.c @@ -1933,7 +1933,7 @@ static char *branch_get_push_1(struct repository *repo, if (remote->mirror) return tracking_for_push_dest(remote, branch->refname, err); - switch (push_default) { + switch (repo_config_values(repo)->push_default) { case PUSH_DEFAULT_NOTHING: return error_buf(err, _("push has no destination (push.default is 'nothing')")); diff --git a/repack-midx.c b/repack-midx.c index 7c7c3620e50b7d..64c7f8d0f42372 100644 --- a/repack-midx.c +++ b/repack-midx.c @@ -954,7 +954,8 @@ static int write_midx_incremental(struct repack_write_midx_opts *opts) lock_name.buf)) die_errno(_("unable to create leading directories of %s"), lock_name.buf); - hold_lock_file_for_update(&lf, lock_name.buf, LOCK_DIE_ON_ERROR); + repo_hold_lock_file_for_update(opts->existing->repo, &lf, lock_name.buf, + LOCK_DIE_ON_ERROR); if (!fdopen_lock_file(&lf, "w")) { ret = error_errno(_("unable to open multi-pack-index chain file")); diff --git a/repository.c b/repository.c index 2ef0778846bcf1..651b0f69339452 100644 --- a/repository.c +++ b/repository.c @@ -390,6 +390,7 @@ void repo_clear(struct repository *repo) FREE_AND_NULL(repo->parsed_objects); repo_settings_clear(repo); + repo_config_values_clear(&repo->config_values_private_); if (repo->config) { git_configset_clear(repo->config); @@ -473,5 +474,5 @@ int repo_hold_locked_index(struct repository *repo, { if (!repo->index_file) BUG("the repo hasn't been setup"); - return hold_lock_file_for_update(lf, repo->index_file, flags); + return repo_hold_lock_file_for_update(repo, lf, repo->index_file, flags); } diff --git a/repository.h b/repository.h index b7673079119e6f..3b467a25133670 100644 --- a/repository.h +++ b/repository.h @@ -167,7 +167,7 @@ struct repository { /* Repository's compatibility hash algorithm. */ const struct git_hash_algo *compat_hash_algo; - /* Repository's config values parsed by git_default_config() */ + /* Repository-specific configuration values. */ struct repo_config_values config_values_private_; /* Repository's reference storage format, as serialized on disk. */ diff --git a/rerere.c b/rerere.c index 1dda246098900b..0015e25050df76 100644 --- a/rerere.c +++ b/rerere.c @@ -911,9 +911,9 @@ int setup_rerere(struct repository *r, struct string_list *merge_rr, int flags) if (flags & RERERE_READONLY) fd = 0; else - fd = hold_lock_file_for_update(&write_lock, - git_path_merge_rr(r), - LOCK_DIE_ON_ERROR); + fd = repo_hold_lock_file_for_update(r, &write_lock, + git_path_merge_rr(r), + LOCK_DIE_ON_ERROR); read_rr(r, merge_rr); return fd; } diff --git a/serve.c b/serve.c index 49a6e39b1dd25c..2b07d922b3dde1 100644 --- a/serve.c +++ b/serve.c @@ -89,7 +89,7 @@ static void session_id_receive(struct repository *r UNUSED, trace2_data_string("transfer", NULL, "client-sid", client_sid); } -static int object_info_advertise(struct repository *r, struct strbuf *value UNUSED) +static int object_info_advertise(struct repository *r, struct strbuf *value) { if (advertise_object_info == -1 && repo_config_get_bool(r, "transfer.advertiseobjectinfo", @@ -97,6 +97,9 @@ static int object_info_advertise(struct repository *r, struct strbuf *value UNUS /* disabled by default */ advertise_object_info = 0; } + /* Currently only size is supported */ + if (value && advertise_object_info) + strbuf_addstr(value, "size"); return advertise_object_info; } diff --git a/t/lib-cat-file.sh b/t/lib-cat-file.sh new file mode 100644 index 00000000000000..7c2e8770164ce3 --- /dev/null +++ b/t/lib-cat-file.sh @@ -0,0 +1,16 @@ +# Library of git-cat-file related test functions. + +# Print a string without a trailing newline. +echo_without_newline () { + printf '%s' "$*" +} + +# Print a string without newlines and replace them with a NUL character (\0). +echo_without_newline_nul () { + echo_without_newline "$@" | tr '\n' '\0' +} + +# Calculate the length of a string. +strlen () { + echo_without_newline "$1" | wc -c | sed -e 's/^ *//' +} diff --git a/t/meson.build b/t/meson.build index d8161c368b6871..a25f37d2f5ae7d 100644 --- a/t/meson.build +++ b/t/meson.build @@ -171,6 +171,7 @@ integration_tests = [ 't1014-read-tree-confusing.sh', 't1015-read-index-unmerged.sh', 't1016-compatObjectFormat.sh', + 't1017-cat-file-remote-object-info.sh', 't1020-subdirectory.sh', 't1022-read-tree-partial-clone.sh', 't1050-large.sh', diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index aadf22bc2f0bb2..548a61cd9064bc 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -108,6 +108,7 @@ test_perf_on_all () { test_perf_on_all git status test_perf_on_all 'git stash && git stash pop' +test_perf_on_all "git stash push -- $SPARSE_CONE/a && git stash pop" test_perf_on_all 'echo >>new && git stash -u && git stash pop' test_perf_on_all git add -A test_perf_on_all git add . diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 762c77c3510254..0c1dc0fb400d7d 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -3,7 +3,8 @@ test_description='git cat-file' . ./test-lib.sh -. "$TEST_DIRECTORY/lib-loose.sh" +. "$TEST_DIRECTORY"/lib-loose.sh +. "$TEST_DIRECTORY"/lib-cat-file.sh test_cmdmode_usage () { test_expect_code 129 "$@" 2>err && @@ -99,18 +100,6 @@ do ' done -echo_without_newline () { - printf '%s' "$*" -} - -echo_without_newline_nul () { - echo_without_newline "$@" | tr '\n' '\0' -} - -strlen () { - echo_without_newline "$1" | wc -c | sed -e 's/^ *//' -} - run_tests () { type=$1 object_name="$2" diff --git a/t/t1017-cat-file-remote-object-info.sh b/t/t1017-cat-file-remote-object-info.sh new file mode 100755 index 00000000000000..116862f9d0b447 --- /dev/null +++ b/t/t1017-cat-file-remote-object-info.sh @@ -0,0 +1,747 @@ +#!/bin/sh + +test_description='git cat-file --batch-command with remote-object-info command' + +. ./test-lib.sh +. "$TEST_DIRECTORY"/lib-cat-file.sh + +hello_content="Hello World" +hello_size=$(strlen "$hello_content") +hello_oid=$(echo_without_newline "$hello_content" | git hash-object --stdin) +hello_short_oid=$(git rev-parse --short "$hello_oid") + +unstored_content="Hello Git" +unstored_oid=$(echo_without_newline "$unstored_content" | git hash-object --stdin) + +# This is how we get 13: +# 13 = + + + , where +# file mode is 100644, which is 6 characters; +# file name is hello, which is 5 characters +# a space is 1 character and a null is 1 character +tree_size=$(($(test_oid rawsz) + 13)) + +commit_message="Initial commit" + +# This is how we get 137: +# 137 = + + + +# + + +# + + +# + +# +# An easier way to calculate is: 1. use `git cat-file commit | wc -c`, +# to get 177, 2. then deduct 40 hex characters to get 137 +commit_size=$(($(test_oid hexsz) + 137)) + +tag_header_without_oid="type blob +tag hellotag +tagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>" +tag_header_without_timestamp="object $hello_oid +$tag_header_without_oid" +tag_description="This is a tag" +tag_content="$tag_header_without_timestamp 0 +0000 + +$tag_description" + +tag_oid=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w) +tag_size=$(strlen "$tag_content") + +set_transport_variables () { + hello_oid=$(echo_without_newline "$hello_content" | git hash-object --stdin) + tree_oid=$(git -C "$1" write-tree) + commit_oid=$(echo_without_newline "$commit_message" | git -C "$1" commit-tree $tree_oid) + tag_oid=$(echo_without_newline "$tag_content" | git -C "$1" hash-object -t tag --stdin -w) + tag_size=$(strlen "$tag_content") +} + +# This section tests --batch-command with remote-object-info command +# Since "%(objecttype)" is currently not supported by the command remote-object-info , +# the filters are set to "%(objectname) %(objectsize)" in some test cases. + +# Test --batch-command remote-object-info with 'git://' transport with +# transfer.advertiseobjectinfo set to true, i.e. server has object-info capability +. "$TEST_DIRECTORY"/lib-git-daemon.sh +start_git_daemon --export-all +daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent + +test_expect_success 'create repo to be served by git-daemon' ' + git init "$daemon_parent" && + echo_without_newline "$hello_content" > $daemon_parent/hello && + git -C "$daemon_parent" update-index --add hello && + git -C "$daemon_parent" config transfer.advertiseobjectinfo true && + git clone "$GIT_DAEMON_URL/parent" -n "$daemon_parent/daemon_client_empty" +' + +test_expect_success 'batch-command remote-object-info git://' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid + remote-object-info "$GIT_DAEMON_URL/parent" $tree_oid + remote-object-info "$GIT_DAEMON_URL/parent" $commit_oid + remote-object-info "$GIT_DAEMON_URL/parent" $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info git:// multiple sha1 per line' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid $tree_oid $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info git:// default filter' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid $tree_oid + remote-object-info "$GIT_DAEMON_URL/parent" $commit_oid $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info does not change the default format of info' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + local_content="local object" && + local_oid=$(echo_without_newline "$local_content" | git hash-object -w --stdin) && + local_size=$(strlen "$local_content") && + + echo "$local_oid blob $local_size" >expect && + echo "$hello_oid $hello_size" >>expect && + echo "$local_oid blob $local_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + info $local_oid + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid + info $local_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info git://' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid $tree_oid + remote-object-info "$GIT_DAEMON_URL/parent" $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command -Z remote-object-info git:// default filter' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + printf "%s\0" "$hello_oid $hello_size" >expect && + printf "%s\0" "$tree_oid $tree_size" >>expect && + printf "%s\0" "$commit_oid $commit_size" >>expect && + printf "%s\0" "$tag_oid $tag_size" >>expect && + + printf "%s\0" "$hello_oid missing" >>expect && + printf "%s\0" "$tree_oid missing" >>expect && + printf "%s\0" "$commit_oid missing" >>expect && + printf "%s\0" "$tag_oid missing" >>expect && + + batch_input="remote-object-info $GIT_DAEMON_URL/parent $hello_oid $tree_oid +remote-object-info $GIT_DAEMON_URL/parent $commit_oid $tag_oid +info $hello_oid +info $tree_oid +info $commit_oid +info $tag_oid +" && + echo_without_newline_nul "$batch_input" >commands_null_delimited && + + git cat-file --batch-command -Z < commands_null_delimited >actual && + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info does not support short oids' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + test_must_fail git cat-file --batch-command 2>err <<-EOF && + remote-object-info $GIT_DAEMON_URL/parent $hello_short_oid + EOF + test_grep "does not support short oids" err + ) +' + +test_expect_success 'remote-object-info does not die on missing oid like info' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + git cat-file --batch-command >local <<-EOF && + info $unstored_oid + EOF + git cat-file --batch-command >remote <<-EOF && + remote-object-info $GIT_DAEMON_URL/parent $unstored_oid + EOF + test_cmp local remote + ) +' + +# This tests depends on %(objecttype) not being supported yet, once supported +# it needs to be updated. +test_expect_success 'unsupported placeholder on remote returns empty string' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + echo "" >expect && + git cat-file --batch-command="%(objecttype)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'requesting only objectname echoes back' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + echo $hello_oid >expect && + git cat-file --batch-command="%(objectname)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'objectname goes through existence check' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + echo "$unstored_oid missing" >expect && + + git cat-file --batch-command="%(objectname)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $unstored_oid + EOF + + test_cmp expect actual + ) +' + +# Test --batch-command remote-object-info with 'git://' and +# transfer.advertiseobjectinfo set to false, i.e. server does not have object-info capability +test_expect_success 'batch-command remote-object-info git:// fails when transfer.advertiseobjectinfo=false' ' + ( + git -C "$daemon_parent" config transfer.advertiseobjectinfo false && + set_transport_variables "$daemon_parent" && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info $GIT_DAEMON_URL/parent $hello_oid $tree_oid $commit_oid $tag_oid + EOF + test_grep "object-info capability is not enabled on the server" err && + + # revert server state back + git -C "$daemon_parent" config transfer.advertiseobjectinfo true + + ) +' + +stop_git_daemon + +# Test --batch-command remote-object-info with 'file://' transport with +# transfer.advertiseobjectinfo set to true, i.e. server has object-info capability +# shellcheck disable=SC2016 +test_expect_success 'create repo to be served by file:// transport' ' + git init server && + git -C server config protocol.version 2 && + git -C server config transfer.advertiseobjectinfo true && + echo_without_newline "$hello_content" > server/hello && + git -C server update-index --add hello && + git clone -n "file://$(pwd)/server" file_client_empty +' + +test_expect_success 'batch-command remote-object-info file://' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_oid + remote-object-info "file://${server_path}" $tree_oid + remote-object-info "file://${server_path}" $commit_oid + remote-object-info "file://${server_path}" $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info file:// multiple sha1 per line' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_oid $tree_oid $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info file://' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_oid $tree_oid + remote-object-info "file://${server_path}" $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info file:// default filter' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_oid $tree_oid + remote-object-info "file://${server_path}" $commit_oid $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command -Z remote-object-info file:// default filter' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + printf "%s\0" "$hello_oid $hello_size" >expect && + printf "%s\0" "$tree_oid $tree_size" >>expect && + printf "%s\0" "$commit_oid $commit_size" >>expect && + printf "%s\0" "$tag_oid $tag_size" >>expect && + + printf "%s\0" "$hello_oid missing" >>expect && + printf "%s\0" "$tree_oid missing" >>expect && + printf "%s\0" "$commit_oid missing" >>expect && + printf "%s\0" "$tag_oid missing" >>expect && + + batch_input="remote-object-info \"file://${server_path}\" $hello_oid $tree_oid +remote-object-info \"file://${server_path}\" $commit_oid $tag_oid +info $hello_oid +info $tree_oid +info $commit_oid +info $tag_oid +" && + echo_without_newline_nul "$batch_input" >commands_null_delimited && + + git cat-file --batch-command -Z < commands_null_delimited >actual && + test_cmp expect actual + ) +' + +# Test --batch-command remote-object-info with 'file://' and +# transfer.advertiseobjectinfo set to false, i.e. server does not have object-info capability +test_expect_success 'batch-command remote-object-info file:// fails when transfer.advertiseobjectinfo=false' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + git -C "${server_path}" config transfer.advertiseobjectinfo false && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "file://${server_path}" $hello_oid $tree_oid $commit_oid $tag_oid + EOF + test_grep "object-info capability is not enabled on the server" err && + + # revert server state back + git -C "${server_path}" config transfer.advertiseobjectinfo true + ) +' + +# Test --batch-command remote-object-info with 'http://' transport with +# transfer.advertiseobjectinfo set to true, i.e. server has object-info capability + +. "$TEST_DIRECTORY"/lib-httpd.sh +start_httpd + +test_expect_success 'create repo to be served by http:// transport' ' + git init "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config http.receivepack true && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config transfer.advertiseobjectinfo true && + echo_without_newline "$hello_content" > $HTTPD_DOCUMENT_ROOT_PATH/http_parent/hello && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" update-index --add hello && + git clone "$HTTPD_URL/smart/http_parent" -n "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" +' + +test_expect_success 'batch-command remote-object-info http://' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid + remote-object-info "$HTTPD_URL/smart/http_parent" $tree_oid + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_oid + remote-object-info "$HTTPD_URL/smart/http_parent" $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info http:// one line' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid $tree_oid $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info http://' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_oid missing" >>expect && + echo "$tree_oid missing" >>expect && + echo "$commit_oid missing" >>expect && + echo "$tag_oid missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid $tree_oid + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_oid $tag_oid + info $hello_oid + info $tree_oid + info $commit_oid + info $tag_oid + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info http:// default filter' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + echo "$hello_oid $hello_size" >expect && + echo "$tree_oid $tree_size" >>expect && + echo "$commit_oid $commit_size" >>expect && + echo "$tag_oid $tag_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid $tree_oid + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_oid $tag_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command -Z remote-object-info http:// default filter' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + printf "%s\0" "$hello_oid $hello_size" >expect && + printf "%s\0" "$tree_oid $tree_size" >>expect && + printf "%s\0" "$commit_oid $commit_size" >>expect && + printf "%s\0" "$tag_oid $tag_size" >>expect && + + batch_input="remote-object-info $HTTPD_URL/smart/http_parent $hello_oid $tree_oid +remote-object-info $HTTPD_URL/smart/http_parent $commit_oid $tag_oid +" && + echo_without_newline_nul "$batch_input" >commands_null_delimited && + + git cat-file --batch-command -Z < commands_null_delimited >actual && + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info fails on unsupported filter option (objectsize:disk)' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + echo "$hello_oid " >expect && + + git cat-file --batch-command="%(objectname) %(objectsize:disk)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info fails on unsupported filter option (deltabase)' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + echo "" >expect && + + git cat-file --batch-command="%(deltabase)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid + EOF + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info fails on server with legacy protocol' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git -c protocol.version=0 cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid + EOF + test_grep "object-info requires protocol v2" err + ) +' + +test_expect_success 'remote-object-info fails on server with legacy protocol with default filter' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git -c protocol.version=0 cat-file --batch-command 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid + EOF + test_grep "object-info requires protocol v2" err + ) +' + +test_expect_success 'remote-object-info fails on malformed OID' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + malformed_object_id="this_id_is_not_valid" && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $malformed_object_id + EOF + test_grep "not a valid object name '$malformed_object_id'" err + ) +' + +test_expect_success 'remote-object-info fails on malformed OID with default filter' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + malformed_object_id="this_id_is_not_valid" && + + test_must_fail git cat-file --batch-command 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $malformed_object_id + EOF + test_grep "not a valid object name '$malformed_object_id'" err + ) +' + +test_expect_success 'remote-object-info fails on not providing OID' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" + EOF + test_grep "remote-object-info requires objects" err + ) +' + + +# Test --batch-command remote-object-info with 'http://' transport and +# transfer.advertiseobjectinfo set to false, i.e. server does not have object-info capability +test_expect_success 'batch-command remote-object-info http:// fails when transfer.advertiseobjectinfo=false ' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config transfer.advertiseobjectinfo false && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_oid $tree_oid $commit_oid $tag_oid + EOF + test_grep "object-info capability is not enabled on the server" err && + + # revert server state back + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config transfer.advertiseobjectinfo true + ) +' + +# DO NOT add non-httpd-specific tests here, because the last part of this +# test script is only executed when httpd is available and enabled. + +test_done diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index 9814431cd74aff..4140c4d8ef2436 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -1598,6 +1598,61 @@ test_expect_success 'sparse-index is not expanded: stash' ' ensure_not_expanded stash pop ' +test_expect_success 'sparse-index is not expanded: stash in-cone pathspec' ' + init_repos && + + echo unrelated >>sparse-index/deep/e && + echo literal >>sparse-index/deep/a && + ensure_not_expanded stash push -- deep/a && + test_grep ! literal sparse-index/deep/a && + test_grep unrelated sparse-index/deep/e && + ensure_not_expanded stash pop && + test_grep literal sparse-index/deep/a && + + echo prefixed >>sparse-index/deep/a && + ensure_not_expanded -C deep stash push -- a && + test_grep ! prefixed sparse-index/deep/a && + test_grep unrelated sparse-index/deep/e && + ensure_not_expanded stash pop && + test_grep prefixed sparse-index/deep/a && + + echo wildcard >>sparse-index/deep/a && + ensure_not_expanded stash push -- "deep/a*" && + test_grep ! wildcard sparse-index/deep/a && + test_grep unrelated sparse-index/deep/e && + ensure_not_expanded stash pop && + test_grep wildcard sparse-index/deep/a && + + echo pathspec-file >>sparse-index/deep/a && + echo deep/a >pathspec-file && + ensure_not_expanded stash push --pathspec-from-file=../pathspec-file && + test_grep ! pathspec-file sparse-index/deep/a && + test_grep unrelated sparse-index/deep/e && + ensure_not_expanded stash pop && + test_grep pathspec-file sparse-index/deep/a && + + echo multiple-a >>sparse-index/deep/a && + echo multiple-e >>sparse-index/deep/e && + ensure_not_expanded stash push -- deep/a deep/e && + test_grep ! multiple-a sparse-index/deep/a && + test_grep ! multiple-e sparse-index/deep/e && + ensure_not_expanded stash pop && + test_grep multiple-a sparse-index/deep/a && + test_grep multiple-e sparse-index/deep/e && + + echo staged >>sparse-index/deep/a && + git -C sparse-index add deep/a && + ensure_not_expanded stash push --staged -- deep/a && + test_grep ! staged sparse-index/deep/a && + test_grep unrelated sparse-index/deep/e && + ensure_not_expanded stash pop --index && + test_grep staged sparse-index/deep/a && + test_must_fail git -C sparse-index diff --cached --quiet -- deep/a && + + ensure_not_expanded ! stash push -- deep/does-not-exist && + test_grep "did not match any file" sparse-index-error +' + test_expect_success 'describe tested on all' ' init_repos && @@ -2119,6 +2174,13 @@ test_expect_success 'sparse index is not expanded: rm' ' ensure_not_expanded rm -r deep ' +test_expect_success 'sparse index is not expanded: prefixed wildcard pathspec' ' + init_repos && + + ensure_not_expanded -C deep rm --dry-run -- "a*" && + ensure_not_expanded -C deep reset base -- "a*" +' + test_expect_success 'grep with and --cached' ' init_repos && diff --git a/t/t1300-config.sh b/t/t1300-config.sh index b99f782d5dee2d..e3f8064889210a 100755 --- a/t/t1300-config.sh +++ b/t/t1300-config.sh @@ -3002,4 +3002,21 @@ test_expect_success 'writing value with trailing CR not stripped on read' ' test_cmp expect actual ' +test_expect_success 'writing config fails immediately with core.configLockTimeout=0' ' + test_when_finished "rm -f .git/config.lock" && + >.git/config.lock && + test_must_fail git -c core.configLockTimeout=0 config foo.bar baz 2>err && + test_grep "could not lock config file" err +' + +test_expect_success 'writing config retries until lock is released' ' + test_when_finished "rm -f .git/config.lock" && + >.git/config.lock && + { + ( sleep 1 && rm -f .git/config.lock ) & + } && + git -c core.configLockTimeout=5000 config retried.key value && + test "$(git config retried.key)" = value +' + test_done diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh index 1ecbafbee18e03..7f7807eb88e928 100755 --- a/t/t3200-branch.sh +++ b/t/t3200-branch.sh @@ -1075,7 +1075,8 @@ test_expect_success '--set-upstream-to fails on locked config' ' test_when_finished "rm -f .git/config.lock" && >.git/config.lock && git branch locked && - test_must_fail git branch --set-upstream-to locked 2>err && + test_must_fail git -c core.configLockTimeout=0 \ + branch --set-upstream-to locked 2>err && test_grep "could not lock config file .git/config" err ' @@ -1106,7 +1107,8 @@ test_expect_success '--unset-upstream should fail if config is locked' ' test_when_finished "rm -f .git/config.lock" && git branch --set-upstream-to locked && >.git/config.lock && - test_must_fail git branch --unset-upstream 2>err && + test_must_fail git -c core.configLockTimeout=0 \ + branch --unset-upstream 2>err && test_grep "could not lock config file .git/config" err ' diff --git a/t/t4018/swift-actor b/t/t4018/swift-actor new file mode 100644 index 00000000000000..e4852f40a7ebd8 --- /dev/null +++ b/t/t4018/swift-actor @@ -0,0 +1,5 @@ +actor RIGHT { + let a = 1 + // a comment + let b = ChangeMe +} diff --git a/t/t4018/swift-attribute-with-args b/t/t4018/swift-attribute-with-args new file mode 100644 index 00000000000000..22b1ee32f110bd --- /dev/null +++ b/t/t4018/swift-attribute-with-args @@ -0,0 +1,7 @@ +struct View { + @available(iOS 13, *) public func RIGHT() { + let a = 1 + // a comment + print(ChangeMe) + } +} diff --git a/t/t4018/swift-class b/t/t4018/swift-class new file mode 100644 index 00000000000000..c3a93360277f81 --- /dev/null +++ b/t/t4018/swift-class @@ -0,0 +1,5 @@ +class RIGHT { + let a = 1 + // a comment + let b = ChangeMe +} diff --git a/t/t4018/swift-enum b/t/t4018/swift-enum new file mode 100644 index 00000000000000..0a843029930409 --- /dev/null +++ b/t/t4018/swift-enum @@ -0,0 +1,5 @@ +enum RIGHT { + case first + // a comment + case ChangeMe +} diff --git a/t/t4018/swift-extension b/t/t4018/swift-extension new file mode 100644 index 00000000000000..cbc18ab6ef42c8 --- /dev/null +++ b/t/t4018/swift-extension @@ -0,0 +1,5 @@ +extension RIGHT { + static let a = 1 + // a comment + static let b = ChangeMe +} diff --git a/t/t4018/swift-failable-init b/t/t4018/swift-failable-init new file mode 100644 index 00000000000000..4bbd6217c9147a --- /dev/null +++ b/t/t4018/swift-failable-init @@ -0,0 +1,7 @@ +class Bar { + init?(RIGHT: Int) { + let x = 0 + // a comment + print(ChangeMe) + } +} diff --git a/t/t4018/swift-func b/t/t4018/swift-func new file mode 100644 index 00000000000000..1fecae0911d411 --- /dev/null +++ b/t/t4018/swift-func @@ -0,0 +1,5 @@ +func RIGHT(x: Int) -> Int { + let y = x + // a comment + return ChangeMe +} diff --git a/t/t4018/swift-generic-subscript b/t/t4018/swift-generic-subscript new file mode 100644 index 00000000000000..423cb589412ea5 --- /dev/null +++ b/t/t4018/swift-generic-subscript @@ -0,0 +1,7 @@ +struct Container { + subscript(index: Int) -> Int { + let a = 0 + // a comment + return ChangeMe + } +} diff --git a/t/t4018/swift-init b/t/t4018/swift-init new file mode 100644 index 00000000000000..dc7a298f3806b5 --- /dev/null +++ b/t/t4018/swift-init @@ -0,0 +1,7 @@ +class Foo { + init(RIGHT: Int) { + let x = 0 + // a comment + print(ChangeMe) + } +} diff --git a/t/t4018/swift-inline-attribute b/t/t4018/swift-inline-attribute new file mode 100644 index 00000000000000..2374c4b6036efa --- /dev/null +++ b/t/t4018/swift-inline-attribute @@ -0,0 +1,7 @@ +class Service { + @objc func RIGHT() { + let path = "/api" + // a comment + log(ChangeMe) + } +} diff --git a/t/t4018/swift-modifiers b/t/t4018/swift-modifiers new file mode 100644 index 00000000000000..9d80685a786bea --- /dev/null +++ b/t/t4018/swift-modifiers @@ -0,0 +1,4 @@ +public static func RIGHT() -> Int { + // a comment + return ChangeMe +} diff --git a/t/t4018/swift-protocol b/t/t4018/swift-protocol new file mode 100644 index 00000000000000..07c39ec2a3ca89 --- /dev/null +++ b/t/t4018/swift-protocol @@ -0,0 +1,5 @@ +protocol RIGHT { + var first: Int { get } + // a comment + var second: ChangeMe { get } +} diff --git a/t/t4018/swift-struct b/t/t4018/swift-struct new file mode 100644 index 00000000000000..e399ed77597656 --- /dev/null +++ b/t/t4018/swift-struct @@ -0,0 +1,5 @@ +struct RIGHT { + let a = 1 + // a comment + let b = ChangeMe +} diff --git a/t/t5505-remote.sh b/t/t5505-remote.sh index 6f5e86dedeb209..81cc251d3bc3a1 100755 --- a/t/t5505-remote.sh +++ b/t/t5505-remote.sh @@ -1327,7 +1327,8 @@ test_expect_success 'remote set-url with locked config' ' test_when_finished "rm -f .git/config.lock" && git config --get-all remote.someremote.url >expect && >.git/config.lock && - test_must_fail git remote set-url someremote baz && + test_must_fail git -c core.configLockTimeout=0 \ + remote set-url someremote baz && git config --get-all remote.someremote.url >actual && cmp expect actual ' diff --git a/t/t5701-git-serve.sh b/t/t5701-git-serve.sh index 108eb30945247b..9a575aa098afd3 100755 --- a/t/t5701-git-serve.sh +++ b/t/t5701-git-serve.sh @@ -7,6 +7,8 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +unknown_oid=$(printf "test" | git hash-object --stdin) + test_expect_success 'setup to generate files with expected content' ' printf "agent=git/%s" "$(git version | cut -d" " -f3)" >agent_capability && @@ -364,6 +366,67 @@ test_expect_success 'basics of object-info' ' test_cmp expect actual ' +test_expect_success 'bare OID request' ' + test_config transfer.advertiseObjectInfo true && + + test-tool pkt-line pack >in <<-EOF && + command=object-info + object-format=$(test_oid algo) + 0001 + oid $(git rev-parse two:two.t) + 0000 + EOF + + cat >expect <<-EOF && + $(git rev-parse two:two.t) + 0000 + EOF + + test-tool serve-v2 --stateless-rpc out && + test-tool pkt-line unpack actual && + test_cmp expect actual +' + +test_expect_success 'object-info with bare unrecognized OID' ' + test_config transfer.advertiseObjectInfo true && + + test-tool pkt-line pack >in <<-EOF && + command=object-info + object-format=$(test_oid algo) + 0001 + oid $unknown_oid + 0000 + EOF + + printf "%s \n" "$unknown_oid" >expect && + printf "0000\n" >>expect && + + test-tool serve-v2 --stateless-rpc out && + test-tool pkt-line unpack actual && + test_cmp expect actual +' + +test_expect_success 'object-info with size for unrecognized OID' ' + test_config transfer.advertiseObjectInfo true && + + test-tool pkt-line pack >in <<-EOF && + command=object-info + object-format=$(test_oid algo) + 0001 + size + oid $unknown_oid + 0000 + EOF + + printf "size\n" >expect && + printf "%s \n" "$unknown_oid" >>expect && + printf "0000\n" >>expect && + + test-tool serve-v2 --stateless-rpc out && + test-tool pkt-line unpack actual && + test_cmp expect actual +' + test_expect_success 'test capability advertisement with uploadpack.advertiseBundleURIs' ' test_config uploadpack.advertiseBundleURIs true && diff --git a/t/t9502-gitweb-standalone-parse-output.sh b/t/t9502-gitweb-standalone-parse-output.sh index 9587ce212ac798..00cf077e52f2a0 100755 --- a/t/t9502-gitweb-standalone-parse-output.sh +++ b/t/t9502-gitweb-standalone-parse-output.sh @@ -115,6 +115,19 @@ test_expect_success 'snapshot: hierarchical branch name (xx/test)' ' ' test_debug 'cat gitweb.headers' +test_expect_success 'commitdiff: index line shortens hashes with mode' ' + old_blob=$(git rev-parse HEAD:foo) && + old_short=$(git rev-parse --short=7 HEAD:foo) && + echo changed >foo && + git commit -am "change foo" && + new_blob=$(git rev-parse HEAD:foo) && + new_short=$(git rev-parse --short=7 HEAD:foo) && + gitweb_run "p=.git;a=commitdiff;h=HEAD" && + test_grep ">${old_short}\\.\\.]*>${new_short} 100644 (file)" \ + gitweb.body && + test_grep ! "index ${old_blob}\\.\\.${new_blob} 100644" gitweb.body +' + # ---------------------------------------------------------------------- # forks of projects diff --git a/t/unit-tests/u-odb-inmemory.c b/t/unit-tests/u-odb-inmemory.c index 6844bfc37ccfdc..ddf2db5c811fb8 100644 --- a/t/unit-tests/u-odb-inmemory.c +++ b/t/unit-tests/u-odb-inmemory.c @@ -1,5 +1,6 @@ #include "unit-test.h" #include "hex.h" +#include "object-file.h" #include "odb/source-inmemory.h" #include "odb/streaming.h" #include "oidset.h" @@ -36,6 +37,17 @@ static void cl_assert_object_info(struct odb_source_inmemory *source, free(actual_content); } +static void cl_assert_write_object(struct odb_source_inmemory *source, + const char *content, + enum object_type type, + struct object_id *oid) +{ + size_t content_len = strlen(content); + hash_object_file(repo.hash_algo, content, content_len, type, oid); + cl_must_pass(odb_source_write_object(&source->base, content, content_len, + type, oid, NULL, NULL, 0)); +} + void test_odb_inmemory__initialize(void) { odb = odb_new(&repo, "", ""); @@ -78,8 +90,7 @@ void test_odb_inmemory__read_written_object(void) const char data[] = "foobar"; struct object_id written_oid; - cl_must_pass(odb_source_write_object(&source->base, data, strlen(data), - OBJ_BLOB, &written_oid, NULL, 0)); + cl_assert_write_object(source, data, OBJ_BLOB, &written_oid); cl_assert_equal_s(oid_to_hex(&written_oid), FOOBAR_OID); cl_assert_object_info(source, &written_oid, OBJ_BLOB, "foobar"); @@ -94,8 +105,7 @@ void test_odb_inmemory__read_stream_object(void) const char data[] = "foobar"; char buf[3] = { 0 }; - cl_must_pass(odb_source_write_object(&source->base, data, strlen(data), - OBJ_BLOB, &written_oid, NULL, 0)); + cl_assert_write_object(source, data, OBJ_BLOB, &written_oid); cl_must_pass(odb_source_read_object_stream(&stream, &source->base, &written_oid)); @@ -141,8 +151,7 @@ void test_odb_inmemory__for_each_object(void) strbuf_reset(&buf); strbuf_addf(&buf, "%d", i); - cl_must_pass(odb_source_write_object(&source->base, buf.buf, buf.len, - OBJ_BLOB, &written_oid, NULL, 0)); + cl_assert_write_object(source, buf.buf, OBJ_BLOB, &written_oid); cl_must_pass(oidset_insert(&expected_oids, &written_oid)); } @@ -174,12 +183,9 @@ void test_odb_inmemory__for_each_object_can_abort_iteration(void) struct object_id written_oid; unsigned counter = 0; - cl_must_pass(odb_source_write_object(&source->base, "1", 1, - OBJ_BLOB, &written_oid, NULL, 0)); - cl_must_pass(odb_source_write_object(&source->base, "2", 1, - OBJ_BLOB, &written_oid, NULL, 0)); - cl_must_pass(odb_source_write_object(&source->base, "3", 1, - OBJ_BLOB, &written_oid, NULL, 0)); + cl_assert_write_object(source, "1", OBJ_BLOB, &written_oid); + cl_assert_write_object(source, "2", OBJ_BLOB, &written_oid); + cl_assert_write_object(source, "3", OBJ_BLOB, &written_oid); cl_assert_equal_i(odb_source_for_each_object(&source->base, NULL, abort_after_two_objects, @@ -199,12 +205,9 @@ void test_odb_inmemory__count_objects(void) cl_must_pass(odb_source_count_objects(&source->base, 0, &count)); cl_assert_equal_u(count, 0); - cl_must_pass(odb_source_write_object(&source->base, "1", 1, - OBJ_BLOB, &written_oid, NULL, 0)); - cl_must_pass(odb_source_write_object(&source->base, "2", 1, - OBJ_BLOB, &written_oid, NULL, 0)); - cl_must_pass(odb_source_write_object(&source->base, "3", 1, - OBJ_BLOB, &written_oid, NULL, 0)); + cl_assert_write_object(source, "1", OBJ_BLOB, &written_oid); + cl_assert_write_object(source, "2", OBJ_BLOB, &written_oid); + cl_assert_write_object(source, "3", OBJ_BLOB, &written_oid); cl_must_pass(odb_source_count_objects(&source->base, 0, &count)); cl_assert_equal_u(count, 3); @@ -228,8 +231,7 @@ void test_odb_inmemory__find_abbrev_len(void) * * With only one blob written we expect a length of 4. */ - cl_must_pass(odb_source_write_object(&source->base, "368317", strlen("368317"), - OBJ_BLOB, &oid1, NULL, 0)); + cl_assert_write_object(source, "368317", OBJ_BLOB, &oid1); cl_must_pass(odb_source_find_abbrev_len(&source->base, &oid1, 4, &abbrev_len)); cl_assert_equal_u(abbrev_len, 4); @@ -238,8 +240,7 @@ void test_odb_inmemory__find_abbrev_len(void) * With both objects present, the shared 10-character prefix means we * need at least 11 characters to uniquely identify either object. */ - cl_must_pass(odb_source_write_object(&source->base, "514796", strlen("514796"), - OBJ_BLOB, &oid2, NULL, 0)); + cl_assert_write_object(source, "514796", OBJ_BLOB, &oid2); cl_must_pass(odb_source_find_abbrev_len(&source->base, &oid1, 4, &abbrev_len)); cl_assert_equal_u(abbrev_len, 11); @@ -255,13 +256,11 @@ void test_odb_inmemory__freshen_object(void) const char *end; cl_must_pass(parse_oid_hex_algop(RANDOM_OID, &oid, &end, repo.hash_algo)); - cl_assert_equal_i(odb_source_freshen_object(&source->base, &oid), 0); + cl_assert_equal_i(odb_source_freshen_object(&source->base, &oid, NULL), 0); - cl_must_pass(odb_source_write_object(&source->base, "foobar", - strlen("foobar"), OBJ_BLOB, - &written_oid, NULL, 0)); + cl_assert_write_object(source, "foobar", OBJ_BLOB, &written_oid); cl_assert_equal_i(odb_source_freshen_object(&source->base, - &written_oid), 1); + &written_oid, NULL), 1); odb_source_free(&source->base); } diff --git a/tempfile.c b/tempfile.c index f0fdf582794ba5..dc9ca4e6459c64 100644 --- a/tempfile.c +++ b/tempfile.c @@ -42,8 +42,6 @@ * file created by its parent. */ -#define USE_THE_REPOSITORY_VARIABLE - #include "git-compat-util.h" #include "abspath.h" #include "path.h" @@ -134,7 +132,8 @@ static void deactivate_tempfile(struct tempfile *tempfile) } /* Make sure errno contains a meaningful value on error */ -struct tempfile *create_tempfile_mode(const char *path, int mode) +struct tempfile *repo_create_tempfile_mode(struct repository *r, + const char *path, int mode) { struct tempfile *tempfile = new_tempfile(); @@ -150,7 +149,7 @@ struct tempfile *create_tempfile_mode(const char *path, int mode) return NULL; } activate_tempfile(tempfile); - if (adjust_shared_perm(the_repository, tempfile->filename.buf)) { + if (adjust_shared_perm(r, tempfile->filename.buf)) { int save_errno = errno; error("cannot fix permission bits on %s", tempfile->filename.buf); delete_tempfile(&tempfile); diff --git a/tempfile.h b/tempfile.h index 2227a095fd4289..f571f3c609c04a 100644 --- a/tempfile.h +++ b/tempfile.h @@ -4,6 +4,8 @@ #include "list.h" #include "strbuf.h" +struct repository; + /* * Handle temporary files. * @@ -92,11 +94,13 @@ struct tempfile { * `core.sharedRepository`, so it is not guaranteed to have the given * mode. */ -struct tempfile *create_tempfile_mode(const char *path, int mode); +struct tempfile *repo_create_tempfile_mode(struct repository *r, + const char *path, int mode); -static inline struct tempfile *create_tempfile(const char *path) +static inline struct tempfile *repo_create_tempfile(struct repository *r, + const char *path) { - return create_tempfile_mode(path, 0666); + return repo_create_tempfile_mode(r, path, 0666); } /* diff --git a/transport-helper.c b/transport-helper.c index 8a25707b038aac..b109fbd8c3ae63 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -266,9 +266,9 @@ static int disconnect_helper(struct transport *transport) close(data->helper->out); fclose(data->out); res = finish_command(data->helper); - FREE_AND_NULL(data->name); FREE_AND_NULL(data->helper); } + FREE_AND_NULL(data->name); return res; } @@ -784,6 +784,15 @@ static int fetch_refs(struct transport *transport, return -1; } +static int fetch_object_info_helper(struct transport *transport) +{ + get_helper(transport); + if (process_connect(transport, 0)) + return transport->vtable->fetch_object_info(transport); + + die(_("object-info requires protocol v2")); +} + struct push_update_ref_state { struct ref *hint; struct ref_push_report *report; @@ -1330,6 +1339,7 @@ static struct transport_vtable vtable = { .get_refs_list = get_refs_list, .get_bundle_uri = get_bundle_uri, .fetch_refs = fetch_refs, + .fetch_object_info = fetch_object_info_helper, .push_refs = push_refs, .connect = connect_helper, .disconnect = release_helper diff --git a/transport-internal.h b/transport-internal.h index 051f3ab0dc95ec..60db0bedcdb9ae 100644 --- a/transport-internal.h +++ b/transport-internal.h @@ -45,6 +45,14 @@ struct transport_vtable { **/ int (*fetch_refs)(struct transport *transport, int refs_nr, struct ref **refs); + /* + * Fetch object info (only size currently) from remote without + * downloading the objects. + * + * Uses object-info capability of v2 protocol. + */ + int (*fetch_object_info)(struct transport *transport); + /** * Push the objects and refs. Send the necessary objects, and * then, for any refs where peer_ref is set and diff --git a/transport.c b/transport.c index fc144f0aedabd9..f0a6a455479800 100644 --- a/transport.c +++ b/transport.c @@ -9,6 +9,7 @@ #include "hook.h" #include "pkt-line.h" #include "fetch-pack.h" +#include "fetch-object-info.h" #include "remote.h" #include "connect.h" #include "send-pack.h" @@ -432,6 +433,47 @@ static int get_bundle_uri(struct transport *transport) transport->bundles, stateless_rpc); } +static int fetch_object_info_via_pack(struct transport *transport) +{ + int ret = 0; + struct git_transport_data *data = transport->data; + struct packet_reader reader; + struct object_info_args args = { 0 }; + + args.server_options = transport->server_options; + args.oids = transport->smart_options->object_info_oids; + args.object_info_options = transport->smart_options->object_info_options; + + connect_setup(transport, 0); + packet_reader_init(&reader, data->fd[0], NULL, 0, + PACKET_READ_CHOMP_NEWLINE | + PACKET_READ_GENTLE_ON_EOF | + PACKET_READ_DIE_ON_ERR_PACKET); + + data->version = discover_version(&reader); + transport->hash_algo = reader.hash_algo; + + ret = fetch_object_info(data->version, &args, &reader, + data->options.object_info_data, + transport->stateless_rpc, data->fd[1]); + + close(data->fd[0]); + if (data->fd[1] >= 0) + close(data->fd[1]); + if (finish_connect(data->conn)) + ret = -1; + data->conn = NULL; + + return ret; +} + +int transport_fetch_object_info(struct transport *transport) +{ + if (!transport->vtable->fetch_object_info) + die(_("remote does not support object-info")); + return transport->vtable->fetch_object_info(transport); +} + static int fetch_refs_via_pack(struct transport *transport, int nr_heads, struct ref **to_fetch) { @@ -1004,6 +1046,7 @@ static struct transport_vtable taken_over_vtable = { .get_refs_list = get_refs_via_connect, .get_bundle_uri = get_bundle_uri, .fetch_refs = fetch_refs_via_pack, + .fetch_object_info = fetch_object_info_via_pack, .push_refs = git_transport_push, .disconnect = disconnect_git }; @@ -1169,6 +1212,7 @@ static struct transport_vtable builtin_smart_vtable = { .get_refs_list = get_refs_via_connect, .get_bundle_uri = get_bundle_uri, .fetch_refs = fetch_refs_via_pack, + .fetch_object_info = fetch_object_info_via_pack, .push_refs = git_transport_push, .connect = connect_git, .disconnect = disconnect_git diff --git a/transport.h b/transport.h index 7e5867cffaaa4a..a7869d18e020fb 100644 --- a/transport.h +++ b/transport.h @@ -55,6 +55,10 @@ struct git_transport_options { * common commits to this oidset instead of fetching any packfiles. */ struct oidset *acked_commits; + + struct oid_array *object_info_oids; + struct object_info *object_info_data; + struct string_list *object_info_options; }; enum transport_family { @@ -309,6 +313,11 @@ int transport_get_remote_bundle_uri(struct transport *transport); const struct git_hash_algo *transport_get_hash_algo(struct transport *transport); int transport_fetch_refs(struct transport *transport, struct ref *refs); +/* + * Fetch the object info from remote + */ +int transport_fetch_object_info(struct transport *transport); + /* * If this flag is set, unlocking will avoid to call non-async-signal-safe * functions. This will necessarily leave behind some data structures which diff --git a/userdiff.c b/userdiff.c index b5412e6bc3ecd3..7129bf148266b7 100644 --- a/userdiff.c +++ b/userdiff.c @@ -362,6 +362,16 @@ PATTERNS("scheme", "\\|([^|\\\\]|\\\\.)*\\|" /* All other words should be delimited by spaces or parentheses. */ "|([^][)(}{ \t])+"), +PATTERNS("swift", + "^[ \t]*((@[A-Za-z_][A-Za-z0-9_]*(\\([^()]*\\))?[ \t]+)*([a-z]+[ \t]+)*(func|init|deinit|subscript|class|struct|enum|protocol|extension|actor)[ \t(?!<].*)$", + /* -- */ + "[a-zA-Z_][a-zA-Z0-9_]*" + /* hexadecimal, octal, and binary literals */ + "|0[xX][0-9a-fA-F_]+|0[oO][0-7_]+|0[bB][01_]+" + /* integers and floating-point numbers */ + "|[0-9][0-9_]*([.][0-9_]+)?([eE][-+]?[0-9]+)?" + /* unary and binary operators */ + "|[-+*/%<>=!&|^~?]=|&&|\\|\\||<<=?|>>=?|\\?\\?|\\.\\.[.<]|->"), PATTERNS("tex", "^(\\\\((sub)*section|chapter|part)\\*{0,1}\\{.*)$", "\\\\[a-zA-Z@]+|\\\\.|([a-zA-Z0-9]|[^\x01-\x7f])+"), { .name = "default", .binary = -1 },