Skip to content

Commit 8e763a7

Browse files
authored
Merge pull request #2308 from gitpython-developers/tuning
fix: preserve legacy path bytes and repair tag, tree and ref edge cases
2 parents 3513562 + f5ca4a5 commit 8e763a7

55 files changed

Lines changed: 1048 additions & 352 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CONTRIBUTING.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,12 @@ Attributing AI assistance in commit metadata, for example with a `Co-authored-by
4848
trailer, is welcome but not required. Code is reviewed the same way regardless of its
4949
origin.
5050

51+
## Documenting subclass interfaces
52+
53+
Use Sphinx's `:meta public:` field in docstrings of underscore-prefixed methods
54+
that form a supported interface for subclasses. This includes their documentation
55+
in the API reference without exposing unrelated private implementation details.
56+
5157
## Temporary test directories
5258

5359
Tests require Python 3.11+, Git 2.52+, Bash, and the `test` extra, which installs

‎README.md‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,9 @@ bindings to execute supported operations through Gitoxide while exposing GitPyth
1111
high-level repository, commit, tree and index interfaces.
1212

1313
Supported object reads, tree traversal, reference/index queries and commit history
14-
can run without launching Git subprocesses. GitPython also retains and reuses a Gix
15-
repository handle across operations, reducing repeated initialization work. These
16-
changes reduce subprocess overhead, particularly for repeated repository queries;
17-
the [backend benchmarks](test/performance/README.md) measure results and CLI launch
18-
counts for both backends.
14+
can run without launching Git subprocesses. Each native operation opens a fresh
15+
Gix repository handle. The [backend benchmarks](test/performance/README.md) measure
16+
results and CLI launch counts for both backends.
1917

2018
### Activate the Gix backend
2119

@@ -161,8 +159,12 @@ GitPython is not suited for long-running processes (like daemons) as it tends to
161159
leak system resources. It was written in a time where destructors (as implemented
162160
in the `__del__` method) still ran deterministically.
163161

164-
In case you still want to use it in such a context, you will want to search the
165-
codebase for `__del__` implementations and call these yourself when you see fit.
162+
Use `Repo` as a context manager (`with Repo(path) as repo:`), or call
163+
`repo.close()` explicitly when finished, to release its native repository handle
164+
and persistent Git object readers. Close every repository instance and any
165+
separately opened readers before removing its directory. On Windows,
166+
`git.util.rmtree(path)` also handles Git's read-only files, which plain
167+
`shutil.rmtree()` may be unable to remove even after handles are closed.
166168

167169
Another way assure proper cleanup of resources is to factor out GitPython into a
168170
separate process which can be dropped periodically.

‎doc/gix-backend.md‎

Lines changed: 99 additions & 72 deletions
Large diffs are not rendered by default.

‎doc/source/changes.rst‎

Lines changed: 45 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -47,34 +47,43 @@ Git executable and safety
4747
* Known command outcomes retain established GitPython/configparser exceptions
4848
where distinguishable. Other failures raise ``GitCommandError`` with Git's
4949
exit status and diagnostic output; exact error text may differ.
50+
Text and streamed command output preserve undecodable bytes with
51+
``surrogateescape``, including fetch/push results, clone errors and timeout
52+
diagnostics containing byte names.
5053

5154
API changes
5255
-----------
5356

57+
* Packed tag names containing non-UTF-8 bytes are represented losslessly as
58+
surrogate-escaped strings. ``safe_encode()`` recovers their original bytes,
59+
and Gix reference validation and resolution receive those bytes directly.
60+
Reference enumeration, fetch results, stale references and remote URLs preserve
61+
Unicode line separators within names instead of treating them as record breaks.
62+
* Raw diffs preserve non-UTF-8 filename bytes in ``a_rawpath``, ``b_rawpath``,
63+
and raw rename fields. Staged-file diffs no longer fail while decoding legacy
64+
filenames; blob contents remain byte streams regardless of their encoding.
65+
Text paths and rename properties use ``surrogateescape`` instead of replacing
66+
undecodable bytes. Diff blob paths can be used for tree lookup, and renames
67+
between distinct byte names are detected correctly. Patch rename fields
68+
contain unquoted filename bytes.
69+
* Tree serialization now delegates construction to Git or the supported Gix
70+
backend. ``Tree.cache`` and ``TreeModifier`` edit one tree's in-memory entries;
71+
``set_done()`` orders entries by filename bytes, including undecodable names.
72+
Object IDs and parent trees do not change automatically. Prefer
73+
``IndexFile.write_tree()`` for changes to staged paths.
5474
* ``Commit.stats`` preserves Unicode line separators in filenames when
5575
``core.quotePath=false``. Git's LF-delimited output remains intact for root,
56-
subsequent and empty commits, including the Gix backend's existing filename
57-
quoting fallback.
58-
* ``GitConfigParser[section] = values`` again replaces a nonempty section,
59-
restoring Kas's Git identity cleanup. Git performs each mutation and locks
60-
it independently; replacement is not one atomic transaction. Empty sections
61-
and ``add_section()`` remain unsupported, including Kas's SSH key setup.
62-
* Explicit ``from git import Union`` and ``Optional`` imports remain available
63-
for Demisto SDK's validator registration and MkDocs RSS Plugin imports,
64-
retaining the historical ``typing`` aliases.
65-
* Editable/source imports expose the release version in ``git.__version__``,
66-
matching built distributions. Salt's GitPython provider checks can parse the
67-
version instead of rejecting the unstamped ``"git"`` build placeholder.
76+
subsequent and empty commits, including the Gix backend's filename
77+
quoting fallback. File keys contain the actual names, unquoting Git's C-style
78+
escapes and preserving undecodable bytes with ``surrogateescape``.
79+
* Imports from a source checkout now read ``git.__version__`` from ``VERSION``.
80+
Built wheels and source distributions continue to carry their stamped release
81+
version.
6882
* ``Repo`` construction validates repository configuration through Git/Gix.
6983
A subclass override of ``config_reader()`` no longer controls discovery or
7084
suppresses its include graph. Malformed configuration and invalid repository
71-
extensions can fail before downstream custom validation runs. sbxloop's
72-
metadata-reader subclass exposes this remaining compatibility break; callers
73-
must prepare their trusted metadata view before opening it with ``Repo``.
74-
* Revision root trees such as ``Repo.rev_parse("HEAD:")`` retain their empty
75-
path and can be iterated. Missing paths within a valid tree revision retain
76-
``Tree.join()``'s ``KeyError`` and missing-component message, restoring MCP
77-
Server Git's directory display and path-error reporting.
85+
extensions can fail before downstream custom validation runs. Callers using
86+
custom metadata views must prepare them before opening with ``Repo``.
7887
* ``Head.tracking_branch()`` returns a local ``Head`` when Git configures the
7988
current repository (``branch.<name>.remote=.``) as the upstream, instead of
8089
constructing an invalid ``refs/remotes/./...`` reference. Git resolves the
@@ -122,17 +131,23 @@ API changes
122131
including updates when ``logmsg`` is omitted.
123132
* ``GitConfigParser`` uses Git's configuration syntax and canonical key spelling,
124133
and no longer subclasses ``configparser.RawConfigParser``.
134+
Subsection names and values preserve undecodable bytes with ``surrogateescape``.
125135
Common getters, typed values, duplicate values, file/stream inputs and mutation
126-
methods remain. Invalid Git syntax previously accepted by Python is rejected.
127-
Empty-section creation and valueless writes are removed; valueless reads remain.
128-
Relative includes from streams are rejected by Git. Locks cover each native
129-
mutation rather than the writer object's lifetime.
136+
methods remain, including ``GitConfigParser[section] = values`` for replacing
137+
nonempty sections. Invalid Git syntax previously accepted by Python is rejected.
138+
Empty-section creation, including ``add_section()``, and valueless writes are
139+
removed; valueless reads remain. Relative includes from streams are rejected
140+
by Git. Locks cover each native mutation rather than the writer object's
141+
lifetime; replacing a section is not one atomic transaction.
130142
* ``Submodule.rename()`` is removed. Moving a submodule preserves its logical name,
131143
matching Git. Normal removal retains Git's recoverable submodule metadata.
132144
Re-adding with ``no_checkout=True`` can reuse that metadata without changing refs;
133145
incompatible URLs, branches, and clone-only options are rejected before mutation.
134146
Fetch/pull results are derived from command output rather than ``FETCH_HEAD``
135-
file parsing. Revision strings follow Git's native revision grammar.
147+
file parsing. Revision strings follow Git's native revision grammar. Root trees
148+
such as ``Repo.rev_parse("HEAD:")`` have an empty path and can be iterated.
149+
Missing paths within a valid tree revision raise ``Tree.join()``'s ``KeyError``
150+
with its missing-component message.
136151

137152
* ``IndexFile.entries`` returns an immutable snapshot keyed by ``(path, stage)``
138153
for compatibility with existing readers. Each access queries the current
@@ -156,8 +171,8 @@ API changes
156171
* Commit/tag metadata uses structured Gix decoders when the Gix backend supports
157172
the repository. The CLI backend retains raw decoding because Git has no faithful
158173
formatted query for all commit headers or arbitrary tag objects. ``gpgsig``
159-
remains readable. The small ``Actor.from_string()`` and
160-
``parse_actor_and_date()`` identity helpers remain supported.
174+
remains readable. ``Actor.from_string()`` and ``parse_actor_and_date()`` provide
175+
identity parsing.
161176
* ``parse_date()`` delegates text syntax and timezone interpretation to Git;
162177
aware datetimes retain direct conversion and invalid inputs raise ``ValueError``.
163178
ISO/RFC dates now apply their timezone to the UTC timestamp, correcting the
@@ -172,9 +187,10 @@ API changes
172187
``Actor.from_string``), ``git.util.Iterable`` (use ``IterableObj``),
173188
``git.compat.is_win/is_posix/is_darwin`` (use ``os.name``/``sys.platform``), and
174189
``git.types.Lit_commit_ish`` (use ``Literal["commit", "tag"]`` or
175-
``GitObjectTypeString``). Top-level typing exports and private module aliases
176-
are removed; import from ``typing`` or the owning module. ``git.util`` now
177-
exposes the actual utility module. Abstract ``Traversable.traverse`` and
190+
``GitObjectTypeString``). Top-level typing exports other than explicit
191+
``Union`` and ``Optional`` imports are removed, along with private module
192+
aliases; import from ``typing`` or the owning module. ``git.util`` now exposes
193+
the actual utility module. Abstract ``Traversable.traverse`` and
178194
``list_traverse`` raise ``NotImplementedError``; use concrete implementations.
179195
* Paths and URLs no longer expand ``$VAR``/``%VAR%`` automatically. ``Repo``
180196
still accepts ``expand_vars=False`` for callers that explicitly select literal
@@ -1135,7 +1151,6 @@ Please note that due to breaking changes, we have to increase the major version.
11351151
* CRITICAL: fixed incorrect `Commit` object serialization when authored or commit date had timezones which were not
11361152
divisiblej by 3600 seconds. This would happen if the timezone was something like `+0530` for instance.
11371153
* A list of all additional fixes can be found `on GitHub <https://github.com/gitpython-developers/GitPython/issues?q=milestone%3A%22v1.0.2+-+Fixes%22+is%3Aclosed>`__
1138-
* CRITICAL: `Tree.cache` was removed without replacement. It is technically impossible to change individual trees and expect their serialization results to be consistent with what *git* expects. Instead, use the `IndexFile` facilities to adjust the content of the staging area, and write it out to the respective tree objects using `IndexFile.write_tree()` instead.
11391154

11401155
1.0.1 - Fixes
11411156
=============

‎doc/source/intro.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Overview / Install
66

77
GitPython is a python library used to interact with git repositories, high-level like git-porcelain, or low-level like git-plumbing.
88

9-
It provides Python objects for repository data and delegates Git operations to the Git executable. The default backend supports both SHA-1 and SHA-256 object IDs and both files and reftable reference storage. The legacy pure-Python GitDB backend is removed.
9+
It provides Python objects for repository data and delegates Git operations to the Git executable. The default backend supports both SHA-1 and SHA-256 object IDs and both files and reftable reference storage. ``GitDB`` is an alias for ``GitCmdObjectDB``; the pure-Python storage implementation is unavailable.
1010

1111
The object database implementation is optimized for handling large quantities of objects and large datasets, which is achieved by using low-level structures and data streaming.
1212

‎doc/source/reference.rst‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,6 @@ Objects.Util
9191
.. automodule:: git.objects.util
9292
:members:
9393
:undoc-members:
94-
:private-members: _get_intermediate_items, _list_traverse, _traverse
9594
:special-members:
9695

9796
Index.Base

‎doc/source/tutorial.rst‎

Lines changed: 30 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,32 @@ Archive the repository contents to a tar file.
6363
:start-after: # [6-test_init_repo_object]
6464
:end-before: # ![6-test_init_repo_object]
6565

66+
Closing a repository
67+
====================
68+
69+
Use a :class:`~git.repo.base.Repo` context manager, or call
70+
:meth:`~git.repo.base.Repo.close` explicitly when finished. Exiting the context
71+
releases its native handle and persistent Git object readers, including when an
72+
exception is raised.
73+
74+
.. literalinclude:: ../../test/test_docs.py
75+
:language: python
76+
:dedent: 8
77+
:start-after: # [close-repo-context]
78+
:end-before: # ![close-repo-context]
79+
80+
Close every repository instance and any separately opened readers before deleting
81+
the directory. Windows prevents deletion while files remain open; Git's read-only
82+
object files can also make plain ``shutil.rmtree()`` fail. The existing
83+
:func:`git.util.rmtree` helper handles those read-only files while still reporting
84+
deletion failures:
85+
86+
.. literalinclude:: ../../test/test_docs.py
87+
:language: python
88+
:dedent: 8
89+
:start-after: # [remove-repo-directory]
90+
:end-before: # ![remove-repo-directory]
91+
6692
Advanced Repo Usage
6793
===================
6894

@@ -522,9 +548,10 @@ resolves abbreviated object IDs through persistent ``git cat-file`` processes::
522548
# Equivalent explicit selection:
523549
repo = Repo("path/to/repo", odbt=GitCmdObjectDB)
524550

525-
Only ``GitCmdObjectDB`` and subclasses can be selected through ``odbt``. The
526-
legacy ``GitDB`` backend is removed. The ``gitdb`` package remains a dependency
527-
for shared types and utilities.
551+
Only ``GitCmdObjectDB`` and subclasses can be selected through ``odbt``.
552+
``GitDB`` is an alias for ``GitCmdObjectDB``; passing ``odbt=GitDB`` selects the
553+
same backend. The pure-Python storage implementation is unavailable. The
554+
``gitdb`` package provides shared types and utilities.
528555

529556
Git Command Debugging and Customization
530557
***************************************

‎fuzzing/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,8 @@ Contains Python files for each fuzz test.
7171
**Things to Know**:
7272

7373
- Each fuzz test targets a specific part of GitPython's functionality.
74-
- Repository and object targets exercise Git-backed workflows. Removed Python index/tree binary parsers are no longer
75-
fuzz targets; malformed configuration and object IDs can now be rejected by Git or by Python input validation.
74+
- Repository and object targets exercise Git-backed workflows. Git or Python input validation may reject malformed
75+
configuration and object IDs.
7676
- Test files adhere to the naming convention: `fuzz_<API Under Test>.py`, where `<API Under Test>` indicates the
7777
functionality targeted by the test.
7878
- Any functionality that involves performing operations on input data is a possible candidate for fuzz testing, but

0 commit comments

Comments
 (0)