Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,30 @@

diskcache-community forked from DiskCache at upstream commit ebfa37c (3 March 2024), two commits after the 5.6.3 release. Each entry names the pull request in this repository. Imported upstream pull requests are referenced by number, and their commits keep their original authors in the git history.

## Unreleased
## 5.7.0 (2026-10-11)

First release under the `diskcache-community` name: upstream commit ebfa37c plus the changes below. The import name stays `diskcache`; the `cache.db` schema, the file layout, the serialization modes, the memoize key format and the persisted settings are unchanged, so a 5.6.3 process opens a directory written by 5.7.0 and the reverse.

### Security

- Add `SafeDisk`, an opt-in `Disk` for CVE-2025-69872: it unpickles only an allowlist of standard-library types, raises `TypeError` at write time for anything the allowlist could not read back, and confines the paths read from the `filename` column to the cache directory. `Deque` and `Index` take a `disk` argument, `FanoutCache.cache()`, `deque()` and `index()` honour theirs, and the Django backend gains a `DISK` option. The default `Disk` is unchanged, so every 5.6.3 cache keeps opening (#11).
- Create every directory the cache makes with mode `0700`, including missing parents and the value subdirectories, and leave existing directories alone; includes upstream pull request 359 (#7).
- Warn at import time when the upstream `diskcache` distribution is installed next to `diskcache-community`, since both own the same package directory (#9).

### Features

- Read-only caches (upstream issue 113): opening an existing cache writes nothing unless a setting differs from the stored one, so a `cache.db` snapshot stays byte-identical across runs; a cache whose database or directory is not writable opens read-only, as does `Cache(directory, read_only=True)`, with reads, iteration, `check()` and memoize hits working and every write raising `ReadOnlyError`, a subclass of `sqlite3.OperationalError` (#17).
- `store_time=True` on `Cache.get`, `FanoutCache.get` and `DjangoCache.get` appends the store time to the returned tuple after any other requested metadata (upstream issue 330) (#16).
- `Cache.iterkeys(tag=...)` iterates the keys with a given tag, using the tag index when enabled; includes upstream pull request 372 (upstream issue 341) (#16).

### Fixes

- Support SQLite builds compiled with `SQLITE_DQS=0`, such as conda-forge's sqlite builds of early 2025: single-quoted literals in the `Settings` triggers and statistics counters (upstream pull request 311), and caches written by earlier versions get their triggers rewritten on first open when the stored text differs (#3).
- `FanoutCache.__getattr__` raises `AttributeError`, so `hasattr()` works (upstream pull request 366), and `FanoutCache.get` and `pop` return the documented tuple when a shard times out (upstream pull request 367) (#3).
- Write bytes values without the `io.BytesIO` copy (upstream pull request 333) and fix a README typo (upstream pull request 365) (#3).
- Accept either path separator in stored filenames, so a cache written on Windows opens on other systems and `check()` stays clean across platforms (#6).
- A parameter name in the `ignore` set of `memoize` matches the argument whether it is passed positionally or by keyword; keys of calls that do not pass an ignored parameter positionally are unchanged (#8).
- Close the SQLite connections of threads that have exited, from `close()` and when a `Cache` is garbage collected; keep connections inherited across `fork()` unused in the child, as SQLite requires, and let `fork()` wait for transactions in flight in other threads (upstream issues 295, 325 and 336; upstream pull request 299 was not taken because it serializes every transaction) (#13).

### Packaging

Expand All @@ -25,9 +34,12 @@ diskcache-community forked from DiskCache at upstream commit ebfa37c (3 March 20
### Docs

- Add the fork notice and migration steps to the README, plus `NOTICE`, `SECURITY.md`, `CONTRIBUTING.md` and this changelog; point the documentation and repository links at this fork and remove the upstream analytics, logo and funding links from the docs (#4).
- Record the changes in this changelog and extend the README fork notice (#12).

### Tests and CI

- Replace the upstream CI with one `ci.yml` workflow: lint checks, tests on Python 3.10 to 3.14 on Linux, macOS and Windows, Django 4.2 and 6.0 jobs, and an import check without Django (#1).
- Add golden compatibility fixtures written by diskcache 5.6.3 for every container, eviction policy, tag index, statistics, `JSONDisk`, memoized calls and file-backed values, with tests that read them with the current code and a reverse-direction test that reads a fork-written cache with 5.6.3 (#5).
- Run the test suite in CI against a SQLite built from source with `SQLITE_DQS=0` (#10).
- Rework the release workflow: tag-version check, build and twine check, TestPyPI dry run by dispatch, PyPI publish through trusted publishing behind the `pypi` environment, and a GitHub release with this changelog section (#14).
- Make the `incr` and `decr` expiry tests and the Django tests deterministic under pytest-xdist, so slow runners no longer fail them (#15).
4 changes: 2 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ and no fixed upstream version exists. This fork publishes the same library as
compatible with 5.6.3 caches and adds an opt-in ``SafeDisk``, selected with
``Cache(directory, disk=SafeDisk)`` and likewise on ``FanoutCache``,
``Deque``, ``Index`` and the Django ``DISK`` option; 6.0.0 makes safe
reading the default. No release has been published yet; ``SECURITY.md`` in the
repository describes the current state.
reading the default. 5.7.0 is the first release of the fork; ``SECURITY.md``
in the repository describes what it does and does not change.

What stays the same:

Expand Down
7 changes: 3 additions & 4 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Supported versions

Security fixes go into the latest 5.x release of diskcache-community only. There are no maintenance branches for older releases; if you are on an older release, upgrade to the latest one. No release has been published yet; until the first one, `master` is the supported line.
Security fixes go into the latest 5.x release of diskcache-community only. There are no maintenance branches for older releases; if you are on an older release, upgrade to the latest one.

## Reporting a vulnerability

Expand All @@ -16,13 +16,12 @@ You will hear back within a week. A confirmed problem is fixed in the next relea

Where this fork stands:

- The code on `master` today reads pickles from the cache directory exactly as 5.6.3 does. Nothing is fixed yet.
- 5.7.0 adds an opt-in `SafeDisk` that unpickles only an allowlist of standard-library types and rejects everything else, and lets every container (`Cache`, `FanoutCache`, `Deque`, `Index` and the Django backend) use it. The default stays unchanged, so 5.7.0 opens every 5.6.3 cache.
- 5.7.0, the first release of this fork, reads pickles with the default `Disk` exactly as 5.6.3 does, so it opens every 5.6.3 cache. It adds an opt-in `SafeDisk` that unpickles only an allowlist of standard-library types, rejects anything outside it at write time and confines value paths to the cache directory; `Cache`, `FanoutCache`, `Deque` and `Index` take it through `disk=SafeDisk` and the Django backend through the `DISK` option. New cache directories are created with mode `0700`.
- 6.0.0 makes safe reading the default and turns the trusted-pickle behaviour into an explicit choice, with a migration guide.

Until you run a version where safe reading is the default or you have opted into it, treat the cache directory as trusted input: keep it on local storage that only the user running the process can write to, and do not share it with other accounts.

A note on scanners: vulnerability scanners match on the distribution name, so `pip-audit`, Dependabot and similar tools report nothing for `diskcache-community` even though the code on `master` has the same behaviour as 5.6.3. A clean scan is not evidence that this issue is fixed. Use the list above instead, and the release notes of the version you run.
A note on scanners: vulnerability scanners match on the distribution name, so `pip-audit`, Dependabot and similar tools report nothing for `diskcache-community` even though the default `Disk` in 5.7.0 behaves as it did in 5.6.3. A clean scan is not evidence that this issue is fixed. Use the list above instead, and the release notes of the version you run.

## Dependencies

Expand Down
4 changes: 2 additions & 2 deletions diskcache/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,8 @@ def _warn_if_upstream_installed(site_dir=None):


__title__ = 'diskcache-community'
__version__ = '5.6.3'
__build__ = 0x050603
__version__ = '5.7.0'
__build__ = 0x050700
__author__ = 'Grant Jenks'
__license__ = 'Apache 2.0'
__copyright__ = 'Copyright 2016-2023 Grant Jenks'
6 changes: 5 additions & 1 deletion tests/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,11 @@

# Caching

CACHE_DIR = os.path.join(BASE_DIR, '.cache')
# Each pytest-xdist worker gets its own directory, otherwise one worker's
# tearDown clears keys another worker has just set.
CACHE_DIR = os.path.join(
BASE_DIR, '.cache', os.environ.get('PYTEST_XDIST_WORKER', 'main')
)

CACHES = {
'default': {
Expand Down
20 changes: 14 additions & 6 deletions tests/test_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -751,21 +751,29 @@ def test_incr(cache):
assert cache.incr('key', 2) == 8
assert cache.get('key', expire_time=True, tag=True) == (8, None, None)
assert cache.delete('key')
assert cache.set('key', 100, expire=0.100)
assert cache.set('key', 100, expire=10)
assert cache.get('key') == 100
time.sleep(0.120)
expire_now(cache, 'key')
assert cache.incr('key') == 1


def expire_now(cache, key):
"""Move the expiry of `key` into the past without waiting for the clock."""
cache._sql(
'UPDATE Cache SET expire_time = ? WHERE key = ?',
(time.time() - 1, key),
)


def test_incr_insert_keyerror(cache):
with pytest.raises(KeyError):
cache.incr('key', default=None)


def test_incr_update_keyerror(cache):
assert cache.set('key', 100, expire=0.100)
assert cache.set('key', 100, expire=10)
assert cache.get('key') == 100
time.sleep(0.120)
expire_now(cache, 'key')
with pytest.raises(KeyError):
cache.incr('key', default=None)

Expand All @@ -775,9 +783,9 @@ def test_decr(cache):
assert cache.decr('key', 2) == 2
assert cache.get('key', expire_time=True, tag=True) == (2, None, None)
assert cache.delete('key')
assert cache.set('key', 100, expire=0.100)
assert cache.set('key', 100, expire=10)
assert cache.get('key') == 100
time.sleep(0.120)
expire_now(cache, 'key')
assert cache.decr('key') == -1


Expand Down
Loading