Skip to content

Keep files deeper than maxdepth in a recursive LocalFileSystem.rm - #2238

Merged
martindurant merged 1 commit into
fsspec:masterfrom
raashish1601:fix/local-rm-maxdepth
Oct 7, 2026
Merged

martindurant merged 1 commit into
fsspec:masterfrom
raashish1601:fix/local-rm-maxdepth

Conversation

@raashish1601

Copy link
Copy Markdown
Contributor

LocalFileSystem.rm(path, recursive=True, maxdepth=...) ignores maxdepth and calls shutil.rmtree, so it deletes the whole tree, including files below maxdepth:

fs = fsspec.filesystem("file")
fs.pipe({f"{root}/src/top.txt": b"", f"{root}/src/sub/mid.txt": b"", f"{root}/src/sub/deeper/low.txt": b""})
fs.rm(f"{root}/src", recursive=True, maxdepth=2)
fs.exists(f"{root}/src/sub/deeper/low.txt")  # False, the whole tree is gone

The base AbstractFileSystem.rm only expands the path down to maxdepth, and #2233 just made mv keep files deeper than maxdepth in the source for the same reason.

With this change, when maxdepth is given, LocalFileSystem.rm expands the path with that depth, removes the files it finds, and removes a directory only once it is empty. A directory that still holds deeper files is kept. Without maxdepth it still uses shutil.rmtree.

Tests: added test_rm_recursive_maxdepth_keeps_deeper_files in test_local.py. It fails on master and passes here. test_local.py passes, and pre-commit is clean on the changed files. Added a changelog entry.

@martindurant
martindurant merged commit cd12d87 into fsspec:master Oct 7, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants