Commit 09b2dc8
rm: replace recursive traversal with iterative stack to reduce allocations
Addresses #11222.
The previous safe_remove_dir_recursive_impl used recursive call frames
and allocated a fresh PathBuf per directory level, an eager Vec<OsString>
per level, and a PathBuf::join() per entry. This commit rewrites it as an
explicit iterative traversal to reduce heap pressure:
- One shared PathBuf for the entire traversal (push/pop per entry)
instead of one join() allocation per child.
- Vec<StackFrame> work-stack with capacity(32) pre-allocation instead of
recursive call frames, eliminating per-level frame overhead and
preventing stack overflow on deep trees.
- Lazy DirIter (nix OwningIter / getdents) instead of eagerly collecting
all children into a Vec<OsString> per level.
- Single fd per StackFrame: DirFd::into_iter_dir transfers ownership
without dup(2).
- StackFrame::dir_path is Option<PathBuf>: child frames start with None
(no allocation); populated lazily by try_reclaim_fd only if the frame
is demoted to Drained.
Fd budget: each Live frame holds exactly one open fd. When the process
runs out of file descriptors (EMFILE/ENFILE), try_reclaim_fd demotes the
oldest Live frame to Drained — its remaining entries are materialised into
a Vec<OsString> and its fd is closed. Subsequent fd-requiring operations
on a Drained frame re-open the directory from dir_path on demand. This
allows traversal of trees of arbitrary depth at the cost of one extra
openat(2) per entry in a drained frame.
Security: open_child_iter uses SymlinkBehavior::NoFollow (O_NOFOLLOW |
O_DIRECTORY) when opening child directories. stat_at(NoFollow) confirms
the entry is a real directory; the subsequent open with O_NOFOLLOW ensures
a concurrent symlink swap between the stat and the open is rejected by the
kernel with ELOOP/ENOTDIR rather than silently followed. Drained frame
helpers (frame_stat_at, frame_open_child, frame_unlink_at) re-open the
directory with O_NOFOLLOW, protecting the final path component against
concurrent symlink swaps during EMFILE recovery.
Bug fix: thread progress_bar through safe_remove_dir_recursive_impl and
handle_unlink so pb.inc() is called for every file and subdirectory
unlinked during traversal. Previously the bar only ticked once (when the
root directory itself was removed), leaving it frozen for large trees.
Bug fix: readdir errors in try_reclaim_fd are now reported via show_error!
and set had_error=true on the frame, preventing silent entry loss during
EMFILE recovery. Previously .filter_map(|r| r.ok()) discarded errors,
causing files to be silently skipped.
Other changes:
- Add DirIter to uucore::safe_traversal: lazy iterator over directory
entries exposing stat_at, open_child_iter, and unlink_at so all
directory operations go through a single fd.
- Add FrameIter enum (Live(DirIter) | Drained(vec::IntoIter<OsString>))
and try_reclaim_fd / frame_stat_at / frame_open_child / frame_unlink_at
helpers for the EMFILE fallback path.
- Add rm_alloc_count bench (counting GlobalAlloc) to measure allocation
reduction directly.
- Add test_recursive_deep_tree_no_stack_overflow (#[ignore], 800-level
deep tree, run manually).
- Add test_recursive_emfile_fd_recycling: caps RLIMIT_NOFILE at 30,
removes a 40-level tree, exercises the Drained fallback end-to-end.
- Add test_recursive_interactive_decline_child_no_error covering the
interactive-decline path without propagating a spurious error.
- Restore #[test] on test_recursive_symlink_loop accidentally dropped
when the preceding test was inserted.
- Document fd budget, push/pop invariants, and TOCTOU windows inline.
Co-authored-by: Copilot <[email protected]>1 parent 12485f2 commit 09b2dc8
File tree
3 files changed
+845
-87
lines changed- src
- uucore/src/lib/features
- uu/rm/src/platform
- tests/by-util
3 files changed
+845
-87
lines changed
0 commit comments