Skip to content

Commit 09b2dc8

Browse files
rm: replace recursive traversal with iterative stack to reduce allocations
Addresses #11222. The previous safe_remove_dir_recursive_impl used recursive call frames and allocated a fresh PathBuf per directory level, an eager Vec<OsString> per level, and a PathBuf::join() per entry. This commit rewrites it as an explicit iterative traversal to reduce heap pressure: - One shared PathBuf for the entire traversal (push/pop per entry) instead of one join() allocation per child. - Vec<StackFrame> work-stack with capacity(32) pre-allocation instead of recursive call frames, eliminating per-level frame overhead and preventing stack overflow on deep trees. - Lazy DirIter (nix OwningIter / getdents) instead of eagerly collecting all children into a Vec<OsString> per level. - Single fd per StackFrame: DirFd::into_iter_dir transfers ownership without dup(2). - StackFrame::dir_path is Option<PathBuf>: child frames start with None (no allocation); populated lazily by try_reclaim_fd only if the frame is demoted to Drained. Fd budget: each Live frame holds exactly one open fd. When the process runs out of file descriptors (EMFILE/ENFILE), try_reclaim_fd demotes the oldest Live frame to Drained — its remaining entries are materialised into a Vec<OsString> and its fd is closed. Subsequent fd-requiring operations on a Drained frame re-open the directory from dir_path on demand. This allows traversal of trees of arbitrary depth at the cost of one extra openat(2) per entry in a drained frame. Security: open_child_iter uses SymlinkBehavior::NoFollow (O_NOFOLLOW | O_DIRECTORY) when opening child directories. stat_at(NoFollow) confirms the entry is a real directory; the subsequent open with O_NOFOLLOW ensures a concurrent symlink swap between the stat and the open is rejected by the kernel with ELOOP/ENOTDIR rather than silently followed. Drained frame helpers (frame_stat_at, frame_open_child, frame_unlink_at) re-open the directory with O_NOFOLLOW, protecting the final path component against concurrent symlink swaps during EMFILE recovery. Bug fix: thread progress_bar through safe_remove_dir_recursive_impl and handle_unlink so pb.inc() is called for every file and subdirectory unlinked during traversal. Previously the bar only ticked once (when the root directory itself was removed), leaving it frozen for large trees. Bug fix: readdir errors in try_reclaim_fd are now reported via show_error! and set had_error=true on the frame, preventing silent entry loss during EMFILE recovery. Previously .filter_map(|r| r.ok()) discarded errors, causing files to be silently skipped. Other changes: - Add DirIter to uucore::safe_traversal: lazy iterator over directory entries exposing stat_at, open_child_iter, and unlink_at so all directory operations go through a single fd. - Add FrameIter enum (Live(DirIter) | Drained(vec::IntoIter<OsString>)) and try_reclaim_fd / frame_stat_at / frame_open_child / frame_unlink_at helpers for the EMFILE fallback path. - Add rm_alloc_count bench (counting GlobalAlloc) to measure allocation reduction directly. - Add test_recursive_deep_tree_no_stack_overflow (#[ignore], 800-level deep tree, run manually). - Add test_recursive_emfile_fd_recycling: caps RLIMIT_NOFILE at 30, removes a 40-level tree, exercises the Drained fallback end-to-end. - Add test_recursive_interactive_decline_child_no_error covering the interactive-decline path without propagating a spurious error. - Restore #[test] on test_recursive_symlink_loop accidentally dropped when the preceding test was inserted. - Document fd budget, push/pop invariants, and TOCTOU windows inline. Co-authored-by: Copilot <[email protected]>
1 parent 12485f2 commit 09b2dc8

File tree

3 files changed

+845
-87
lines changed

3 files changed

+845
-87
lines changed

0 commit comments

Comments
 (0)