SourceAccessor: Allow the lstat cache to be invalidated

[edolstra]

Motivation

After lockFlake() creates a flake.lock in a PosixSourceAccessor, it needs to be able to invalidate the lstat cache, otherwise a subsequent attempt to read flake.lock will fail with an erroneous "path does not exist" error.

This isn't an issue yet on master, but it arises when we make the path fetcher lazy (DeterminateSystems#312).

Context

---

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[xokdvium]

Not sure we should make such a deficiency a part of the object model of the SourceAccessor. The stat cache is sadly very TOCTOU prone. If anything, we should cache parent directory file descriptors — that wouldn't be such a footgun and wouldn't need explicit invalidation.

[edolstra]

How would that work? It's not the directory that's being invalidated here.

[xokdvium]

Just never do stat caching.

It's only hot at the moment because of the symlink check and that can be easily remedied by either using openat2 with RESOLVE_NO_SYMLINKS to ban symlink traversal or by doing one-path-component at a time traversal with O_NOFOLLOW. That would solve the primary issue (of not allowing symlink traversal) in a pretty cheap and race-free manner.

I have a PoC on the unix-source-accessor branch, but I still need to implement parent directory file descriptor caching to work better when openat2 is not available.

[xokdvium]

What I'm really worried about at the moment is that the stat cache makes the TOCTOU window arbitrarily large. Suppose it happens that daemon (for whatever reason) does a dumpPath on a tree that an adversary has a writable file descriptor to (think the FOD file descriptor smuggling). I don't think the daemon now uses the source accessor for such directory trees, but I'm not 100% sure.

Then the daemon does:

readDirectory("a")
readFile("a/b") # large file
# directory a gets swapped with a symlink to /some/sensitive/path
readFile("a/verysecretfile") # Doesn't barf because stat cache

[edolstra]

Just never do stat caching.

Sure, that would be much nicer, but we can always remove this invalidation code when we get rid of the cache.

[xokdvium]

Also true. Just pointing out that it's not the best approach in the long run. @Ericson2314 would love to hear your opinion on this too.

[Ericson2314]

I am OK doing this on a temporary basis, but once we have @xokdvium's DescriptorSourceAccessor we should revert it and delete the lstat cache. And we should do that soon!