[Bug]: `jsc.experimental.cacheRoot` destroys SWC performance

[walkerburgin]

What happened?

Transpilation using SWC is incredibly slow if swc_plugin() is used.

Something about setting jsc.experimental.cacheRoot inside of the Bazel sandbox when invoking SWC is absolutely destroying performance.

I put together a standalone repro here: walkerburgin/rules_swc-plugin-cache-repro using the @swc/plugin-noop.

➜  hyperfine --runs 5 'bazel clean && bazel build //...'
Benchmark 1: bazel clean && bazel build //...
  Time (mean ± σ):     26.342 s ±  0.855 s    [User: 0.182 s, System: 0.076 s]
  Range (min … max):   25.613 s … 27.800 s    5 runs

If I patch aspect_rules_swc to stop specifying jsc.experimental.cacheRoot:

Benchmark 1: bazel clean && bazel build //...
  Time (mean ± σ):      8.647 s ±  0.229 s    [User: 0.177 s, System: 0.070 s]
  Range (min … max):    8.346 s …  8.939 s    5 runs

The effect in our codebase using an actual plugin is much, much more pronounced. It slows typescript transpilation down by something like +500x and slows my machine to a crawl.

It feels like there's an SWC bug related to symlinks at the heart of this…

Version

Development (host) and target OS/architectures: darwin-arm64

Output of bazel --version:

➜  rules_swc-plugin-cache-repro git:(master) bazel --version
aspect 2025.05.46+36b9b384e

Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file:

bazel_dep(name = "aspect_rules_js", version = "2.1.1")
bazel_dep(name = "aspect_rules_ts", version = "3.3.1")
bazel_dep(name = "aspect_rules_swc", version = "2.3.0")

Language(s) and/or frameworks involved:

• SWC
• TypeScript

How to reproduce

See https://github.com/walkerburgin/rules_swc-plugin-cache-repro

Any other information?

No response

[jbedard]

Is it not simply the use of plugins that makes it so slow? SWC is fast (even when run per file) because there is almost no startup cost, where node is extremely slow to startup and using an SWC plugin often means it will launch node on every single file being transpiled...

[walkerburgin]

Is it not simply the use of plugins that makes it so slow?

I don't think so. The plugin we're using (@formatjs/swc-plugin-experimental) and the @swc/plugin-noop plugin are both compiled to WASM, so it shouldn't be running Node.

I'm seeing a meaningful difference in how long it takes to run swc in the sandbox with jsc.experimental.cacheRoot…

➜  _main git:(master) ✗ time ./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile --config-file pkgs/foo-lib/.swcrc --source-maps false --config-json '{"jsc":{"experimental":{"cacheRoot":"bazel-out/darwin_arm64-fastbuild/bin/pkgs/foo-lib/typescript_transpile_plugin_cache","plugins":[["./bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/@swc+plugin-noop@3.0.3/node_modules/@swc/plugin-noop",{}]]}}}' --out-file /dev/null pkgs/foo-lib/src/Component1.tsx

./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile     0.41s user 0.04s system 401% cpu 0.112 total
➜  _main git:(master) ✗ time ./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile --config-file pkgs/foo-lib/.swcrc --source-maps false --config-json '{"jsc":{"experimental":{"cacheRoot":"bazel-out/darwin_arm64-fastbuild/bin/pkgs/foo-lib/typescript_transpile_plugin_cache","plugins":[["./bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/@swc+plugin-noop@3.0.3/node_modules/@swc/plugin-noop",{}]]}}}' --out-file /dev/null pkgs/foo-lib/src/Component1.tsx

./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile     0.47s user 0.02s system 701% cpu 0.070 total
➜  _main git:(master) ✗ time ./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile --config-file pkgs/foo-lib/.swcrc --source-maps false --config-json '{"jsc":{"experimental":{"cacheRoot":"bazel-out/darwin_arm64-fastbuild/bin/pkgs/foo-lib/typescript_transpile_plugin_cache","plugins":[["./bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/@swc+plugin-noop@3.0.3/node_modules/@swc/plugin-noop",{}]]}}}' --out-file /dev/null pkgs/foo-lib/src/Component1.tsx

./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile     0.48s user 0.02s system 681% cpu 0.075 total

… and without jsc.experimental.cacheRoot:

➜  _main git:(master) ✗ time ./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile --config-file pkgs/foo-lib/.swcrc --source-maps false --config-json '{"jsc":{"experimental":{"plugins":[["./bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/@swc+plugin-noop@3.0.3/node_modules/@swc/plugin-noop",{}]]}}}' --out-file /dev/null pkgs/foo-lib/src/Component1.tsx
./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile     0.01s user 0.02s system 40% cpu 0.076 total
➜  _main git:(master) ✗ time ./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile --config-file pkgs/foo-lib/.swcrc --source-maps false --config-json '{"jsc":{"experimental":{"plugins":[["./bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/@swc+plugin-noop@3.0.3/node_modules/@swc/plugin-noop",{}]]}}}' --out-file /dev/null pkgs/foo-lib/src/Component1.tsx
./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile     0.01s user 0.01s system 81% cpu 0.026 total
➜  _main git:(master) ✗ time ./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile --config-file pkgs/foo-lib/.swcrc --source-maps false --config-json '{"jsc":{"experimental":{"plugins":[["./bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/@swc+plugin-noop@3.0.3/node_modules/@swc/plugin-noop",{}]]}}}' --out-file /dev/null pkgs/foo-lib/src/Component1.tsx
./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile     0.01s user 0.01s system 79% cpu 0.025 total

Or with jsc.experimental.cacheRoot pointing elsewhere?

➜  _main git:(master) ✗ time ./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile --config-file pkgs/foo-lib/.swcrc --source-maps false --config-json '{"jsc":{"experimental":{"cacheRoot":"swc-cache","plugins":[["./bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/@swc+plugin-noop@3.0.3/node_modules/@swc/plugin-noop",{}]]}}}' --out-file /dev/null pkgs/foo-lib/src/Component1.tsx

./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile     0.41s user 0.05s system 387% cpu 0.119 total
➜  _main git:(master) ✗ time ./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile --config-file pkgs/foo-lib/.swcrc --source-maps false --config-json '{"jsc":{"experimental":{"cacheRoot":"swc-cache","plugins":[["./bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/@swc+plugin-noop@3.0.3/node_modules/@swc/plugin-noop",{}]]}}}' --out-file /dev/null pkgs/foo-lib/src/Component1.tsx

./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile     0.01s user 0.01s system 75% cpu 0.024 total
➜  _main git:(master) ✗ time ./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile --config-file pkgs/foo-lib/.swcrc --source-maps false --config-json '{"jsc":{"experimental":{"cacheRoot":"swc-cache","plugins":[["./bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/@swc+plugin-noop@3.0.3/node_modules/@swc/plugin-noop",{}]]}}}' --out-file /dev/null pkgs/foo-lib/src/Component1.tsx

./external/aspect_rules_swc~~swc~swc_darwin-arm64/swc-darwin-arm64 compile     0.01s user 0.01s system 76% cpu 0.025 total

Maybe it's related to not having write permission on the plugin cache file 🤔

[walkerburgin]

I think SWC is using wasmer's FileSystemCache (here), which looks like it fails if the directory is read only (here)?

[walkerburgin]

Running this in https://github.com/walkerburgin/rules_swc-plugin-cache-repro seems to confirm the theory:

➜   hyperfine --runs 5 --prepare 'bazel clean' 'bazel build //...'
Benchmark 1: bazel build //...
  Time (mean ± σ):     26.402 s ±  0.829 s    [User: 0.092 s, System: 0.037 s]
  Range (min … max):   25.776 s … 27.831 s    5 runs

➜   hyperfine --runs 5 --prepare 'bazel clean' 'bazel build //... --experimental_writable_outputs'
Benchmark 1: bazel build //... --experimental_writable_outputs
  Time (mean ± σ):      7.490 s ±  0.535 s    [User: 0.083 s, System: 0.032 s]
  Range (min … max):    6.878 s …  8.007 s    5 runs

[alexeagle]

@walkerburgin sorry we dropped the thread here, could you summarize what the next steps should be, if any?

[walkerburgin]

Adding build --experimental_writable_outputs to our .bazelrc for local builds alleviated the immediate pain and I've been focused on other things 😰 . I think the right way to fix this would be to PR SWC (or wasmer) to either:

1. Not fail so eagerly if the bytecode cache directory is read only, as it is under Bazel without --experimental_writable_outputs.
2. Handle multiple processes writing to the bytecode cache at the same time

Ideally both. In the meantime, maybe we should add a warning to the docs?

[peter-cardenas-ai]

Could we remove the cacheRoot option as you did in your patch instead?