support Yarn's PnP mode

[turadg]

What is the Problem Being Solved?

In agoric-sdk the yarn install takes ~7s on a recent MBP, using hard links. It would be zero with Yarn’s PnP mode. That requires patching or augmenting Endo’s bundler to resolve packages using .pnp.cjs instead of node_modules.

Add Yarn PnP Resolution to compartment-mapper and bundle-source. PnP support must preserve strict undeclared-import failures while still producing self-contained bundles for execution environments like xsnap that cannot satisfy host imports.

Description of the Design

(in collaboration with GPT-5.4)

Summary

Add an internal resolver mode switch inside packages/compartment-mapper/src/node-modules.js that auto-detects a nearby .pnp.cjs for the entry module, uses Yarn’s PnP API for package discovery/linkage instead of walking node_modules, and keeps bundle-source working via its existing mapNodeModules() call path. The implementation should self-activate the detected .pnp.cjs so it works from a plain node process, with an explicit v1 assumption that a process only targets one PnP project.

Key Changes

• Introduce a private resolver abstraction with two modes:
  • node-modules: existing search() + ancestor node_modules lookup behavior.
  • pnp: new .pnp.cjs-backed entry search and dependency lookup behavior.
• Detect PnP by walking ancestor directories of the canonicalized entry module path for .pnp.cjs, loading the closest candidate with createRequire, and selecting it only if findPackageLocator(realModulePath) succeeds.
• In PnP mode, call pnp.setup() once after selecting the owning .pnp.cjs so existing fs-based read powers and requireResolve can access Yarn zip-cache package paths. Cache the activated .pnp.cjs identity, and if a later call targets a different .pnp.cjs in the same process, fail fast with a clear unsupported-in-v1 error instead of silently mixing graphs.
• Add a PnP entry-search path equivalent to today’s search():
  • canonicalize the module URL to a real filesystem path before giving it to PnP,
  • get the owning locator with findPackageLocator,
  • read that package’s package.json,
  • compute packageLocation, packageDescriptorLocation, and the entry moduleSpecifier from the locator’s packageLocation.
• Replace the physical findPackage() walk with a resolver-specific dependency lookup used by gatherDependency():
  • keep dependency enumeration from package.json exactly as it is now,
  • for PnP, resolve each declared dependency from the issuer locator’s packageDependencies,
  • treat null or missing references as unresolved and keep the existing optional/non-strict behavior,
  • turn {name, reference} into a package location through getPackageInformation.
• Preserve the current graph/canonical-name machinery:
  • graph edges and shortest-path weights still use dependency names,
  • graph nodes stay keyed by package file URLs,
  • policy hooks and packageDataHook continue to see the same external shapes.
• Fix workspace-language detection for PnP:
  • stop inferring “workspace package” purely from !location.includes('/node_modules/'),
  • add internal package metadata such as isWorkspace from the active resolver,
  • only apply workspaceLanguageForExtension overrides to true workspaces, not to third-party PnP packages living outside node_modules.
• Keep bundle-source API unchanged. Its only code change should be any minimal plumbing needed if mapNodeModules() needs an internal helper export; otherwise it should inherit support automatically through packages/bundle-source/src/zip-base64.js.

Public Interfaces / Types

• No public API or option changes for mapNodeModules(), loadLocation(), or bundleSource().
• Internal-only additions are acceptable:
  • a PnP resolver helper module or helper functions,
  • an internal isWorkspace flag on graph nodes or equivalent resolver metadata.
• Add a short README or changelog note that PnP is auto-detected via .pnp.cjs and that v1 supports one active PnP project per process.

Test Plan

Success means “the produced bundle is self-contained under PnP,” not just “resolution succeeds during bundling.”

• Add a shared test helper that copies a checked-in PnP fixture template to a temp directory, runs yarn install --mode=skip-build, and returns the realpath-based entry file URL. Use only workspace packages plus one tiny npm dependency so the fixture exercises both workspace and zip-cache package locations.
• Add compartment-mapper tests that cover:
  • successful mapping of a PnP project with a workspace dependency and an npm dependency,
  • snapshot/idempotence of the produced compartment map,
  • correct workspace-language behavior by proving only workspace packages get workspace parser treatment,
  • failure for an undeclared transitive import in PnP mode so mode-switching cannot silently fall back to flat node_modules semantics.
• Add bundle-source end-to-end tests that bundle the same PnP fixture into endoZipBase64 and import/execute it successfully, proving the resolver, archive creation, and zip-cache reads all work together.
• Run targeted package tests:
  • yarn workspace @endo/compartment-mapper test
  • yarn workspace @endo/bundle-source test
  • if snapshots are added, update and review only the new PnP-specific ones.
• Add bundle-source end-to-end tests for nestedEvaluate, not just endoZipBase64.

These were from an exploration in agoric-sdk:

• Bundle a PnP fixture whose entry imports one or more Endo packages by bare specifier, then execute the produced nestedEvaluate bundle in an environment with no host-module fallback; assert it does not try to import host modules at runtime.
• Add a regression test that inspects the produced nestedEvaluate bundle and verifies no bare package specifiers survive in the emitted bundle under PnP.
• Add a regression test for package imports that resolve through Yarn zip-cache paths, proving those dependencies are bundled rather than emitted as host imports.
• Add a fixture that exercises config-driven / non-module referrers, then verify resolution still finds workspace package subpaths under PnP.
• Add a test for a workspace package importing another workspace package subpath, to confirm PnP resolution works for both published deps and workspace-to-workspace edges.
• Add a test that runs the same fixture under plain node after auto-activating .pnp.cjs, to prove the implementation does not rely on Yarn’s wrapper process.
• Add a negative test where a generated nestedEvaluate bundle would previously have tried to load async_hooks or another Node-only path, and assert the PnP-aware path keeps that out of the runtime bundle.

Assumptions

• PnP support should match Yarn semantics, including rejecting undeclared package access instead of preserving flat node_modules leakage.
• Self-activating .pnp.cjs is acceptable for v1, and the implementation may reject mixing multiple distinct PnP projects in one process.
• The repo can rely on yarn being available in tests; generating the temp fixture at test time is preferred over checking in .pnp.cjs blobs and lockfile artifacts.

Security Considerations

Scaling Considerations

Compatibility Considerations

Upgrade Considerations

[naugtur]

This would be unusable for some lavamoat usecases unless architected in a way that we could consume the pre-compartmentMap state (graph) - we could use that as a data source in @lavamoat/aa transparently and get support for PnP to all of lavamoat tools potentially.
Exposing it should not be a big undertaking. It'd need to keep the existing hook and maybe even expand on what it does a little (@boneskull let's look into that)