Skip to content

feat(core): eagerly shutdown plugins that don't provide later hooks#34253

Merged
FrozenPandaz merged 6 commits intomasterfrom
feat/plugin-lifecycle-managemetn
Feb 6, 2026
Merged

feat(core): eagerly shutdown plugins that don't provide later hooks#34253
FrozenPandaz merged 6 commits intomasterfrom
feat/plugin-lifecycle-managemetn

Conversation

@AgentEnder
Copy link
Member

@AgentEnder AgentEnder commented Jan 29, 2026

Plugin Isolation Architecture

1. Plugin Loading Flow

1a. Entry Point - Isolation Decision

flowchart TD
    Start([getPlugins called]) --> CheckIsolation{Isolation<br/>enabled?}
    CheckIsolation -->|Yes| LoadIsolated[loadIsolatedNxPlugin]
    CheckIsolation -->|No| LoadInProcess[loadNxPluginInProcess]
    LoadIsolated --> IsolatedPath([See: Isolated Loading])
    LoadInProcess --> InProcessPath([See: In-Process Loading])
Loading

1b. Isolated Plugin Loading

flowchart TD
    subgraph Main["Main Process"]
        Start([loadIsolatedNxPlugin]) --> CheckCache{In cache?}
        CheckCache -->|Yes| ReturnCached([Return cached promise])
        CheckCache -->|No| StaticLoad[IsolatedPlugin.load]
        StaticLoad --> Resolve[resolveNxPlugin<br/>find plugin path]
        Resolve --> SpawnWorker[spawn child process]
    end

    SpawnWorker -.->|"start process"| WorkerStart

    subgraph Worker["Worker Process (plugin-worker.ts)"]
        WorkerStart([process starts]) --> CreateServer[create Unix socket server]
        CreateServer --> Listen[listen for connections]
        Listen --> WaitForConnect[wait for main process]
        WaitForConnect --> HandleLoad[receive 'load' message]

        subgraph InProcess["In-Process Loading (same as 1c)"]
            HandleLoad --> RequirePlugin[require plugin module]
            RequirePlugin --> NormalizePlugin[normalizeNxPlugin]
        end

        NormalizePlugin --> SendLoadResult[send 'loadResult'<br/>with hook capabilities]
        SendLoadResult --> WaitForMessages[wait for hook messages<br/>or socket close]
        WaitForMessages --> HandleHook{message<br/>received?}
        HandleHook -->|hook message| ExecuteHook[call plugin.hook]
        ExecuteHook --> SendResult[send result]
        SendResult --> WaitForMessages
        HandleHook -->|socket closed| Cleanup[cleanup & exit]
    end

    subgraph Main2["Main Process (continued)"]
        ConnectSocket[connect via<br/>Unix socket] --> SendLoad[send 'load' message]
        SendLoad --> WaitLoad[wait for 'loadResult']
        WaitLoad --> SetupHooks[setupHooks<br/>create lifecycle manager]
        SetupHooks --> CheckGraphHooks{Has graph<br/>phase hooks?}
        CheckGraphHooks -->|No| EarlyShutdown[socket.end<br/>shutdown worker]
        CheckGraphHooks -->|Yes| KeepAlive[keep worker alive]
        EarlyShutdown --> Done([Plugin ready])
        KeepAlive --> Done
    end

    SpawnWorker --> ConnectSocket
    SendLoadResult -.->|"loadResult"| WaitLoad
    EarlyShutdown -.->|"socket close"| Cleanup
Loading

1c. In-Process Plugin Loading

flowchart TD
    Start([loadNxPluginInProcess]) --> Resolve[resolveNxPlugin]
    Resolve --> Require[require plugin module]
    Require --> Normalize[normalizeNxPlugin<br/>wrap hooks]
    Normalize --> Done([Plugin ready])
Loading

2. Hook Execution Flow

2a. Isolated Hook Execution

flowchart TD
    Start([hook called<br/>e.g. createNodes]) --> EnsureAlive{_alive?}
    EnsureAlive -->|No| Restart[spawnAndConnect<br/>restart worker]
    Restart --> SetAlive[_alive = true]
    SetAlive --> EnsureAlive

    EnsureAlive -->|Yes| EnterHook[lifecycle.enterHook<br/>increment session count]
    EnterHook --> SendRequest[sendRequest<br/>over socket]
    SendRequest --> WaitResponse[wait for response<br/>with timeout]

    WaitResponse --> CheckSuccess{success?}
    CheckSuccess -->|No| ExitHookError[lifecycle.exitHook]
    ExitHookError --> ThrowError[throw error]

    CheckSuccess -->|Yes| ExitHook[lifecycle.exitHook]
    ExitHook --> CheckShutdown{should<br/>shutdown?}
    CheckShutdown -->|Yes| Shutdown[shutdown worker]
    CheckShutdown -->|No| Return([return result])
    Shutdown --> Return
Loading

2b. Shutdown Decision Logic

flowchart TD
    Start([exitHook called]) --> IsLastHook{Last hook<br/>in phase?}
    IsLastHook -->|No| NoShutdown1([return false])

    IsLastHook -->|Yes| CheckSessions{sessionCount<br/>== 0?}
    CheckSessions -->|No| NoShutdown2([return false<br/>other callers active])

    CheckSessions -->|Yes| CheckLaterPhases{Has later<br/>active phases?}
    CheckLaterPhases -->|Yes| NoShutdown3([return false<br/>needed later])
    CheckLaterPhases -->|No| YesShutdown([return true<br/>safe to shutdown])
Loading

3. Developer Workflow: Adding/Modifying Plugin Hooks

Step 1: Design Public API

flowchart TD
    A1[public-api.ts] --> A2[Define context type<br/>e.g. MyHookContext]
    A2 --> A3[Export new types]
    A3 --> A4[loaded-nx-plugin.ts]
    A4 --> A5[Add hook to<br/>LoadedNxPlugin interface]
Loading

Step 2: Define Message Types

flowchart TD
    B1[messaging.ts] --> B2[Add entry to PluginMessageDefs]
    B2 --> B3[Define payload and result types]
    B3 --> B4[Add to MESSAGE_TYPES array]
    B4 --> B5[Add to RESULT_TYPES array]
Loading

The messaging system uses a unified DefineMessages pattern. To add a new message:

// In PluginMessageDefs, add a new entry:
type PluginMessageDefs = DefineMessages<{
  // ... existing messages ...

  myHook: {
    payload: {
      context: MyHookContext;
    };
    result:
      | { success: true; data: MyResultData }
      | { success: false; error: Error };
  };
}>;

The individual message/result types (PluginWorkerMyHookMessage, PluginWorkerMyHookResult)
are automatically derived. Export them if needed for external use:

export type PluginWorkerMyHookMessage = MessageOf<PluginMessageDefs, 'myHook'>;
export type PluginWorkerMyHookResult = ResultOf<PluginMessageDefs, 'myHook'>;

Step 3: Handle in Worker Process

flowchart TD
    C1[plugin-worker.ts] --> C2[Add handler in<br/>consumeMessage]
    C2 --> C3["Call plugin.myHook()"]
    C3 --> C4[Return result payload]
Loading

Handlers return just the result payload - the infrastructure wraps it automatically:

// In consumeMessage handlers:
myHook: async ({ context }) => {
  try {
    const data = await plugin.myHook(context);
    return { success: true as const, data };
  } catch (e) {
    return { success: false as const, error: createSerializableError(e) };
  }
},

Step 4: Update Load Result

flowchart TD
    D1[messaging.ts] --> D2[Add hasMyHook to<br/>load.result in PluginMessageDefs]
    D2 --> D3[plugin-worker.ts]
    D3 --> D4[Populate hasMyHook<br/>in load handler]
Loading

Step 5: Wire Up IsolatedPlugin

flowchart TD
    E1[isolated-plugin.ts] --> E2[Add hook property<br/>to class]
    E2 --> E3[Update LoadResultPayload<br/>type export]
    E3 --> E4[Add to registeredHooks<br/>array in setupHooks]
    E4 --> E5[Add wrapped hook<br/>implementation]
    E5 --> E6["wrap('myHook', async (ctx) => {<br/>  sendRequest('myHook', { context: ctx })<br/>})"]
Loading

Step 6: Update Lifecycle Phases (if needed)

flowchart TD
    F1{New phase<br/>needed?} -->|Yes| F2[plugin-lifecycle-manager.ts]
    F2 --> F3[Add phase to<br/>HOOKS_BY_PHASE]
    F1 -->|No| F4[Add hook to existing<br/>phase array in HOOKS_BY_PHASE]
Loading

Step 7: Add Tests

flowchart TD
    G1[isolated-plugin.spec.ts] --> G2[Test hook registration]
    G2 --> G3[Test hook execution]
    G3 --> G4[Test restart behavior]
    G4 --> G5[plugin-lifecycle-manager.spec.ts]
    G5 --> G6[Test phase transitions<br/>with new hook]
    G6 --> G7[Test shutdown decisions]
Loading

File Reference

File Purpose
../public-api.ts Public types exported to plugin authors
../loaded-nx-plugin.ts Interface definition for loaded plugins
messaging.ts Message type definitions for worker communication
plugin-worker.ts Worker process - receives messages, calls plugin functions
isolated-plugin.ts Main class - spawns worker, sends messages, manages lifecycle
plugin-lifecycle-manager.ts Tracks phases, decides when to shutdown
load-isolated-plugin.ts Caching layer for isolated plugins
../get-plugins.ts Entry point - decides isolation mode

Lifecycle Phases

LOADED → [graph] → [pre-task] → {tasks run} → [post-task]
           │           │                           │
           │           └── preTasksExecution ──────┤
           │                                       │
           ├── createNodes                         │
           ├── createDependencies                  │
           └── createMetadata                      │
                                                   │
                                    postTasksExecution

Shutdown rules:

  • Plugin shuts down after its last active phase completes
  • If only postTasksExecution: shutdown immediately after load, restart when needed
  • Concurrent callers tracked via session count (ref counting)

@vercel
Copy link

vercel bot commented Jan 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
nx-dev Ready Ready Preview Feb 4, 2026 4:42pm

Request Review

@netlify
Copy link

netlify bot commented Jan 29, 2026

Deploy Preview for nx-docs ready!

Name Link
🔨 Latest commit 1a38243
🔍 Latest deploy log https://app.netlify.com/projects/nx-docs/deploys/69863a0649dbe10008925bd0
😎 Deploy Preview https://deploy-preview-34253--nx-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@nx-cloud
Copy link
Contributor

nx-cloud bot commented Jan 29, 2026

View your CI Pipeline Execution ↗ for commit 1a38243

Command Status Duration Result
nx affected --targets=lint,test,test-kt,build,e... ✅ Succeeded 18m 38s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 2m 56s View ↗
nx-cloud record -- nx-cloud conformance:check ✅ Succeeded 11s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 3s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded <1s View ↗

☁️ Nx Cloud last updated this comment at 2026-02-06 19:23:29 UTC

nx-cloud[bot]

This comment was marked as outdated.

@github-actions
Copy link
Contributor

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx create-nx-workspace@0.0.0-pr-34253-d811c6c my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-34253-d811c6c
Release details 📑
Published version 0.0.0-pr-34253-d811c6c
Triggered by @FrozenPandaz
Branch feat/plugin-lifecycle-managemetn
Commit d811c6c
Workflow run 21486147939

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

nx-cloud[bot]

This comment was marked as outdated.

@AgentEnder AgentEnder force-pushed the feat/plugin-lifecycle-managemetn branch from f9e1d83 to b96340d Compare February 2, 2026 17:46
@AgentEnder AgentEnder marked this pull request as ready for review February 2, 2026 19:28
@AgentEnder AgentEnder requested a review from a team as a code owner February 2, 2026 19:28
@AgentEnder AgentEnder force-pushed the feat/plugin-lifecycle-managemetn branch from b96340d to d49bd3f Compare February 2, 2026 19:29
@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

Failed to publish a PR release of this pull request, triggered by @AgentEnder.
See the failed workflow run at: https://github.com/nrwl/nx/actions/runs/21647952385

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx create-nx-workspace@0.0.0-pr-34253-1d2335a my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-34253-1d2335a
Release details 📑
Published version 0.0.0-pr-34253-1d2335a
Triggered by @AgentEnder
Branch feat/plugin-lifecycle-managemetn
Commit 1d2335a
Workflow run 21647952385

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

@netlify
Copy link

netlify bot commented Feb 4, 2026

Deploy Preview for nx-dev ready!

Name Link
🔨 Latest commit 1a38243
🔍 Latest deploy log https://app.netlify.com/projects/nx-dev/deploys/69863a06cf647e0008ab43e9
😎 Deploy Preview https://deploy-preview-34253--nx-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx create-nx-workspace@22.5.0-pr.34253.802dca0 my-workspace

Or just copy this version and use it in your own command:

22.5.0-pr.34253.802dca0
Release details 📑
Published version 22.5.0-pr.34253.802dca0
Triggered by @AgentEnder
Branch feat/plugin-lifecycle-managemetn
Commit 802dca0
Workflow run 21689445708

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

}

destroy(): void {
this.shutdown();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inline this

export type PluginWorkerResult = AllResults<PluginMessageDefs>;

/** Any message (request or result) */
export type AnyMessage = PluginWorkerMessage | PluginWorkerResult;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the unused types

};

/** Extract the full result type for a given key */
type ResultOf<
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move these library types to a separate file.

const registered = new Set(registeredHooks);

// Determine which phases are active and find first/last hooks per phase
this.activePhases = {};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than active phases can we go with registered phases

private readonly phaseSessionCount: Partial<Record<Phase, number>> = {};

/** Ordered list of phases (derived from HOOKS_BY_PHASE key order, narrowed to active phases) */
private readonly phaseOrder: Phase[] = [];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make this registered phase order

'preTasksExecution',
]);

expect(lifecycle.hasLaterActivePhases('graph')).toBe(true);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a check for preTask

expect(lifecycle.hasLaterActivePhases('post-task')).toBe(false);
});

it('should return false after post-task phase', () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combine with above

* This happens when the plugin has no hooks in the first phase.
*/
shouldShutdownImmediately(): boolean {
return this.phaseOrder[0] !== PHASE_ORDER[0];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this check only if the first phase is POST TASK

it('should identify active phases for single-hook plugin', () => {
const lifecycle = new PluginLifecycleManager(['createNodes']);

expect(lifecycle.hasHooksInPhase('graph')).toBe(true);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getRegisteredPhases().matchesSnapshot() or .registeredPhases.matchesSnapshot()

private shutdownIfInactive(): void {
if (this.pendingCount > 0) {
logger.verbose(
`[plugin-pool] worker for "${this.name}" has ${this.pendingCount} pending request(s), not shutting down yet`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change these logs.. because there's no more plugin-pool

Comment on lines 184 to 200
private async ensureAlive(): Promise<void> {
if (this._alive) {
return;
}

logger.verbose(`[plugin] restarting worker for "${this.name}"`);
await this.spawnAndConnect();
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Race condition when restarting worker

When _alive is false and multiple hooks are called concurrently, they will all pass the if (this._alive) check and call spawnAndConnect() multiple times, spawning duplicate workers.

Example scenario:

  1. Plugin shuts down after graph phase (_alive = false)
  2. Two concurrent calls to postTasksExecution occur
  3. Both check _alive (still false), both call spawnAndConnect()
  4. Two workers get spawned for the same plugin

Fix: Add a flag to track restart-in-progress:

private _restarting = false;

private async ensureAlive(): Promise<void> {
  if (this._alive) {
    return;
  }

  if (this._restarting) {
    // Wait for in-progress restart
    while (this._restarting) {
      await new Promise(resolve => setTimeout(resolve, 10));
    }
    return;
  }

  this._restarting = true;
  try {
    logger.verbose(`[plugin] restarting worker for "${this.name}"`);
    await this.spawnAndConnect();
  } finally {
    this._restarting = false;
  }
}
Suggested change
private async ensureAlive(): Promise<void> {
if (this._alive) {
return;
}
logger.verbose(`[plugin] restarting worker for "${this.name}"`);
await this.spawnAndConnect();
}
private _restarting = false;
private async ensureAlive(): Promise<void> {
if (this._alive) {
return;
}
if (this._restarting) {
// Wait for in-progress restart
while (this._restarting) {
await new Promise(resolve => setTimeout(resolve, 10));
}
return;
}
this._restarting = true;
try {
logger.verbose(`[plugin] restarting worker for "${this.name}"`);
await this.spawnAndConnect();
} finally {
this._restarting = false;
}
}

Spotted by Graphite Agent

Fix in Graphite


Is this helpful? React 👍 or 👎 to let us know.

nx-cloud[bot]

This comment was marked as outdated.

@AgentEnder AgentEnder force-pushed the feat/plugin-lifecycle-managemetn branch from a804993 to bb26e7a Compare February 6, 2026 15:11
@AgentEnder AgentEnder requested a review from Coly010 as a code owner February 6, 2026 15:11
nx-cloud[bot]

This comment was marked as outdated.

Introduce a unified type system for plugin worker messaging that reduces
boilerplate. Handlers now return just the payload and infrastructure wraps
responses automatically. Update ARCHITECTURE.md with new workflow.
…hitecture

Add "why" documentation to help developers understand the intent behind
design decisions, not just the technical flows. New sections cover:
- Why plugin isolation exists (memory, crashes, global state)
- Design principles (Unix sockets, eager shutdown, session counting)
- Phase system rationale and shutdown rules
- Common pitfalls when modifying the system
@AgentEnder AgentEnder force-pushed the feat/plugin-lifecycle-managemetn branch from ad458aa to 9c278b5 Compare February 6, 2026 18:00
Copy link
Contributor

@nx-cloud nx-cloud bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ The fix from Nx Cloud was applied

These changes fix the snapshot test failures by updating the regex pattern to filter out the new [isolated-plugin] debug logs introduced in this PR. The independent-projects.test.ts file was missing the same regex update that was already applied to independent-projects.workspaces.test.ts, causing snapshot mismatches when the new verbose logging appeared in test output.

Suggested Fix changes
diff --git a/e2e/release/src/independent-projects.test.ts b/e2e/release/src/independent-projects.test.ts
index e531dc9205..b4af7ecc60 100644
--- a/e2e/release/src/independent-projects.test.ts
+++ b/e2e/release/src/independent-projects.test.ts
@@ -238,7 +238,7 @@ describe('nx release - independent projects', () => {
         `release version 999.9.9-version-git-operations-test.2 -p ${pkg1} --git-commit --git-tag --verbose` // add verbose so we get richer output
       );
       const filteredOutput = versionWithGitActionsCLIOutput.replace(
-        /\[plugin-(pool|worker)\].*\n/g,
+        /\[(isolated-plugin|plugin-worker)\].*\n/g,
         ''
       );
       expect(filteredOutput).toMatchInlineSnapshot(`
@@ -325,7 +325,7 @@ describe('nx release - independent projects', () => {
         `release version 999.9.9-version-git-operations-test.3 --verbose --gitTag` // add verbose so we get richer output
       );
       const filteredConfigOutput = versionWithGitActionsConfigOutput.replace(
-        /\[plugin-(pool|worker)\].*\n/g,
+        /\[(isolated-plugin|plugin-worker)\].*\n/g,
         ''
       );
       expect(filteredConfigOutput).toMatchInlineSnapshot(`
@@ -525,7 +525,7 @@ describe('nx release - independent projects', () => {
         `release changelog 999.9.9-changelog-git-operations-test.1 -p ${pkg1} --verbose`
       );
       const filteredChangelogOutput = versionWithGitActionsCLIOutput.replace(
-        /\[plugin-(pool|worker)\].*\n/g,
+        /\[(isolated-plugin|plugin-worker)\].*\n/g,
         ''
       );
       expect(filteredChangelogOutput).toMatchInlineSnapshot(`

Revert fix via Nx Cloud  

View interactive diff ↗
This fix was applied by Craigory Coppola

🎓 Learn more about Self-Healing CI on nx.dev

Co-authored-by: AgentEnder <AgentEnder@users.noreply.github.com>
@FrozenPandaz FrozenPandaz merged commit 37a69f0 into master Feb 6, 2026
24 checks passed
@FrozenPandaz FrozenPandaz deleted the feat/plugin-lifecycle-managemetn branch February 6, 2026 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants