Skip to content

fix(core): separate daemon runtime env from graph identity - #36565

Merged
leosvelperez merged 2 commits into
nrwl:masterfrom
sdjayna:fix/berry-daemon-graph-identity
Sep 3, 2026
Merged

leosvelperez merged 2 commits into
nrwl:masterfrom
sdjayna:fix/berry-daemon-graph-identity

Conversation

@sdjayna

@sdjayna sdjayna commented Aug 4, 2026

Copy link
Copy Markdown

Current Behavior

"Berry" is Yarn's name for Yarn 2 and later; this reproduction uses Yarn 4.15.0. When Yarn runs a workspace binary it creates a fresh temporary xfs-* directory for that invocation, writes wrappers into it for the selected node, yarn, node-gyp and package binaries, exports it as BERRY_BIN_FOLDER, and prepends it to PATH.

The daemon compares the raw client environment against the previous one and invalidates the project graph when any value differs. The Berry folder differs on every invocation by design, so every yarn nx command on a completely unchanged workspace pays full graph construction:

Graph recompute necessary due to env variable refresh.
Changed keys: PATH, BERRY_BIN_FOLDER

Measured on a large workspace: 10.0-16.5s per invocation, against 0.27s from the retained graph when the daemon is reached without Yarn's wrapper environment. Reproduced on stock Nx across Windows, macOS and Ubuntu (workflow linked from #36564).

Expected Behavior

The daemon separates two concerns that currently share one comparison:

  • Runtime environment: the daemon and plugin workers always receive the full, latest client environment, including the currently valid Berry wrapper folder. Unchanged.
  • Graph identity: invalidation is decided on a normalized copy. normalizeDaemonEnvironmentForGraph removes the exact BERRY_BIN_FOLDER segment from PATH (case- and separator-insensitively on win32) and pins BERRY_BIN_FOLDER to a sentinel, so two Yarn invocations compare equal while a Yarn invocation and a direct Node invocation remain distinct identities.

The decision happens in handle-client-env.ts and unions two normalized diffs: process.env before versus after the existing applyDaemonEnvFromClient (whose signature, behaviour and generation semantics are unchanged), and the previously applied client environment versus the incoming one. A real toolchain change (a different SDK on PATH) still invalidates; lockfile and file changes continue to invalidate through file watching.

Tests: 54/54 in the touched suites, including every pre-existing upstream test unchanged, plus a Berry-rotation test (two environments differing only by xfs-* folder produce no invalidation and the new folder is still forwarded to workers) and unit coverage for the normalization and changed-key helpers.

Related Issue(s)

Fixes #36564

@netlify

netlify Bot commented Aug 4, 2026

Copy link
Copy Markdown

👷 Deploy request for nx-docs pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 47c243a

@netlify

netlify Bot commented Aug 4, 2026

Copy link
Copy Markdown

👷 Deploy request for nx-dev pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 47c243a

Yarn Berry creates a fresh BERRY_BIN_FOLDER for every invocation and puts
it first on PATH. The daemon compared raw environments, so every yarn nx
invocation discarded an unchanged cached graph. Graph identity is now
compared after normalizing the Berry folder out; workers still receive
every runtime change.

Fixes nrwl#36564
@sdjayna
sdjayna force-pushed the fix/berry-daemon-graph-identity branch from fd477cd to 6fe7f96 Compare September 2, 2026 17:12
@sdjayna
sdjayna marked this pull request as ready for review September 2, 2026 17:14
@sdjayna
sdjayna requested a review from a team as a code owner September 2, 2026 17:14
@sdjayna
sdjayna requested a review from leosvelperez September 2, 2026 17:14
@nx-cloud

nx-cloud Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

View your CI Pipeline Execution ↗ for commit 47c243a

Command Status Duration Result
nx affected --targets=lint,oxlint,test,build,e2... ✅ Succeeded 10m 32s View ↗
nx run-many -t check-imports check-lock-files c... ✅ Succeeded 3s View ↗
nx-cloud record -- pnpm nx-cloud conformance:check ✅ Succeeded 47s View ↗
nx build workspace-plugin ✅ Succeeded <1s View ↗
nx-cloud record -- nx sync:check ✅ Succeeded 17s View ↗
nx-cloud record -- nx format:check ✅ Succeeded 1s View ↗

☁️ Nx Cloud last updated this comment at 2026-09-03 15:09:52 UTC

@leosvelperez leosvelperez self-assigned this Sep 3, 2026

@nx-cloud nx-cloud Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nx Cloud has identified a flaky task in your failed CI:

Since the failure was identified as flaky, the solution is to rerun CI. Because this branch comes from a fork, it is not possible for us to push directly, but you can rerun by pushing an empty commit:

git commit --allow-empty -m "chore: trigger rerun"
git push

Nx Cloud View detailed reasoning in Nx Cloud ↗

🔔 Heads up, your workspace has pending recommendations ↗ to auto-apply fixes for similar failures.


🎓 Learn more about Self-Healing CI on nx.dev

@leosvelperez

Copy link
Copy Markdown
Member

@sdjayna thanks for the contribution! Could you please rebase? I merged a fix for a failing e2e that should unblock this one.

@leosvelperez leosvelperez left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@leosvelperez
leosvelperez merged commit 5a686ce into nrwl:master Sep 3, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Yarn Berry temporary PATH invalidates an unchanged daemon graph

3 participants