Skip to content

[rush] rushd: Ctrl+C on rush-client build does not stop running operations; the client exits 1 after 5 s (instead of 130/143) while other clients stay blocked #6060

Description

Summary

rush-client build handles SIGINT/SIGTERM by sending requestCancel. The daemon then calls OperationGraph.abortCurrentIterationAsync(), which by design only prevents unstarted operations. Running operation processes are never terminated, and the abort waits for every in-flight operation to finish naturally. So after Ctrl+C:

  • the client gives up after 5 s with Daemon did not finish cancellation; disconnected without retrying the command. and exits 1 (native Rush exits 130 for SIGINT and 143 for SIGTERM);
  • the cancelled operation keeps running to completion (about 26 s in our repro), holding the execution lease, so other clients stay blocked behind a build nobody wants.

Repro steps

# 8-project synthetic workspace, p02 build sleeps 30 s; RUSH_DAEMON=1; rush-client from main @ 60007c9a8c
rush-client build & C=$!; sleep 8; kill -INT $C       # p02 is running
sleep 1; rush-client build --only p05                  # second client

Expected result: Cancellation promptly terminates the running operations' process trees (native Ctrl+C in a terminal kills the whole foreground process group), marks them Aborted (no cache write), releases the lease, and returns an aborted result well under a second. The client exits 130/143 and prints a short cancellation notice.

Actual result: At t=13.2 s the client exits 1. p02 keeps running until t≈34 s. The second client is blocked about 26 s and exits 0 at 38.2 s. With 3 s operations, cancel latency equals the remaining time of the running operation (median about 2.1 s), and in 1 of 8 runs a new operation started 9 ms after the signal. The exit code was 1 for SIGINT in 8 of 8 runs, and 1 for SIGTERM.

Details

Root cause (main @ 60007c9):

  • libraries/rush-lib OperationGraph.ts:630 abortCurrentIterationAsync is a soft abort (watch-mode semantics). Running runners get no abort signal, and the signal is only checked before an operation starts (:998).
  • libraries/rush-daemon/src/PhasedRequestRouter.ts:484-527 uses that soft abort for client cancellation. The daemon already has process-tree termination for global commands (GlobalCommandExecutionContext → SubprocessTerminator.killProcessTree), but not for phased operations.
  • Exit code: the daemon returns { outcome: 'aborted', exitCode: 1 } (CommandResultPolicy.ts:67-68), and apps/rush-cli-client/src/launchClient.ts:170-171 copies it verbatim. The 130 branch (:179) only handles non-result outcomes.

Suggested fix: add a hard-abort path. Pass the iteration abort signal to operation runners, and have the shell and IPC runners kill their subprocess tree on abort (report Aborted, skip the cache write). Use it when the last live participant of a batch cancels, and keep the soft abort for watch-mode invalidation. In the client, exit 128 + signo and print rush-client: build cancelled. Handle SIGHUP like SIGTERM.

This was found during an automated performance/behavior analysis of rush-client/rushd on Linux and reproduced independently twice (including the coalesced case: cancelling 1 of 3 coalesced clients works for the other 2, but the cancelled client still waits 5 s and exits 1).

Standard questions

Question Answer
@microsoft/rush globally installed version? built from main @ 60007c9 (5.179.0)
rushVersion from rush.json? 5.179.0
pnpmVersion, npmVersion, or yarnVersion from rush.json? pnpm@10.27.0
(if pnpm) useWorkspaces from pnpm-config.json? true
Operating system? Linux (WSL2 Ubuntu 24.04)
Would you consider contributing a PR? Yes
Node.js version (node -v)? 22.23.2

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions