Skip to content

fix(ps): mark streaming sub active before opening grpc stream - #1234

Merged
sicoyle merged 1 commit into
dapr:mainfrom
sicoyle:samcoyle/python-sdk-merge-queue-fix
Sep 24, 2026
Merged

sicoyle merged 1 commit into
dapr:mainfrom
sicoyle:samcoyle/python-sdk-merge-queue-fix

Conversation

@sicoyle

@sicoyle sicoyle commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes a race in the sync streaming Subscription that made test_streaming_subscribe_receives_published_message fail intermittently in CI. The same race can drop real user subscriptions on connect.

Subscription.start() created the bidi stream and only then marked it active. gRPC starts reading the request iterator on its own thread before SubscribeTopicEventsAlpha1() returns. When that thread won the race, the iterator sent the initial request, saw the stream as inactive, and ended. That half-closed the stream, so the sidecar unsubscribed right away and the client got UNKNOWN: EOF. The reconnect hit the same race, next_message() returned None, and the test failed with 'NoneType' object has no attribute 'data'.

The async Subscription is not affected, since it marks the stream active before its first await.

Changes

  • dapr/clients/grpc/subscription.py: mark the stream active before creating it, and mark it inactive again if setup fails.
  • dapr/clients/grpc/subscription.py: give each stream its own send queue, and wait on it with a timeout. Before this, an iterator left over after close() or a reconnect blocked forever on a shared queue and could take acks meant for the new stream.
  • tests/clients/test_subscription.py: add unit tests with a stub that reads the request iterator before start() returns, which reproduces the race every time. Also check that the iterator exits after close().
  • tests/integration/test_pubsub.py: keep reading when next_message() returns None, since the API documents that as the normal result of a reconnect.

Testing

  • The new unit test fails on master and passes with this fix.
  • The full unit suite, ruff and mypy pass.
  • The integration tests run in CI against a real sidecar.

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: #[issue number]

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

  • Code compiles correctly
  • Created/updated tests
  • Extended the documentation

…tream

Signed-off-by: Samantha Coyle <sam@diagrid.io>
Copilot AI lite review requested due to automatic review settings September 24, 2026 14:18
@sicoyle
sicoyle requested review from a team as code owners September 24, 2026 14:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codecov

codecov Bot commented Sep 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.91%. Comparing base (bffb749) to head (d6bb874).
⚠️ Report is 227 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1234      +/-   ##
==========================================
- Coverage   86.63%   83.91%   -2.72%     
==========================================
  Files          84      123      +39     
  Lines        4473    10260    +5787     
==========================================
+ Hits         3875     8610    +4735     
- Misses        598     1650    +1052     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sicoyle
sicoyle added this pull request to the merge queue Sep 24, 2026
Merged via the queue into dapr:main with commit 6215f4e Sep 24, 2026
21 of 22 checks passed
nelson-parente added a commit to nelson-parente/python-sdk that referenced this pull request Sep 24, 2026
Resolves the conflict in tests/integration/test_pubsub.py. The next_message None loop from dapr#1234 moves into the shared _next_message helper, so the bulk metadata test gets it too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Nelson Parente <nelson_parente@live.com.pt>
CasperGN added a commit to CasperGN/python-sdk that referenced this pull request Sep 24, 2026
Resolve conflicts with dapr#1234 in the sync subscription. Keep this branch's
per-stream send queue with a None sentinel and activation under the lock,
and take dapr#1234's activation before the stream is opened and its reset to
inactive when the initial read fails. Keep dapr#1234's tests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Casper Nielsen <casper@diagrid.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants