Skip to content

Fix ThreadPool.shrink() accounting of pending shutdowns - #859

Open
rm-you wants to merge 1 commit into
cherrypy:mainfrom
rm-you:fix/threadpool-shrink-accounting
Open

rm-you wants to merge 1 commit into
cherrypy:mainfrom
rm-you:fix/threadpool-shrink-accounting

Conversation

@rm-you

@rm-you rm-you commented Oct 19, 2026 •

Copy link
Copy Markdown

What kind of change does this PR introduce?

  • 🐞 bug fix
  • 🐣 feature
  • 📋 docs update
  • 📋 tests/coverage improvement
  • 📋 refactoring
  • 💥 other

What do these changes do?

shrink() subtracted the shutdowns that earlier calls had queued from the amount it was asked for, before it removed the threads that had already exited. So a call made before an earlier call's threads were reaped was mostly ignored. idle already leaves those threads out, so a caller that sizes the amount from it, as dynpool does, had them subtracted twice. The threads above the minimum were also counted together with those already shutting down, so the pool could drop below min.

stop() queued a shutdown request for every thread it listed, including those that shrink() had already retired but that were not reaped yet, and never reset _pending_shutdowns. The requests left over then stopped the threads of the next start(), and its grow() waited for them forever. HTTPServer.prepare() on a stopped server starts the pool again, as CherryPy does when its bus restarts.

shrink() now asks up to that many more threads to shut down, among those above the minimum that are not shutting down yet. stop() queues one request for each live thread that shrink() has not asked already, and forgets the pending shutdowns once the threads are joined.

cheroot 11.1.2 this change
4 × shrink(20) on 90 threads, min 10 70 left 10 left
shrink(40), then shrink(80), on 50 threads, min 10 0 left, 30 requests queued 10 left, none queued
start() after shrink() and stop() waits forever returns

The three new tests fail without the change; the restart test fails by a timeout rather than hanging the run.

Are there changes in behavior for the user?

shrink(amount) now retires up to amount more threads, counting those already shutting down as gone, as idle does, and never below min. #204 settled that amount counts the shutdowns already scheduled. That still holds for a caller that sizes it from idle, but one that subtracts pending shutdowns itself would now shrink more. Nothing in cheroot or CherryPy calls shrink().

Is it a substantial burden for the maintainers to support this?

No. It changes how two methods count, in a few lines, and the new tests cover all three cases.

Related issue number

Fixes #861. A related statistics retention problem, #860, is fixed in #858.

Checklist

Tip

New to open source contributions? Making a lovely PR is a great read.

  • I wrote descriptive pull request text above
  • The PR relates to only one subject with a clear title
    and description in grammatically correct, complete sentences
  • I think the code is well written
  • Unit tests for the changes exist
  • Integration tests for the changes exist (if applicable)
  • I used the same coding conventions as the rest of the project
  • The new code doesn't generate linter offenses
  • Project documentation (in docs/) and inline docstrings reflect the changes
  • My commits each have a descriptive title and a body explaining the why
    (see good commit messages)
  • I have added a change log entry
    (can be done once the PR number is known; alternatively, you can
    use a related issue number if one exists)
  • I'm planning to squash related commits together before final merge
  • I have read the contribution guide and the
    code of conduct

This change, its tests and this description were written by an AI agent (Anthropic Claude Opus 5.5) on behalf of Adam Harwell (@rm-you). The tests and the pre-commit checks were run locally with Python 3.11; mypy-stubtest was skipped, as Python 3.12 is not installed there.

🤖 Generated with Claude Code

@agentscanapp

agentscanapp Bot commented Oct 19, 2026

Copy link
Copy Markdown

Mixed activity

Activity patterns show a mix of organic and automated signals.

View full analysis →

Evidence
  • Very high PR volume (daily): 38 PRs within a single 24-hour window

Last 5 PRs:

This is an automated analysis by AgentScan

@read-the-docs-community

read-the-docs-community Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Documentation build overview

📚 cheroot | 🛠️ Build #35235941 | 📁 Comparing 1df9482 against latest (8402e55)

  🔍 Preview build  

5 files changed · ± 5 modified

± Modified

shrink() subtracted the shutdowns that earlier calls had queued from the
amount it was asked for, before it removed the threads that had already
exited. So a call made before an earlier call's threads were reaped was
mostly ignored: four shrink(20) calls in a row retired 20 of 90 threads,
and 40 when made a second apart. The idle property already leaves those
threads out, so a caller that sizes the amount from it had them
subtracted twice. The threads above the minimum were also counted with
those already shutting down: shrink(40), then shrink(80), on 50 threads
with a minimum of 10 retired all 50, and left 30 requests queued.

shrink() now asks up to that many more threads to shut down, among
those above the minimum that are not shutting down yet. stop() used to
queue a request for every thread it listed, dead ones included, so the
next start() lost its new threads to the requests left over and waited
for them forever. It now queues one for each live thread that shrink()
has not asked already, and forgets the pending shutdowns.

Generated-By: Anthropic Claude Opus 5.5
Signed-off-by: Adam Harwell <flux.adam@gmail.com>
@rm-you
rm-you force-pushed the fix/threadpool-shrink-accounting branch from d6337a6 to 1df9482 Compare October 19, 2026 08:12
@pyba-chronographer pyba-chronographer Bot added the bot:chronographer:provided A mark meaning that a new change log entry is present within the patch. label Oct 19, 2026
@codecov

codecov Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.63%. Comparing base (8402e55) to head (1df9482).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #859      +/-   ##
==========================================
+ Coverage   78.20%   78.63%   +0.43%     
==========================================
  Files          41       41              
  Lines        4791     4836      +45     
  Branches      548      550       +2     
==========================================
+ Hits         3747     3803      +56     
+ Misses        904      893      -11     
  Partials      140      140              

@rm-you

rm-you commented Oct 19, 2026 •

Copy link
Copy Markdown
Author

Hey, just dropping a quick note here since I just posted this and #858 with Claude, as I want to be clear that while Claude wrote most/all of the code here, there is a real human behind these driving and doing first-pass reviews, and these are bugs I've been tracking internally for quite a while.

We use cheroot to serve parts of an in-development RPC system in OpenStack's oslo.messaging project, and we trip over this periodically depending on deployment style. I will be tracking these and am happy to make adjustments based on feedback, or drop these PRs if there's a better approach from someone who knows the codebase better than I (and AI) do. Thanks!

@rm-you
rm-you marked this pull request as ready for review October 19, 2026 10:23
@rm-you

rm-you commented Oct 19, 2026

Copy link
Copy Markdown
Author

I'm pretty sure the failed test is not related to this, maybe someone can re-run it?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agentscan:mixed-signals bot:chronographer:provided A mark meaning that a new change log entry is present within the patch.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ThreadPool.shrink() miscounts pending shutdowns, and a stopped pool can hang on restart

1 participant