Skip to content

fix(scheduler): tolerate partial cluster failures in replica estimation - #8083

Open
Sidharth-Singh10 wants to merge 1 commit into
karmada-io:masterfrom
Sidharth-Singh10:fix/estimator-partial-cluster-failures
Open

Sidharth-Singh10 wants to merge 1 commit into
karmada-io:masterfrom
Sidharth-Singh10:fix/estimator-partial-cluster-failures

Conversation

@Sidharth-Singh10

Copy link
Copy Markdown

What type of PR is this?
/kind feature

What this PR does / why we need it:

When multiple replica estimators are configured and the accurate estimator cannot reach one candidate cluster, getClusterReplicasConcurrently returns a batch error. Both scheduler adapters discarded the whole batch, so estimates for healthy clusters were lost and every cluster fell back to the general estimator. This implements #6815.

The change:

  • marks failed clusters with UnauthenticReplica in both concurrent helpers, keeping successful peers in the slice;
  • preserves partial results through runSingleTemplateEstimator and calculateMultiTemplateAvailableSets;
  • merges them in calAvailableReplicas, taking the per-cluster minimum and keeping 0 authoritative;
  • refuses to merge an error result that lacks a failure sentinel, so a zero-filled error result cannot become real zero capacity;
  • leaves the descheduler's all-or-nothing error handling unchanged.

Which issue(s) this PR fixes:
Fixes #6815

Does this PR introduce a user-facing change?:

`karmada-scheduler`: Fixed the issue where a single cluster without a reachable scheduler-estimator caused accurate replica estimates for all other clusters to be discarded.

@karmada-bot karmada-bot added kind/feature Categorizes issue or PR as related to a new feature. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Nov 17, 2026
@karmada-bot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign garrybest for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@karmada-bot

Copy link
Copy Markdown
Contributor

Welcome @Sidharth-Singh10! It looks like this is your first PR to karmada-io/karmada 🎉

@karmada-bot karmada-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Nov 17, 2026
@codecov-commenter

codecov-commenter commented Nov 17, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 77.35849% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.66%. Comparing base (a17d6e0) to head (1fc6f77).
⚠️ Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
pkg/estimator/client/accurate.go 16.66% 10 Missing ⚠️
pkg/scheduler/core/util.go 91.30% 1 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8083      +/-   ##
==========================================
+ Coverage   42.47%   42.66%   +0.19%     
==========================================
  Files         880      880              
  Lines       55481    55516      +35     
==========================================
+ Hits        23566    23687     +121     
+ Misses      30153    30064      -89     
- Partials     1762     1765       +3     
Flag Coverage Δ
unittests 42.66% <77.35%> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Sidharth-Singh10
Sidharth-Singh10 marked this pull request as ready for review November 20, 2026 01:21
Copilot AI balanced review requested due to automatic review settings November 20, 2026 01:21
@karmada-bot karmada-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the change preserving healthy clusters’ replica estimates when another cluster’s estimator fails.

No open issues 🎉

What changed in this PR

Addresses #6815 by preserving accurate replica estimates for healthy clusters when another cluster’s estimator fails, instead of falling back to the general estimator for every cluster.

Changes:

  • Marks failed clusters with UnauthenticReplica while retaining successful results and aggregate errors.
  • Preserves partial results through both scheduler adapters and merges per-cluster minimums, keeping zero authoritative and rejecting error results without a failure sentinel.
  • Adds regression tests for partial failures, fallback behavior, and unchanged descheduler error handling.
File Description
pkg/​scheduler/​core/​util.go Accepts trustworthy partial estimates and merges per-cluster minimums.
pkg/​scheduler/​core/​util_test.go Tests partial-result merging, zero capacity, fallback, and real estimator integration.
pkg/​scheduler/​core/​estimation.go Preserves partial component-set estimates and failure sentinels.
pkg/​scheduler/​core/​estimation_test.go Covers partial errors, sentinel preservation, and duplicate cluster entries.
pkg/​estimator/​client/​interface.go Documents the per-cluster failure sentinel contract.
pkg/​estimator/​client/​accurate.go Marks failed concurrent estimates with sentinels and preserves wrapped errors.
pkg/​estimator/​client/​accurate_test.go Tests concurrent helpers and scheduler-estimator partial failures.
pkg/​descheduler/​core/​helper_test.go Verifies that the descheduler still discards the whole batch on error.

Review effort: Balanced

When one candidate cluster has no reachable scheduler-estimator, the
accurate estimator returns a batch error and both scheduler adapters
discard the estimates of the clusters that succeeded, forcing every
cluster onto the general estimator.

Mark failed clusters with UnauthenticReplica in both concurrent helpers,
preserve per-cluster results through the single-template and
component-set adapters, and merge them in calAvailableReplicas while
keeping the descheduler's all-or-nothing error handling unchanged.

Tests cover helper-level partial results, the outer merge for both
dispatch paths, minimum and zero semantics, and a real SchedulerEstimator
with a missing cache entry.

Refs: karmada-io#6815
Signed-off-by: Sidharth-Singh10 <sidharthsingh714@gmail.com>
@Sidharth-Singh10
Sidharth-Singh10 force-pushed the fix/estimator-partial-cluster-failures branch from 53b5788 to 1fc6f77 Compare November 21, 2026 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature Categorizes issue or PR as related to a new feature. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Handle partial cluster failures in scheduler estimator gracefully

4 participants