From fe2463ee970fc2ffd9fa0e3f8f92a9f91512c939 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Mon, 16 Nov 2026 12:29:16 -0500 Subject: [PATCH 01/26] Plan native SCIM identity provider connections Co-Authored-By: Claude Opus 5.5 --- .../2026-11-16-scim-identity-providers.md | 267 ++++++++++++++++++ 1 file changed, 267 insertions(+) create mode 100644 docs/plans/2026-11-16-scim-identity-providers.md diff --git a/docs/plans/2026-11-16-scim-identity-providers.md b/docs/plans/2026-11-16-scim-identity-providers.md new file mode 100644 index 0000000000..d05867271f --- /dev/null +++ b/docs/plans/2026-11-16-scim-identity-providers.md @@ -0,0 +1,267 @@ +# Identity providers: SCIM provisioning and group mapping + +## Outcome and scope + +Give every organization, the provider organization included, one or more +**identity provider connections** that keep Bifrost users and role +assignments in step with an external directory. NinjaOne's identity provider +screens are the UX reference: a list of connections, a detail page with +provisioning status, credential expiry and a group-mapping table, and an edit +dialog with a left-hand section nav. + +Delivery happens on the MTG contribution fork first. The upstream RFC issue +comes later and carries the open questions listed at the end. + +In scope: + +- Organization domain claims, verified by DNS TXT. An organization can claim + several domains. +- A provisioning core that does not depend on the protocol. SCIM 2.0 + (RFC 7643 / 7644) is the first way in; Google Directory sync is the second. +- Group-to-role mapping, revocation and an "Identity Source" for each user. +- Platform events so workflows can react to provisioning. + +Out of scope for v1: + +- **Sign-in stays instance-global.** The existing Microsoft / Google / OIDC + configuration does not change. Connection records are shaped so per-IdP + sign-in can move into them later; that is a question for the RFC. +- **SAML.** OIDC covers sign-in. Google's custom SCIM profile hangs off a SAML + app in Google Admin, but that app is only a container, so users still sign + in through Google OIDC. +- **Customer-side workarounds** that federate Google into another IdP. Rejected. + +## Current seams + +- `User.organization_id` gives each user one home organization, and + `User.email` is unique across the instance + (`api/src/models/orm/users.py`). +- `Organization.domain` is one unverified string per org. Login + auto-provisioning (`api/src/services/user_provisioning.py`) and the + password-registration check read it. Verified claims are a separate model + and do not change that path in v1. +- A `user_roles` row is unique per `(user_id, role_id)`. Its scope comes from + `user_role_boundaries`. Nothing records where an assignment came from: + `assigned_by` is free text. +- Changing `is_active`, `email`, `organization_id` or `base_role_id` fires the + deferred `authz_epoch_upd` trigger. Existing access tokens then fail with + `refresh_required`, and refresh rejects inactive users. Deactivating a user + therefore revokes access with no extra session bookkeeping. + `revoke_all_human_sessions_in_transaction` is still called on deprovision + for an immediate, explicit cut-off. +- The user-lifecycle helpers in `api/shared/sdk_users.py` all take a user + `Caller` and enforce its permissions. A SCIM request is a machine caller with + connection-scoped authority, so the provisioning core writes through + narrow, purpose-built helpers. It reuses the guards + (`assert_role_assignment_allowed_for_user`, `insert_assignment`, + `set_user_base_role`, `role_cache.invalidate_user`) without inventing a + system `Caller`. +- `emit_audit` skips records with no actor. Provisioning passes an + `ActorContext` that names the connection. +- Platform events: topic `EventSource`s, `emit_event`, and + `CURATED_TOPICS` (`api/src/services/events/registry.py`). +- Hashed bearer secrets: `user_invites` (unique `token_hash`, `expires_at`, + `revoked_at`) and workflow keys (`last_used_at`) are the templates. SHA-256 + of `secrets.token_urlsafe(32)`, looked up by hash. + +## Decisions + +| # | Decision | +|---|---| +| 1 | Connections exist per organization. The provider org is just another org. Customer orgs and technicians use the same model. | +| 2 | v1 is provisioning plus group mapping per org. Sign-in stays instance-global. Consolidating sign-in into the connection goes into the RFC. | +| 3 | A connection may only create or change users whose email domain is a **verified** claim of the connection's org. It never touches a user whose home org is a different one. Claims are verified by DNS TXT. An org can claim several domains, and a domain can be verified by only one org. | +| 4 | An existing user in the same org (for example, someone invited earlier) is **adopted** by email, then matched by `externalId` from then on. A user who belongs to another org gets `409 uniqueness`. | +| 5 | `active=false` disables the user and revokes sessions. `DELETE` disables and unlinks; it does not hard-delete. One exception: a user the connection created who has **never signed in** has no history to keep, so `DELETE` removes them permanently. This handles Google's create, update and delete connection test. Losing the last mapped group revokes access (decision 7). | +| 6 | New users get the organization's default base role (`shared.base_role_defaults.resolve_base_role`). Groups can optionally map to roles (roles are global). Provisioning removes only roles that provisioning itself granted. Mapping can never grant Platform Admin or any other role the account guards refuse. | +| 7 | Once a connection has at least one group mapping, a user needs at least one mapped group to stay active. A mapping with no roles means "access only". Sources that push no groups (the Google SCIM beta) have no mappings, so this rule does not apply to them. | +| 8 | One active bearer token per connection, stored hashed, with an expiry. Rotation creates a new token and shortens the previous token's expiry to **one hour** from the rotation, so the old and new tokens overlap briefly. | +| 9 | Keep all the data. The full SCIM payload, enterprise extension included, is stored on the user's link. Odd vendor extensions are normalized or ignored as they come up. | +| 10 | Provision / update / deprovision / group-membership changes emit platform events. | +| 11 | External (portal) users: a per-connection **Provision as External users** setting, off by default. An External user (`User.is_external`) keeps the same org cascade but is not covered by `access_level="authenticated"` sharing; they see only what a role or an explicit grant gives them. | +| 12 | The provisioning core is protocol-independent. Connections have `source_type` = `scim` \| `google_directory`. The UI is the same for both: groups seen, "N/N mapped", last sync, credential expiry. | + +## Data model + +All new tables are identity entities (see `api/src/repositories/README.md`). +They are looked up by ID within their org and are never resolved by name with +cascade. + +- `organization_domains`: `id`, `organization_id` (FK, cascade), `domain` + (lower-case), `verification_token`, `verified_at`, `last_checked_at`, + `created_by`, timestamps. Unique `(organization_id, domain)`, plus a partial + unique index on `domain WHERE verified_at IS NOT NULL`. +- `identity_providers`: `id`, `organization_id` (FK, not null), `name`, + `source_type`, `vendor` (`entra` \| `okta` \| `jumpcloud` \| `google` \| + `other`, for display only), `is_enabled`, `provision_as_external`, + `settings` JSONB, `last_request_at`, `last_sync_at`, `created_by`, + timestamps. +- `identity_provider_tokens`: `id`, `identity_provider_id` (FK, cascade), + `token_hash` (unique), `token_hint` (last 4 characters, for display), + `expires_at`, `revoked_at`, `last_used_at`, `created_by`, `created_at`. +- `identity_provider_users`: the link. `id` is the SCIM `id`, kept stable for + the IdP. Other columns: `identity_provider_id` (FK, cascade), `user_id` (FK, + unique, so a user has one identity source), `external_id`, `user_name`, + `active`, `attributes` JSONB (the last full payload), `created_user` + (whether this connection created the user), timestamps. Unique + `(identity_provider_id, external_id)`. +- `identity_provider_groups`: `id` is the SCIM `id`. Other columns: + `identity_provider_id`, `external_id`, `display_name`, `attributes` JSONB, + timestamps. +- `identity_provider_group_members`: `(group_id, identity_provider_user_id)`. +- `identity_provider_group_roles`: `(group_id, role_id)`. A "mapping" is a row + in `identity_provider_group_mappings` (`group_id` PK, `created_by`, + `created_at`), and its roles hang off it, so an access-only mapping is a + mapping with no roles. +- `user_roles.identity_provider_id`: nullable FK, `ON DELETE SET NULL`. Set + when provisioning created the assignment. If the user already held the + role, nothing is recorded, so a manual grant always survives. Clearing it is + how a manual edit or a deleted connection takes ownership. + +A user's **Identity Source** is the linked connection's name, or "Native" if +the user has no link. + +## Provisioning core (`api/src/services/identity_provisioning/`) + +Inputs are protocol-neutral commands: upsert user, set active, delete user, +upsert group, set or add or remove group members, delete group. Every command +runs inside the request transaction and: + +1. Checks the domain claim (decision 3) before any create or email change. +2. Resolves the user by link, then by `externalId`, then adopts by email in + the same org (decision 4). +3. Writes user fields through narrow helpers. The IdP owns `name`, `email` + and `is_active` for linked users; the UI shows them as managed. +4. Recomputes access and roles for the affected users: desired roles = the + union of mapped-group roles. It inserts missing roles, tagged with the + connection, and deletes tagged roles that are no longer wanted. Active = + `link.active` and (no mappings, or at least one mapped group). +5. Emits audit and events after the commit. + +The Google Directory sync (phase 4) calls the same commands with a full +snapshot. It adds snapshot-only guards: never deprovision after a failed or +partial fetch, and refuse a sync that would remove an unusually large share of +users at once. + +## SCIM 2.0 endpoint + +The base URL (the "Tenant URL" in Entra) is +`/api/scim/v2/{identity_provider_id}`. Requests carry +`Authorization: Bearer `, which must belong to that connection, must +not be expired or revoked, and the connection must be enabled. Responses use +`application/scim+json`, with SCIM `Error` bodies. + +- Discovery: `ServiceProviderConfig`, `ResourceTypes`, `Schemas`. +- `Users`: list with filters, `POST` (returns `201`), `GET`, `PUT`, `PATCH`, + `DELETE`. +- `Groups`: the same set, plus `excludedAttributes=members`. +- Filters: only what the IdPs send, `userName|externalId|id eq` on Users and + `displayName|externalId|id eq` on Groups. Anything else gets + `400 invalidFilter`. Paging uses `startIndex` / `count`. +- Not supported (advertised in `ServiceProviderConfig`): bulk, sort, etag, + changePassword. +- PATCH dialects: Entra's capitalized ops, string booleans (`"False"`), + pathless value objects and `members[value eq "…"]` removals, plus Okta's + pathed and unpathed forms. +- Hardening: CSRF exemption for the prefix, a per-connection rate limit + (`src/core/rate_limit.py`) and a body-size cap. + +Libraries: +- `scim2-models` (Apache-2.0, Pydantic v2, pinned) for resource and message + validation and serialization. +- `dnspython`, now a direct dependency (it was transitive), for TXT lookups. +- `scim2-tester` runs RFC compliance checks in the e2e suite. +- No filter-grammar dependency; the supported subset is small and explicit. + +## Management API and permissions + +- New permission domain `identityproviders`, under "Identity & Access". + `read` and `readwrite` are checked at the connection's organization. +- Group-to-role mappings also require that the caller could assign those + roles at that org (the manual role-assignment privilege check). This stops + a mapping from being used to escalate privilege. +- Routes: + - `/api/identity-providers`: list, create, get, update, delete. + - `.../{id}/tokens`: rotate; the token is returned once. + - `.../{id}/groups`: list, with mapping status. + - `.../{id}/groups/{group_id}/mapping`: upsert or delete. + - `/api/organizations/{org_id}/domains`: list, add, verify, delete. +- Every route is registered in the operation catalog and the access list. +- `UserPublic` gains `identity_source` (`{ identity_provider_id, name, + vendor }` or null). + +## Events + +Topics `user.provisioned`, `user.updated`, `user.deprovisioned` and +`user.groups_changed`, registered in `CURATED_TOPICS` with the standard +`_base_body` envelope. The actor is the connection. + +## UI + +- Users view gains an **Identity Providers** tab. The list columns are Name, + Organization, Vendor, Verified Domains, Provisioning Status, Credential + Expiry, Users and Groups Mapped. +- The detail page has: + - an expiry banner (expired or within 14 days); + - a provisioning card (status, Tenant URL with copy, token hint and expiry, + Regenerate with a one-time reveal, last request); + - a domains card (claims with TXT instructions and Verify); + - a group-mapping table (group → mapping status → roles; edit). +- The edit dialog uses a left-hand section nav: General, Provisioning, Group + Mapping. +- The users list gains an Identity Source column in both the wide and narrow + layouts. Managed fields are locked on user detail. + +## Phases + +1. **Backend foundation:** migration and models, domain claims and DNS + verification, connection and token management, the provisioning core, the + SCIM endpoint, events, audit, the permission domain, and tests (unit, e2e, + scim2-tester). +2. **UI:** list, detail and edit dialog, the domains card, the token reveal, + the mapping editor, and the Identity Source column; vitest plus one + Playwright happy path. +3. **Credential expiry warnings:** a daily scheduled task that re-raises admin + notifications (they are Redis-backed with a TTL) for tokens within 14 days + of expiry. +4. **Google Directory sync:** connections with `source_type="google_directory"`. + The customer supplies a service account with domain-wide delegation and + read-only Admin SDK scopes (`admin.directory.group.readonly`, plus + `admin.directory.user.readonly` for a full user pull). A recurring + scheduled **PlatformJob** per connection (see + `docs/architecture/platform-jobs.md`) reads `groups.list` / + `members.list`, plus an optional user pull for tenants without the SCIM + beta (this also covers secondary domains), and feeds the same core with the + snapshot guards. This is the only way to get Google group membership, + because Google's OIDC ID tokens carry no groups claim. +5. **Upstream RFC:** per-IdP sign-in consolidation, retiring or keeping the + legacy `Organization.domain` login auto-join in favour of verified claims, + and whether customer orgs may self-manage connections. + +## IdP notes + +- **Entra ID** (non-gallery app): + - Provisioning cycles run about every 40 minutes; use "Provision on demand" + for testing. + - It sends the PATCH quirks above. + - Validate with Microsoft's SCIM Validator. +- **Okta:** validate with Okta's SCIM test suite. +- **JumpCloud:** custom SCIM app. +- **Google Workspace custom SCIM** (limited beta; it works only where + "Autoprovisioning" appears): + - Users only: create, update, suspend (`active:false`) and `DELETE`, with + configurable delays. + - Bearer token only, and primary domain only. + - No groups; group selection only limits which users get provisioned. + - "Test connection" creates, updates and deletes a test user and expects + `201`. The test user must pass the domain claim. The never-signed-in + `DELETE` rule in decision 5 keeps it from leaving disabled users behind. + +## Open questions (for the RFC) + +- Deleting a connection: the proposal is to unlink its users (they become + Native) and convert the roles it granted to manual. The alternative is to + require disabling or removing the users first. +- How often to re-check verified domains, and what happens when a TXT record + disappears. +- Whether admins may override IdP-managed fields locally. From e9bebada2d0569745bb7c4f65c2b084d04b50e9f Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Mon, 16 Nov 2026 13:30:01 -0500 Subject: [PATCH 02/26] Add native SCIM provisioning for identity provider connections Organizations can connect an identity provider (Entra ID, Okta, JumpCloud, Google Workspace's SCIM beta) that provisions their users over SCIM 2.0 at /api/scim/v2/{identity_provider_id}: - Domain claims verified by DNS TXT gate which users a connection may create or change; a domain is verified by one organization only. - Existing same-org users are adopted by email; users of another org are a uniqueness conflict. - Directory groups map to roles. Provisioning tags the assignments it makes (user_roles.identity_provider_id) and only removes those, and once a connection has a mapping a user needs a mapped group to stay active. - DELETE disables and unlinks; a user the connection created who never signed in is removed outright. - Tokens are stored hashed and rotate with a one-hour overlap. The provisioning core is protocol-independent so a Google Directory sync can feed it later. scim2-models validates resources, applies PatchOp and parses filters; requests are normalized for Entra and Okta quirks first. Administration is a new identityproviders permission domain, evaluator-gated and catalogued; provisioning emits user.provisioned / updated / deprovisioned / groups_changed events and audit rows naming the connection. Co-Authored-By: Claude Opus 5.5 --- .../bifrost-build/generated/openapi-digest.md | 13 + .../bifrost-build/generated/operations.md | 13 + .../versions/20261116_identity_providers.py | 252 ++++++ api/shared/sdk_users.py | 12 +- api/src/core/app_wiring.py | 9 + api/src/core/csrf.py | 1 + api/src/main.py | 4 + .../models/contracts/identity_providers.py | 123 +++ api/src/models/contracts/permissions.py | 12 + api/src/models/contracts/users.py | 8 + api/src/models/orm/__init__.py | 18 + api/src/models/orm/identity_providers.py | 249 ++++++ api/src/models/orm/users.py | 6 + api/src/routers/identity_providers.py | 220 +++++ api/src/routers/scim.py | 465 ++++++++++ api/src/services/access_list.py | 30 + api/src/services/events/builtins.py | 38 + api/src/services/events/registry.py | 81 ++ api/src/services/identity_providers.py | 578 +++++++++++++ .../identity_provisioning/__init__.py | 7 + .../services/identity_provisioning/core.py | 708 ++++++++++++++++ .../services/identity_provisioning/domains.py | 155 ++++ .../services/identity_provisioning/scim.py | 570 +++++++++++++ .../services/identity_provisioning/tokens.py | 124 +++ api/src/services/operation_catalog.py | 191 +++++ api/src/services/operation_inventory.py | 9 + api/src/services/user_role_assignments.py | 8 +- api/tests/e2e/api/test_scim_provisioning.py | 218 +++++ .../test_identity_provisioning_core.py | 242 ++++++ .../test_identity_provisioning_scim.py | 150 ++++ .../unit/services/test_operation_catalog.py | 41 + .../unit/services/test_operation_inventory.py | 2 +- .../unit/test_org_scoping_enforcement.py | 5 + api/tests/unit/test_permissions.py | 1 + docs/generated/access-list.json | 420 +++++++++ .../operation-surface-inventory.json | 798 +++++++++++++++++- .../2026-11-16-scim-identity-providers.md | 16 +- pyproject.toml | 6 + requirements-dev.lock | 30 +- requirements.lock | 19 +- 40 files changed, 5837 insertions(+), 15 deletions(-) create mode 100644 api/alembic/versions/20261116_identity_providers.py create mode 100644 api/src/models/contracts/identity_providers.py create mode 100644 api/src/models/orm/identity_providers.py create mode 100644 api/src/routers/identity_providers.py create mode 100644 api/src/routers/scim.py create mode 100644 api/src/services/identity_providers.py create mode 100644 api/src/services/identity_provisioning/__init__.py create mode 100644 api/src/services/identity_provisioning/core.py create mode 100644 api/src/services/identity_provisioning/domains.py create mode 100644 api/src/services/identity_provisioning/scim.py create mode 100644 api/src/services/identity_provisioning/tokens.py create mode 100644 api/tests/e2e/api/test_scim_provisioning.py create mode 100644 api/tests/unit/services/test_identity_provisioning_core.py create mode 100644 api/tests/unit/services/test_identity_provisioning_scim.py diff --git a/.claude/skills/bifrost-build/generated/openapi-digest.md b/.claude/skills/bifrost-build/generated/openapi-digest.md index 27671bf71c..1c50c59078 100644 --- a/.claude/skills/bifrost-build/generated/openapi-digest.md +++ b/.claude/skills/bifrost-build/generated/openapi-digest.md @@ -300,6 +300,19 @@ | POST | `/api/identities` | | DELETE | `/api/identities/{identity_id}` | | PATCH | `/api/identities/{identity_id}` | +| GET | `/api/identity-providers` | +| POST | `/api/identity-providers` | +| GET | `/api/identity-providers/domains` | +| POST | `/api/identity-providers/domains` | +| DELETE | `/api/identity-providers/domains/{claim_id}` | +| POST | `/api/identity-providers/domains/{claim_id}/verify` | +| DELETE | `/api/identity-providers/{identity_provider_id}` | +| GET | `/api/identity-providers/{identity_provider_id}` | +| PATCH | `/api/identity-providers/{identity_provider_id}` | +| GET | `/api/identity-providers/{identity_provider_id}/groups` | +| DELETE | `/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping` | +| PUT | `/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping` | +| POST | `/api/identity-providers/{identity_provider_id}/token` | | GET | `/api/integrations` | | POST | `/api/integrations` | | GET | `/api/integrations/by-name/{name}` | diff --git a/.claude/skills/bifrost-build/generated/operations.md b/.claude/skills/bifrost-build/generated/operations.md index 379f030983..af2a90f811 100644 --- a/.claude/skills/bifrost-build/generated/operations.md +++ b/.claude/skills/bifrost-build/generated/operations.md @@ -187,3 +187,16 @@ caller's own items. | `workspace.files.editor.folder.create` | — | — | `repository.readwrite` | | `workspace.files.editor.delete` | — | — | `repository.readwrite` | | `workspace.files.editor.rename` | — | — | `repository.readwrite` | +| `identityproviders.domains.list` | — | — | `identityproviders.read` | +| `identityproviders.domains.create` | — | — | `identityproviders.readwrite` | +| `identityproviders.domains.verify` | — | — | `identityproviders.readwrite` | +| `identityproviders.domains.delete` | — | — | `identityproviders.readwrite` | +| `identityproviders.list` | — | — | `identityproviders.read` | +| `identityproviders.create` | — | — | `identityproviders.readwrite` | +| `identityproviders.get` | — | — | `identityproviders.read` | +| `identityproviders.update` | — | — | `identityproviders.readwrite` | +| `identityproviders.delete` | — | — | `identityproviders.readwrite` | +| `identityproviders.token.rotate` | — | — | `identityproviders.readwrite` | +| `identityproviders.groups.list` | — | — | `identityproviders.read` | +| `identityproviders.groups.map` | — | — | `identityproviders.readwrite`, `roleassignments.readwrite` | +| `identityproviders.groups.unmap` | — | — | `identityproviders.readwrite` | diff --git a/api/alembic/versions/20261116_identity_providers.py b/api/alembic/versions/20261116_identity_providers.py new file mode 100644 index 0000000000..dc94eb231c --- /dev/null +++ b/api/alembic/versions/20261116_identity_providers.py @@ -0,0 +1,252 @@ +"""Add identity provider connections, organization domain claims and the +directory state SCIM provisioning keeps in step. + +Revision ID: 20261116_identity_providers +Revises: 20261116_organization_tags +""" + +import sqlalchemy as sa +from alembic import op +from sqlalchemy.dialects.postgresql import JSONB, UUID + +revision = "20261116_identity_providers" +down_revision = "20261116_organization_tags" +branch_labels = None +depends_on = None + + +def _timestamps(*, updated: bool) -> list[sa.Column]: + columns = [ + sa.Column( + "created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("NOW()") + ) + ] + if updated: + columns.append( + sa.Column( + "updated_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.text("NOW()"), + ) + ) + return columns + + +def upgrade() -> None: + op.create_table( + "organization_domains", + sa.Column("id", UUID(as_uuid=True), primary_key=True), + sa.Column( + "organization_id", + UUID(as_uuid=True), + sa.ForeignKey("organizations.id", ondelete="CASCADE"), + nullable=False, + ), + sa.Column("domain", sa.String(253), nullable=False), + sa.Column("verification_token", sa.String(64), nullable=False), + sa.Column("verified_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("last_checked_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("created_by", sa.String(255), nullable=False), + *_timestamps(updated=False), + sa.UniqueConstraint("organization_id", "domain", name="uq_organization_domains_org_domain"), + sa.CheckConstraint("domain = lower(domain)", name="ck_organization_domains_lowercase"), + ) + op.create_index( + "uq_organization_domains_verified_domain", + "organization_domains", + ["domain"], + unique=True, + postgresql_where=sa.text("verified_at IS NOT NULL"), + ) + + op.create_table( + "identity_providers", + sa.Column("id", UUID(as_uuid=True), primary_key=True), + sa.Column( + "organization_id", + UUID(as_uuid=True), + sa.ForeignKey("organizations.id", ondelete="CASCADE"), + nullable=False, + ), + sa.Column("name", sa.String(255), nullable=False), + sa.Column("source_type", sa.String(32), nullable=False), + sa.Column("vendor", sa.String(32), nullable=False), + sa.Column("is_enabled", sa.Boolean(), nullable=False, server_default=sa.text("true")), + sa.Column( + "provision_as_external", sa.Boolean(), nullable=False, server_default=sa.text("false") + ), + sa.Column("settings", JSONB(), nullable=False, server_default=sa.text("'{}'::jsonb")), + sa.Column("last_request_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("last_sync_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("created_by", sa.String(255), nullable=False), + *_timestamps(updated=True), + sa.CheckConstraint( + "source_type IN ('scim', 'google_directory')", name="ck_identity_providers_source_type" + ), + sa.CheckConstraint( + "vendor IN ('entra', 'okta', 'jumpcloud', 'google', 'other')", + name="ck_identity_providers_vendor", + ), + ) + op.create_index( + "ix_identity_providers_organization_id", "identity_providers", ["organization_id"] + ) + + op.create_table( + "identity_provider_tokens", + sa.Column("id", UUID(as_uuid=True), primary_key=True), + sa.Column( + "identity_provider_id", + UUID(as_uuid=True), + sa.ForeignKey("identity_providers.id", ondelete="CASCADE"), + nullable=False, + ), + sa.Column("token_hash", sa.String(64), nullable=False, unique=True), + sa.Column("token_hint", sa.String(8), nullable=False), + sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("last_used_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("created_by", sa.String(255), nullable=False), + *_timestamps(updated=False), + ) + op.create_index( + "ix_identity_provider_tokens_identity_provider_id", + "identity_provider_tokens", + ["identity_provider_id"], + ) + + op.create_table( + "identity_provider_users", + sa.Column("id", UUID(as_uuid=True), primary_key=True), + sa.Column( + "identity_provider_id", + UUID(as_uuid=True), + sa.ForeignKey("identity_providers.id", ondelete="CASCADE"), + nullable=False, + ), + sa.Column( + "user_id", + UUID(as_uuid=True), + sa.ForeignKey("users.id", ondelete="CASCADE"), + nullable=False, + unique=True, + ), + sa.Column("external_id", sa.String(255), nullable=True), + sa.Column("user_name", sa.String(320), nullable=False), + sa.Column("active", sa.Boolean(), nullable=False), + sa.Column("attributes", JSONB(), nullable=False, server_default=sa.text("'{}'::jsonb")), + sa.Column("created_user", sa.Boolean(), nullable=False), + *_timestamps(updated=True), + sa.UniqueConstraint( + "identity_provider_id", "external_id", name="uq_identity_provider_users_external_id" + ), + sa.UniqueConstraint( + "identity_provider_id", "user_name", name="uq_identity_provider_users_user_name" + ), + ) + + op.create_table( + "identity_provider_groups", + sa.Column("id", UUID(as_uuid=True), primary_key=True), + sa.Column( + "identity_provider_id", + UUID(as_uuid=True), + sa.ForeignKey("identity_providers.id", ondelete="CASCADE"), + nullable=False, + ), + sa.Column("external_id", sa.String(255), nullable=True), + sa.Column("display_name", sa.String(255), nullable=False), + sa.Column("attributes", JSONB(), nullable=False, server_default=sa.text("'{}'::jsonb")), + *_timestamps(updated=True), + sa.UniqueConstraint( + "identity_provider_id", "external_id", name="uq_identity_provider_groups_external_id" + ), + sa.UniqueConstraint( + "identity_provider_id", + "display_name", + name="uq_identity_provider_groups_display_name", + ), + ) + + op.create_table( + "identity_provider_group_members", + sa.Column( + "group_id", + UUID(as_uuid=True), + sa.ForeignKey("identity_provider_groups.id", ondelete="CASCADE"), + primary_key=True, + ), + sa.Column( + "identity_provider_user_id", + UUID(as_uuid=True), + sa.ForeignKey("identity_provider_users.id", ondelete="CASCADE"), + primary_key=True, + ), + ) + op.create_index( + "ix_identity_provider_group_members_user", + "identity_provider_group_members", + ["identity_provider_user_id"], + ) + + op.create_table( + "identity_provider_group_mappings", + sa.Column( + "group_id", + UUID(as_uuid=True), + sa.ForeignKey("identity_provider_groups.id", ondelete="CASCADE"), + primary_key=True, + ), + sa.Column("created_by", sa.String(255), nullable=False), + *_timestamps(updated=False), + ) + + op.create_table( + "identity_provider_group_roles", + sa.Column( + "group_id", + UUID(as_uuid=True), + sa.ForeignKey("identity_provider_group_mappings.group_id", ondelete="CASCADE"), + primary_key=True, + ), + sa.Column( + "role_id", + UUID(as_uuid=True), + sa.ForeignKey("roles.id", ondelete="CASCADE"), + primary_key=True, + ), + ) + + op.add_column( + "user_roles", + sa.Column( + "identity_provider_id", + UUID(as_uuid=True), + sa.ForeignKey( + "identity_providers.id", + ondelete="SET NULL", + name="fk_user_roles_identity_provider_id", + ), + nullable=True, + ), + ) + + +def downgrade() -> None: + op.drop_constraint("fk_user_roles_identity_provider_id", "user_roles", type_="foreignkey") + op.drop_column("user_roles", "identity_provider_id") + op.drop_table("identity_provider_group_roles") + op.drop_table("identity_provider_group_mappings") + op.drop_index("ix_identity_provider_group_members_user", "identity_provider_group_members") + op.drop_table("identity_provider_group_members") + op.drop_table("identity_provider_groups") + op.drop_table("identity_provider_users") + op.drop_index( + "ix_identity_provider_tokens_identity_provider_id", "identity_provider_tokens" + ) + op.drop_table("identity_provider_tokens") + op.drop_index("ix_identity_providers_organization_id", "identity_providers") + op.drop_table("identity_providers") + op.drop_index("uq_organization_domains_verified_domain", "organization_domains") + op.drop_table("organization_domains") diff --git a/api/shared/sdk_users.py b/api/shared/sdk_users.py index 5d0d571588..5bb03e163a 100644 --- a/api/shared/sdk_users.py +++ b/api/shared/sdk_users.py @@ -235,11 +235,21 @@ async def _with_protection(session: AsyncSession, users: list) -> list["UserPubl from src.models import UserPublic from src.services.authorization.enforce import privileged_user_ids - privileged = await privileged_user_ids(session, [u.id for u in users]) + from src.models.contracts.identity_providers import IdentitySource + from src.services.identity_providers import identity_sources + + user_ids = [u.id for u in users] + privileged = await privileged_user_ids(session, user_ids) + sources = await identity_sources(session, user_ids) out: list[UserPublic] = [] for u in users: public = UserPublic.model_validate(u) public.is_protected = u.id in privileged + if u.id in sources: + provider_id, name, vendor = sources[u.id] + public.identity_source = IdentitySource( + identity_provider_id=provider_id, name=name, vendor=vendor # type: ignore[arg-type] + ) out.append(public) return out diff --git a/api/src/core/app_wiring.py b/api/src/core/app_wiring.py index e0d33022b7..a9984f9c77 100644 --- a/api/src/core/app_wiring.py +++ b/api/src/core/app_wiring.py @@ -67,6 +67,15 @@ async def app_compiler_unavailable_handler( ).model_dump(), ) + from src.services.identity_provisioning.scim import ScimError + + @app.exception_handler(ScimError) + async def scim_error_handler(request: Request, exc: ScimError) -> JSONResponse: + """SCIM endpoints answer with RFC 7644 Error bodies.""" + from src.routers.scim import scim_error_response + + return scim_error_response(exc) + @app.exception_handler(RequestValidationError) async def request_validation_handler( request: Request, exc: RequestValidationError diff --git a/api/src/core/csrf.py b/api/src/core/csrf.py index c1122f0a51..25bfd5ec3d 100644 --- a/api/src/core/csrf.py +++ b/api/src/core/csrf.py @@ -56,6 +56,7 @@ CSRF_EXEMPT_PREFIXES = ( "/api/hooks/", # Webhook receiver - called by external services, no auth "/api/endpoints/", # API key auth - called with X-Bifrost-Key header + "/api/scim/", # SCIM provisioning - bearer token from an identity provider "/embed/", # Embed entry points - HMAC-verified, no cookie auth ) diff --git a/api/src/main.py b/api/src/main.py index ebd065725f..4b88b26bef 100644 --- a/api/src/main.py +++ b/api/src/main.py @@ -55,6 +55,8 @@ from src.routers.health import close_health_check_clients from src.routers.health import router as health_router from src.routers.hooks import router as hooks_router +from src.routers.scim import router as scim_router +from src.routers.identity_providers import router as identity_providers_router from src.routers.identities import router as identities_router from src.routers.base_role_defaults import router as base_role_defaults_router from src.routers.integrations import router as integrations_router @@ -506,6 +508,8 @@ def create_app() -> FastAPI: app.include_router(events_router) app.include_router(services_router) app.include_router(hooks_router) + app.include_router(scim_router) + app.include_router(identity_providers_router) app.include_router(tables_router) app.include_router(claims_router) app.include_router(solutions_router) diff --git a/api/src/models/contracts/identity_providers.py b/api/src/models/contracts/identity_providers.py new file mode 100644 index 0000000000..9c4e5e63d6 --- /dev/null +++ b/api/src/models/contracts/identity_providers.py @@ -0,0 +1,123 @@ +"""Request/response contracts for identity provider connections and +organization domain claims.""" + +from datetime import datetime +from typing import Literal +from uuid import UUID + +from pydantic import BaseModel, ConfigDict, Field + +IdentityProviderVendor = Literal["entra", "okta", "jumpcloud", "google", "other"] +# google_directory joins once the Directory sync lands; v1 creates SCIM only. +IdentityProviderSourceType = Literal["scim"] +TokenLifetimeDays = Literal[90, 180, 365, 730] + + +class OrganizationDomainCreate(BaseModel): + organization_id: UUID + domain: str = Field(min_length=1, max_length=253) + + +class OrganizationDomainResponse(BaseModel): + model_config = ConfigDict(from_attributes=True) + + id: UUID + organization_id: UUID + domain: str + verified_at: datetime | None + last_checked_at: datetime | None + # Publish a TXT record named ``txt_record_name`` with this exact value. + txt_record_name: str + txt_record_value: str + created_at: datetime + + +class IdentitySource(BaseModel): + """Where a user's identity comes from (None on a user means Native).""" + + identity_provider_id: UUID + name: str + vendor: IdentityProviderVendor + + +class IdentityProviderTokenInfo(BaseModel): + hint: str + expires_at: datetime + created_at: datetime + last_used_at: datetime | None + + +class IdentityProviderCreate(BaseModel): + organization_id: UUID + name: str = Field(min_length=1, max_length=255) + vendor: IdentityProviderVendor + source_type: IdentityProviderSourceType = "scim" + provision_as_external: bool = False + token_lifetime_days: TokenLifetimeDays = 365 + + +class IdentityProviderUpdate(BaseModel): + name: str | None = Field(default=None, min_length=1, max_length=255) + vendor: IdentityProviderVendor | None = None + is_enabled: bool | None = None + provision_as_external: bool | None = None + + +class IdentityProviderResponse(BaseModel): + id: UUID + organization_id: UUID + organization_name: str + name: str + source_type: str + vendor: IdentityProviderVendor + is_enabled: bool + provision_as_external: bool + # The SCIM "Tenant URL" to configure in the identity provider. + scim_base_url: str + token: IdentityProviderTokenInfo | None + verified_domains: list[str] + user_count: int + group_count: int + mapped_group_count: int + last_request_at: datetime | None + last_sync_at: datetime | None + created_at: datetime + updated_at: datetime + + +class IdentityProviderTokenRotate(BaseModel): + lifetime_days: TokenLifetimeDays = 365 + + +class IdentityProviderTokenIssued(BaseModel): + """The raw token is returned only here, once.""" + + token: str + info: IdentityProviderTokenInfo + # Earlier tokens stop working at this time (the rotation overlap). + previous_tokens_expire_at: datetime | None + + +class IdentityProviderCreated(BaseModel): + identity_provider: IdentityProviderResponse + token: IdentityProviderTokenIssued + + +class RoleSummary(BaseModel): + id: UUID + name: str + + +class IdentityProviderGroupResponse(BaseModel): + id: UUID + display_name: str + external_id: str | None + member_count: int + mapped: bool + roles: list[RoleSummary] + + +class IdentityProviderGroupMappingUpdate(BaseModel): + """Map a group. An empty ``role_ids`` maps it for access only.""" + + role_ids: list[UUID] = Field(default_factory=list, max_length=50) diff --git a/api/src/models/contracts/permissions.py b/api/src/models/contracts/permissions.py index 48fe2b1e26..8602166b00 100644 --- a/api/src/models/contracts/permissions.py +++ b/api/src/models/contracts/permissions.py @@ -94,6 +94,18 @@ class PermissionDomain(BaseModel): who_should_hold="Platform admins. Others only through an admin-assigned role at a specific boundary; never the User base role. The Platform Operator role gets user support at Managed organizations, never Impersonate Users.", verbs={"impersonate": "Impersonate Users"}, ), + "identityproviders": PermissionDomain( + title="Identity Providers", + area="Identity & Access", + description=( + "Identity provider connections that provision an organization's " + "users from an external directory (SCIM), their tokens, the " + "organization's verified email domains, and the mapping of " + "directory groups to roles. Mapping a group to a role also needs " + "Role Assignments for that role." + ), + who_should_hold="Platform admins. Others only through an admin-assigned role at a specific boundary; never the User base role.", + ), "userlifecycle": PermissionDomain( title="User Lifecycle", area="Identity & Access", diff --git a/api/src/models/contracts/users.py b/api/src/models/contracts/users.py index 72643bbb9a..6e4ec5495e 100644 --- a/api/src/models/contracts/users.py +++ b/api/src/models/contracts/users.py @@ -9,6 +9,7 @@ from pydantic import BaseModel, ConfigDict, EmailStr, Field, field_serializer, model_validator from src.models.contracts.role_assignments import RoleBoundaryPublic +from src.models.contracts.identity_providers import IdentitySource if TYPE_CHECKING: pass @@ -138,6 +139,13 @@ class UserPublic(UserBase): "Admin), so only a Platform Admin can change them." ), ) + identity_source: IdentitySource | None = Field( + default=None, + description=( + "The identity provider connection that manages this user; null " + "for a Native user." + ), + ) @field_serializer("created_at", "updated_at", "last_login") def serialize_dt(self, dt: datetime | None) -> str | None: diff --git a/api/src/models/orm/__init__.py b/api/src/models/orm/__init__.py index f10aa92360..f13f093e18 100644 --- a/api/src/models/orm/__init__.py +++ b/api/src/models/orm/__init__.py @@ -70,6 +70,16 @@ from src.models.orm.tables import Document, Table from src.models.orm.users import Role, RolePermission, User, UserRole, UserRoleBoundary from src.models.orm.user_invites import UserInvite +from src.models.orm.identity_providers import ( + IdentityProvider, + IdentityProviderGroup, + IdentityProviderGroupMapping, + IdentityProviderGroupMember, + IdentityProviderGroupRole, + IdentityProviderToken, + IdentityProviderUser, + OrganizationDomain, +) from src.models.orm.workflow_operation_usage import WorkflowOperationUsage from src.models.orm.workflow_permissions import WorkflowPermissionGrant from src.models.orm.workflow_roles import WorkflowRole @@ -127,6 +137,14 @@ "UserRole", "UserRoleBoundary", "UserInvite", + "IdentityProvider", + "IdentityProviderGroup", + "IdentityProviderGroupMapping", + "IdentityProviderGroupMember", + "IdentityProviderGroupRole", + "IdentityProviderToken", + "IdentityProviderUser", + "OrganizationDomain", "HumanAuthSession", "ProductUpdateReceipt", # Agent Runs diff --git a/api/src/models/orm/identity_providers.py b/api/src/models/orm/identity_providers.py new file mode 100644 index 0000000000..01c325c2ab --- /dev/null +++ b/api/src/models/orm/identity_providers.py @@ -0,0 +1,249 @@ +"""Identity provider connections, organization domain claims and the +directory state an identity provider keeps in step with Bifrost. + +All of these are identity entities (see ``api/src/repositories/README.md``): +they belong to one organization and are looked up by id, never resolved by +name through the org-to-global cascade. +""" + +from datetime import datetime, timezone +from uuid import UUID, uuid4 + +from sqlalchemy import ( + Boolean, + CheckConstraint, + DateTime, + ForeignKey, + Index, + String, + UniqueConstraint, + text, +) +from sqlalchemy.dialects.postgresql import JSONB +from sqlalchemy.orm import Mapped, mapped_column + +from src.models.orm.base import Base + + +def _now() -> datetime: + return datetime.now(timezone.utc) + + +class OrganizationDomain(Base): + """An email domain an organization claims, proven by a DNS TXT record. + + A domain may be claimed (pending) by several organizations but verified + by only one: the partial unique index enforces it at the database. + """ + + __tablename__ = "organization_domains" + + id: Mapped[UUID] = mapped_column(primary_key=True, default=uuid4) + organization_id: Mapped[UUID] = mapped_column( + ForeignKey("organizations.id", ondelete="CASCADE"), nullable=False + ) + domain: Mapped[str] = mapped_column(String(253), nullable=False) + verification_token: Mapped[str] = mapped_column(String(64), nullable=False) + verified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None) + last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None) + created_by: Mapped[str] = mapped_column(String(255), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), default=_now, server_default=text("NOW()") + ) + + __table_args__ = ( + UniqueConstraint("organization_id", "domain", name="uq_organization_domains_org_domain"), + Index( + "uq_organization_domains_verified_domain", + "domain", + unique=True, + postgresql_where=text("verified_at IS NOT NULL"), + ), + CheckConstraint("domain = lower(domain)", name="ck_organization_domains_lowercase"), + ) + + +class IdentityProvider(Base): + """One organization's connection to an external directory.""" + + __tablename__ = "identity_providers" + + id: Mapped[UUID] = mapped_column(primary_key=True, default=uuid4) + organization_id: Mapped[UUID] = mapped_column( + ForeignKey("organizations.id", ondelete="CASCADE"), nullable=False + ) + name: Mapped[str] = mapped_column(String(255), nullable=False) + source_type: Mapped[str] = mapped_column(String(32), nullable=False) + vendor: Mapped[str] = mapped_column(String(32), nullable=False) + is_enabled: Mapped[bool] = mapped_column( + Boolean, nullable=False, default=True, server_default=text("true") + ) + provision_as_external: Mapped[bool] = mapped_column( + Boolean, nullable=False, default=False, server_default=text("false") + ) + settings: Mapped[dict] = mapped_column( + JSONB, nullable=False, default=dict, server_default=text("'{}'::jsonb") + ) + last_request_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None) + last_sync_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None) + created_by: Mapped[str] = mapped_column(String(255), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), default=_now, server_default=text("NOW()") + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), default=_now, server_default=text("NOW()"), onupdate=_now + ) + + __table_args__ = ( + Index("ix_identity_providers_organization_id", "organization_id"), + CheckConstraint( + "source_type IN ('scim', 'google_directory')", + name="ck_identity_providers_source_type", + ), + CheckConstraint( + "vendor IN ('entra', 'okta', 'jumpcloud', 'google', 'other')", + name="ck_identity_providers_vendor", + ), + ) + + +class IdentityProviderToken(Base): + """A bearer credential for a connection. Only the SHA-256 digest is kept; + the raw token is shown once, when it is generated.""" + + __tablename__ = "identity_provider_tokens" + + id: Mapped[UUID] = mapped_column(primary_key=True, default=uuid4) + identity_provider_id: Mapped[UUID] = mapped_column( + ForeignKey("identity_providers.id", ondelete="CASCADE"), nullable=False + ) + token_hash: Mapped[str] = mapped_column(String(64), nullable=False, unique=True) + token_hint: Mapped[str] = mapped_column(String(8), nullable=False) + expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) + revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None) + last_used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None) + created_by: Mapped[str] = mapped_column(String(255), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), default=_now, server_default=text("NOW()") + ) + + __table_args__ = ( + Index("ix_identity_provider_tokens_identity_provider_id", "identity_provider_id"), + ) + + +class IdentityProviderUser(Base): + """Links a Bifrost user to the directory entry a connection manages. + + ``id`` is the SCIM resource id handed to the identity provider. A user has + at most one identity source (``user_id`` is unique). + """ + + __tablename__ = "identity_provider_users" + + id: Mapped[UUID] = mapped_column(primary_key=True, default=uuid4) + identity_provider_id: Mapped[UUID] = mapped_column( + ForeignKey("identity_providers.id", ondelete="CASCADE"), nullable=False + ) + user_id: Mapped[UUID] = mapped_column( + ForeignKey("users.id", ondelete="CASCADE"), nullable=False, unique=True + ) + external_id: Mapped[str | None] = mapped_column(String(255), default=None) + user_name: Mapped[str] = mapped_column(String(320), nullable=False) + active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True) + # The last full resource the provider sent, extensions included. + attributes: Mapped[dict] = mapped_column( + JSONB, nullable=False, default=dict, server_default=text("'{}'::jsonb") + ) + # True when this connection created the Bifrost user (vs adopting one). + created_user: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), default=_now, server_default=text("NOW()") + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), default=_now, server_default=text("NOW()"), onupdate=_now + ) + + __table_args__ = ( + UniqueConstraint( + "identity_provider_id", "external_id", name="uq_identity_provider_users_external_id" + ), + UniqueConstraint( + "identity_provider_id", "user_name", name="uq_identity_provider_users_user_name" + ), + ) + + +class IdentityProviderGroup(Base): + """A group the identity provider pushed. ``id`` is the SCIM resource id.""" + + __tablename__ = "identity_provider_groups" + + id: Mapped[UUID] = mapped_column(primary_key=True, default=uuid4) + identity_provider_id: Mapped[UUID] = mapped_column( + ForeignKey("identity_providers.id", ondelete="CASCADE"), nullable=False + ) + external_id: Mapped[str | None] = mapped_column(String(255), default=None) + display_name: Mapped[str] = mapped_column(String(255), nullable=False) + attributes: Mapped[dict] = mapped_column( + JSONB, nullable=False, default=dict, server_default=text("'{}'::jsonb") + ) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), default=_now, server_default=text("NOW()") + ) + updated_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), default=_now, server_default=text("NOW()"), onupdate=_now + ) + + __table_args__ = ( + UniqueConstraint( + "identity_provider_id", "external_id", name="uq_identity_provider_groups_external_id" + ), + UniqueConstraint( + "identity_provider_id", "display_name", name="uq_identity_provider_groups_display_name" + ), + ) + + +class IdentityProviderGroupMember(Base): + __tablename__ = "identity_provider_group_members" + + group_id: Mapped[UUID] = mapped_column( + ForeignKey("identity_provider_groups.id", ondelete="CASCADE"), primary_key=True + ) + identity_provider_user_id: Mapped[UUID] = mapped_column( + ForeignKey("identity_provider_users.id", ondelete="CASCADE"), primary_key=True + ) + + __table_args__ = ( + Index( + "ix_identity_provider_group_members_user", + "identity_provider_user_id", + ), + ) + + +class IdentityProviderGroupMapping(Base): + """Marks a group as mapped. A mapping with no roles grants access only.""" + + __tablename__ = "identity_provider_group_mappings" + + group_id: Mapped[UUID] = mapped_column( + ForeignKey("identity_provider_groups.id", ondelete="CASCADE"), primary_key=True + ) + created_by: Mapped[str] = mapped_column(String(255), nullable=False) + created_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), default=_now, server_default=text("NOW()") + ) + + +class IdentityProviderGroupRole(Base): + __tablename__ = "identity_provider_group_roles" + + group_id: Mapped[UUID] = mapped_column( + ForeignKey("identity_provider_group_mappings.group_id", ondelete="CASCADE"), + primary_key=True, + ) + role_id: Mapped[UUID] = mapped_column( + ForeignKey("roles.id", ondelete="CASCADE"), primary_key=True + ) diff --git a/api/src/models/orm/users.py b/api/src/models/orm/users.py index fa285cffe7..f6c3f059e3 100644 --- a/api/src/models/orm/users.py +++ b/api/src/models/orm/users.py @@ -176,6 +176,12 @@ class UserRole(Base): assigned_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), default=lambda: datetime.now(timezone.utc), server_default=text("NOW()") ) + # Set when identity provisioning created this assignment from a group + # mapping; provisioning only ever removes assignments it created. NULL is + # a manual assignment (a manual edit or a deleted connection clears it). + identity_provider_id: Mapped[UUID | None] = mapped_column( + ForeignKey("identity_providers.id", ondelete="SET NULL"), default=None + ) # Relationships user: Mapped["User"] = relationship(back_populates="roles") diff --git a/api/src/routers/identity_providers.py b/api/src/routers/identity_providers.py new file mode 100644 index 0000000000..36fddf4b43 --- /dev/null +++ b/api/src/routers/identity_providers.py @@ -0,0 +1,220 @@ +"""Identity provider connections, their tokens and group mappings, and the +organization domain claims they provision against.""" + +from __future__ import annotations + +from uuid import UUID + +from fastapi import APIRouter, Query, Response + +from src.core.auth import CurrentActiveUser +from src.core.db_deps import DbSession +from src.models.contracts.identity_providers import ( + IdentityProviderCreate, + IdentityProviderCreated, + IdentityProviderGroupMappingUpdate, + IdentityProviderGroupResponse, + IdentityProviderResponse, + IdentityProviderTokenIssued, + IdentityProviderTokenRotate, + IdentityProviderUpdate, + OrganizationDomainCreate, + OrganizationDomainResponse, +) +from src.services import identity_providers as service +from src.services.authorization.enforce import load_caller +from src.services.operation_catalog import operation_route + +router = APIRouter(prefix="/api/identity-providers", tags=["Identity Providers"]) + + +# --------------------------------------------------------------------------- +# Domain claims (declared before /{identity_provider_id}) +# --------------------------------------------------------------------------- + + +@router.get( + "/domains", + response_model=list[OrganizationDomainResponse], + **operation_route("identityproviders.domains.list"), +) +async def list_domains( + db: DbSession, user: CurrentActiveUser, organization_id: UUID = Query(...) +) -> list[OrganizationDomainResponse]: + caller = await load_caller(db, user) + return await service.list_domains(db, caller, organization_id) + + +@router.post( + "/domains", + response_model=OrganizationDomainResponse, + status_code=201, + **operation_route("identityproviders.domains.create"), +) +async def add_domain( + request: OrganizationDomainCreate, db: DbSession, user: CurrentActiveUser +) -> OrganizationDomainResponse: + caller = await load_caller(db, user) + return await service.add_domain(db, caller, request.organization_id, request.domain) + + +@router.post( + "/domains/{claim_id}/verify", + response_model=OrganizationDomainResponse, + **operation_route("identityproviders.domains.verify"), +) +async def verify_domain( + claim_id: UUID, db: DbSession, user: CurrentActiveUser +) -> OrganizationDomainResponse: + caller = await load_caller(db, user) + return await service.verify_domain(db, caller, claim_id) + + +@router.delete( + "/domains/{claim_id}", + status_code=204, + **operation_route("identityproviders.domains.delete"), +) +async def delete_domain(claim_id: UUID, db: DbSession, user: CurrentActiveUser) -> Response: + caller = await load_caller(db, user) + await service.delete_domain(db, caller, claim_id) + return Response(status_code=204) + + +# --------------------------------------------------------------------------- +# Connections +# --------------------------------------------------------------------------- + + +@router.get( + "", + response_model=list[IdentityProviderResponse], + **operation_route("identityproviders.list"), +) +async def list_identity_providers( + db: DbSession, user: CurrentActiveUser, organization_id: UUID | None = Query(None) +) -> list[IdentityProviderResponse]: + caller = await load_caller(db, user) + return await service.list_connections(db, caller, organization_id) + + +@router.post( + "", + response_model=IdentityProviderCreated, + status_code=201, + **operation_route("identityproviders.create"), +) +async def create_identity_provider( + request: IdentityProviderCreate, db: DbSession, user: CurrentActiveUser +) -> IdentityProviderCreated: + caller = await load_caller(db, user) + return await service.create_connection(db, caller, request) + + +@router.get( + "/{identity_provider_id}", + response_model=IdentityProviderResponse, + **operation_route("identityproviders.get"), +) +async def get_identity_provider( + identity_provider_id: UUID, db: DbSession, user: CurrentActiveUser +) -> IdentityProviderResponse: + caller = await load_caller(db, user) + return await service.get_connection(db, caller, identity_provider_id) + + +@router.patch( + "/{identity_provider_id}", + response_model=IdentityProviderResponse, + **operation_route("identityproviders.update"), +) +async def update_identity_provider( + identity_provider_id: UUID, + request: IdentityProviderUpdate, + db: DbSession, + user: CurrentActiveUser, +) -> IdentityProviderResponse: + caller = await load_caller(db, user) + return await service.update_connection(db, caller, identity_provider_id, request) + + +@router.delete( + "/{identity_provider_id}", + status_code=204, + **operation_route("identityproviders.delete"), +) +async def delete_identity_provider( + identity_provider_id: UUID, db: DbSession, user: CurrentActiveUser +) -> Response: + caller = await load_caller(db, user) + await service.delete_connection(db, caller, identity_provider_id) + return Response(status_code=204) + + +@router.post( + "/{identity_provider_id}/token", + response_model=IdentityProviderTokenIssued, + **operation_route("identityproviders.token.rotate"), +) +async def rotate_identity_provider_token( + identity_provider_id: UUID, + request: IdentityProviderTokenRotate, + db: DbSession, + user: CurrentActiveUser, +) -> IdentityProviderTokenIssued: + caller = await load_caller(db, user) + return await service.rotate_token(db, caller, identity_provider_id, request.lifetime_days) + + +# --------------------------------------------------------------------------- +# Groups and mappings +# --------------------------------------------------------------------------- + + +@router.get( + "/{identity_provider_id}/groups", + response_model=list[IdentityProviderGroupResponse], + **operation_route("identityproviders.groups.list"), +) +async def list_identity_provider_groups( + identity_provider_id: UUID, db: DbSession, user: CurrentActiveUser +) -> list[IdentityProviderGroupResponse]: + caller = await load_caller(db, user) + return await service.list_groups(db, caller, identity_provider_id) + + +@router.put( + "/{identity_provider_id}/groups/{group_id}/mapping", + response_model=IdentityProviderGroupResponse, + **operation_route("identityproviders.groups.map"), +) +async def map_identity_provider_group( + identity_provider_id: UUID, + group_id: UUID, + request: IdentityProviderGroupMappingUpdate, + db: DbSession, + user: CurrentActiveUser, +) -> IdentityProviderGroupResponse: + caller = await load_caller(db, user) + response, provisioner = await service.set_group_mapping( + db, caller, identity_provider_id, group_id, request.role_ids + ) + await db.commit() + await provisioner.emit_pending_events() + return response + + +@router.delete( + "/{identity_provider_id}/groups/{group_id}/mapping", + status_code=204, + **operation_route("identityproviders.groups.unmap"), +) +async def unmap_identity_provider_group( + identity_provider_id: UUID, group_id: UUID, db: DbSession, user: CurrentActiveUser +) -> Response: + caller = await load_caller(db, user) + provisioner = await service.delete_group_mapping(db, caller, identity_provider_id, group_id) + await db.commit() + await provisioner.emit_pending_events() + return Response(status_code=204) + diff --git a/api/src/routers/scim.py b/api/src/routers/scim.py new file mode 100644 index 0000000000..caed8a8f79 --- /dev/null +++ b/api/src/routers/scim.py @@ -0,0 +1,465 @@ +"""SCIM 2.0 service provider endpoint for identity provider connections. + +Base URL (the "Tenant URL" an identity provider is configured with): +``/api/scim/v2/{identity_provider_id}``. Requests authenticate with the +connection's bearer token. Translation lives in +``src.services.identity_provisioning.scim``; the rules live in the +protocol-independent core. +""" + +from __future__ import annotations + +import json +from typing import Annotated, Any +from uuid import UUID + +from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response, status +from fastapi.responses import JSONResponse +from sqlalchemy import func, select + +from src.core.db_deps import DbSession +from src.core.rate_limit import RateLimiter +from src.models.orm.identity_providers import ( + IdentityProvider, + IdentityProviderGroup, + IdentityProviderGroupMember, + IdentityProviderUser, +) +from src.services.identity_provisioning import scim, tokens +from src.services.identity_provisioning.core import ProvisioningError, Provisioner +from src.services.identity_provisioning.scim import ScimError + +router = APIRouter( + prefix="/api/scim/v2/{identity_provider_id}", + tags=["SCIM"], + include_in_schema=False, +) + +# Generous: a large Entra or Okta cycle sends bursts of user and membership +# changes. Applied per connection, before any database work. +_RATE_LIMIT = RateLimiter(max_requests=1200, window_seconds=60) +_MAX_BODY_BYTES = 1024 * 1024 + + +def scim_response(body: Any, status_code: int = 200, *, location: str | None = None) -> Response: + headers = {"Location": location} if location else None + return JSONResponse( + body, status_code=status_code, media_type=scim.SCIM_MEDIA_TYPE, headers=headers + ) + + +def scim_error_response(exc: ScimError) -> JSONResponse: + headers = {"WWW-Authenticate": 'Bearer realm="SCIM"'} if exc.status == 401 else None + return JSONResponse( + exc.body(), status_code=exc.status, media_type=scim.SCIM_MEDIA_TYPE, headers=headers + ) + + +async def scim_connection( + identity_provider_id: str, request: Request, db: DbSession +) -> IdentityProvider: + scheme, _, token = request.headers.get("authorization", "").partition(" ") + if scheme.lower() != "bearer" or not token.strip(): + raise ScimError(401, "A bearer token is required") + try: + connection_id = UUID(identity_provider_id) + except ValueError: + raise ScimError(401, "Invalid or expired token") from None + try: + await _RATE_LIMIT.check("scim", str(connection_id)) + except HTTPException as exc: + if exc.status_code == status.HTTP_429_TOO_MANY_REQUESTS: + raise ScimError(429, "Too many requests") from None + raise + connection = await tokens.authenticate(db, connection_id, token.strip()) + if connection is None or connection.source_type != "scim": + raise ScimError(401, "Invalid or expired token") + return connection + + +Connection = Annotated[IdentityProvider, Depends(scim_connection)] + + +def _base_url(connection: IdentityProvider) -> str: + from src.config import get_settings + + return f"{get_settings().public_url.rstrip('/')}/api/scim/v2/{connection.id}" + + +async def _json_body(request: Request) -> Any: + raw = await request.body() + if len(raw) > _MAX_BODY_BYTES: + raise ScimError(413, "The request body is too large", "tooLarge") + try: + return json.loads(raw) + except ValueError: + raise ScimError(400, "The request body is not valid JSON", "invalidSyntax") from None + + +def _paging(start_index: int, count: int | None) -> tuple[int, int]: + start = max(start_index, 1) + limit = scim.MAX_RESULTS if count is None else max(0, min(count, scim.MAX_RESULTS)) + return start, limit + + +async def _commit_and_emit(db: DbSession, provisioner: Provisioner) -> None: + await db.commit() + await provisioner.emit_pending_events() + + +def _parse_id(resource_id: str) -> UUID: + try: + return UUID(resource_id) + except ValueError: + raise ScimError(404, "Resource not found") from None + + +# --------------------------------------------------------------------------- +# Discovery +# --------------------------------------------------------------------------- + + +@router.get("/ServiceProviderConfig") +async def service_provider_config(connection: Connection) -> Response: + return scim_response(scim.service_provider_config(_base_url(connection))) + + +@router.get("/ResourceTypes") +async def resource_types(connection: Connection) -> Response: + types = scim.resource_types(_base_url(connection)) + return scim_response(scim.list_response(types, len(types), 1)) + + +@router.get("/ResourceTypes/{type_id}") +async def resource_type(type_id: str, connection: Connection) -> Response: + for item in scim.resource_types(_base_url(connection)): + if item["id"] == type_id: + return scim_response(item) + raise ScimError(404, "Resource type not found") + + +@router.get("/Schemas") +async def schemas(connection: Connection) -> Response: + items = scim.schemas(_base_url(connection)) + return scim_response(scim.list_response(items, len(items), 1)) + + +@router.get("/Schemas/{schema_id}") +async def schema(schema_id: str, connection: Connection) -> Response: + for item in scim.schemas(_base_url(connection)): + if item["id"] == schema_id: + return scim_response(item) + raise ScimError(404, "Schema not found") + + +# --------------------------------------------------------------------------- +# Users +# --------------------------------------------------------------------------- + + +async def _user_link(provisioner: Provisioner, user_id: str) -> IdentityProviderUser: + link = await provisioner.get_user_link(_parse_id(user_id)) + if link is None: + raise ScimError(404, "User not found") + return link + + +@router.get("/Users") +async def list_users( + connection: Connection, + db: DbSession, + filter_: Annotated[str | None, Query(alias="filter")] = None, + start_index: Annotated[int, Query(alias="startIndex")] = 1, + count: int | None = None, + attributes: str | None = None, + excluded_attributes: Annotated[str | None, Query(alias="excludedAttributes")] = None, +) -> Response: + criteria = scim.parse_filter(filter_, scim.USER_FILTER_ATTRIBUTES) + stmt = select(IdentityProviderUser).where( + IdentityProviderUser.identity_provider_id == connection.id + ) + if criteria is not None: + if criteria.attribute == "userName": + # userName is case-insensitive (RFC 7643 §4.1.1). + stmt = stmt.where( + func.lower(IdentityProviderUser.user_name) == criteria.value.lower() + ) + elif criteria.attribute == "externalId": + stmt = stmt.where(IdentityProviderUser.external_id == criteria.value) + else: + try: + stmt = stmt.where(IdentityProviderUser.id == UUID(criteria.value)) + except ValueError: + return scim_response(scim.list_response([], 0, 1)) + start, limit = _paging(start_index, count) + total = await db.scalar(select(func.count()).select_from(stmt.subquery())) or 0 + links = ( + await db.scalars( + stmt.order_by(IdentityProviderUser.created_at, IdentityProviderUser.id) + .offset(start - 1) + .limit(limit) + ) + ).all() + base_url = _base_url(connection) + resources = [ + scim.project(scim.user_response(link, base_url), attributes, excluded_attributes) + for link in links + ] + return scim_response(scim.list_response(resources, total, start)) + + +@router.post("/Users") +async def create_user(request: Request, connection: Connection, db: DbSession) -> Response: + resource, unknown = scim.parse_user(await _json_body(request)) + provisioner = Provisioner(db, connection) + try: + link = await provisioner.create_user(scim.user_record(resource, unknown)) + except ProvisioningError as exc: + raise scim.from_provisioning_error(exc) from None + await db.refresh(link) + base_url = _base_url(connection) + body = scim.user_response(link, base_url) + await _commit_and_emit(db, provisioner) + return scim_response(body, 201, location=body["meta"]["location"]) + + +@router.get("/Users/{user_id}") +async def get_user( + user_id: str, + connection: Connection, + db: DbSession, + attributes: str | None = None, + excluded_attributes: Annotated[str | None, Query(alias="excludedAttributes")] = None, +) -> Response: + link = await _user_link(Provisioner(db, connection), user_id) + body = scim.user_response(link, _base_url(connection)) + return scim_response(scim.project(body, attributes, excluded_attributes)) + + +@router.put("/Users/{user_id}") +async def replace_user( + user_id: str, request: Request, connection: Connection, db: DbSession +) -> Response: + provisioner = Provisioner(db, connection) + link = await _user_link(provisioner, user_id) + resource, unknown = scim.parse_user(await _json_body(request)) + try: + link = await provisioner.replace_user(link, scim.user_record(resource, unknown)) + except ProvisioningError as exc: + raise scim.from_provisioning_error(exc) from None + await db.refresh(link) + body = scim.user_response(link, _base_url(connection)) + await _commit_and_emit(db, provisioner) + return scim_response(body) + + +@router.patch("/Users/{user_id}") +async def patch_user( + user_id: str, request: Request, connection: Connection, db: DbSession +) -> Response: + provisioner = Provisioner(db, connection) + link = await _user_link(provisioner, user_id) + resource, unknown = scim.apply_user_patch(link, await _json_body(request)) + try: + link = await provisioner.replace_user(link, scim.user_record(resource, unknown)) + except ProvisioningError as exc: + raise scim.from_provisioning_error(exc) from None + await db.refresh(link) + body = scim.user_response(link, _base_url(connection)) + await _commit_and_emit(db, provisioner) + return scim_response(body) + + +@router.delete("/Users/{user_id}") +async def delete_user(user_id: str, connection: Connection, db: DbSession) -> Response: + provisioner = Provisioner(db, connection) + link = await _user_link(provisioner, user_id) + try: + await provisioner.delete_user(link) + except ProvisioningError as exc: + raise scim.from_provisioning_error(exc) from None + await _commit_and_emit(db, provisioner) + return Response(status_code=204) + + +# --------------------------------------------------------------------------- +# Groups +# --------------------------------------------------------------------------- + + +async def _group(provisioner: Provisioner, group_id: str) -> IdentityProviderGroup: + group = await provisioner.get_group(_parse_id(group_id)) + if group is None: + raise ScimError(404, "Group not found") + return group + + +def _wants_members(attributes: str | None, excluded_attributes: str | None) -> bool: + excluded = {a.strip().lower() for a in (excluded_attributes or "").split(",")} + included = {a.strip().split(".")[0].lower() for a in (attributes or "").split(",") if a.strip()} + if "members" in excluded: + return False + return not included or "members" in included + + +async def _members(db: DbSession, group_ids: list[UUID]) -> dict[UUID, list[tuple[UUID, str]]]: + members: dict[UUID, list[tuple[UUID, str]]] = {group_id: [] for group_id in group_ids} + if not group_ids: + return members + rows = await db.execute( + select( + IdentityProviderGroupMember.group_id, + IdentityProviderUser.id, + IdentityProviderUser.user_name, + ) + .join( + IdentityProviderUser, + IdentityProviderUser.id == IdentityProviderGroupMember.identity_provider_user_id, + ) + .where(IdentityProviderGroupMember.group_id.in_(group_ids)) + .order_by(IdentityProviderUser.user_name) + ) + for group_id, link_id, user_name in rows.all(): + members[group_id].append((link_id, user_name)) + return members + + +async def _group_body( + db: DbSession, + group: IdentityProviderGroup, + base_url: str, + attributes: str | None = None, + excluded_attributes: str | None = None, +) -> dict[str, Any]: + members = None + if _wants_members(attributes, excluded_attributes): + members = (await _members(db, [group.id]))[group.id] + body = scim.group_response(group, members, base_url) + return scim.project(body, attributes, excluded_attributes) + + +@router.get("/Groups") +async def list_groups( + connection: Connection, + db: DbSession, + filter_: Annotated[str | None, Query(alias="filter")] = None, + start_index: Annotated[int, Query(alias="startIndex")] = 1, + count: int | None = None, + attributes: str | None = None, + excluded_attributes: Annotated[str | None, Query(alias="excludedAttributes")] = None, +) -> Response: + criteria = scim.parse_filter(filter_, scim.GROUP_FILTER_ATTRIBUTES) + stmt = select(IdentityProviderGroup).where( + IdentityProviderGroup.identity_provider_id == connection.id + ) + if criteria is not None: + if criteria.attribute == "displayName": + stmt = stmt.where(IdentityProviderGroup.display_name == criteria.value) + elif criteria.attribute == "externalId": + stmt = stmt.where(IdentityProviderGroup.external_id == criteria.value) + else: + try: + stmt = stmt.where(IdentityProviderGroup.id == UUID(criteria.value)) + except ValueError: + return scim_response(scim.list_response([], 0, 1)) + start, limit = _paging(start_index, count) + total = await db.scalar(select(func.count()).select_from(stmt.subquery())) or 0 + groups = ( + await db.scalars( + stmt.order_by(IdentityProviderGroup.created_at, IdentityProviderGroup.id) + .offset(start - 1) + .limit(limit) + ) + ).all() + base_url = _base_url(connection) + members: dict[UUID, list[tuple[UUID, str]]] = {} + if _wants_members(attributes, excluded_attributes): + members = await _members(db, [group.id for group in groups]) + resources = [ + scim.project( + scim.group_response(group, members.get(group.id) if members else None, base_url), + attributes, + excluded_attributes, + ) + for group in groups + ] + return scim_response(scim.list_response(resources, total, start)) + + +@router.post("/Groups") +async def create_group(request: Request, connection: Connection, db: DbSession) -> Response: + resource = scim.parse_group(await _json_body(request)) + provisioner = Provisioner(db, connection) + try: + group = await provisioner.create_group( + scim.group_record(resource), scim.group_member_ids(resource) + ) + except ProvisioningError as exc: + raise scim.from_provisioning_error(exc) from None + await db.refresh(group) + body = await _group_body(db, group, _base_url(connection)) + await _commit_and_emit(db, provisioner) + return scim_response(body, 201, location=body["meta"]["location"]) + + +@router.get("/Groups/{group_id}") +async def get_group( + group_id: str, + connection: Connection, + db: DbSession, + attributes: str | None = None, + excluded_attributes: Annotated[str | None, Query(alias="excludedAttributes")] = None, +) -> Response: + group = await _group(Provisioner(db, connection), group_id) + body = await _group_body( + db, group, _base_url(connection), attributes, excluded_attributes + ) + return scim_response(body) + + +@router.put("/Groups/{group_id}") +async def replace_group( + group_id: str, request: Request, connection: Connection, db: DbSession +) -> Response: + provisioner = Provisioner(db, connection) + group = await _group(provisioner, group_id) + resource = scim.parse_group(await _json_body(request)) + try: + await provisioner.replace_group(group, scim.group_record(resource)) + await provisioner.set_group_members(group, scim.group_member_ids(resource)) + except ProvisioningError as exc: + raise scim.from_provisioning_error(exc) from None + await db.refresh(group) + body = await _group_body(db, group, _base_url(connection)) + await _commit_and_emit(db, provisioner) + return scim_response(body) + + +@router.patch("/Groups/{group_id}") +async def patch_group( + group_id: str, request: Request, connection: Connection, db: DbSession +) -> Response: + provisioner = Provisioner(db, connection) + group = await _group(provisioner, group_id) + current_members = await provisioner.group_member_ids(group.id) + resource = scim.apply_group_patch(group, current_members, await _json_body(request)) + try: + await provisioner.replace_group(group, scim.group_record(resource)) + await provisioner.set_group_members(group, scim.group_member_ids(resource)) + except ProvisioningError as exc: + raise scim.from_provisioning_error(exc) from None + await _commit_and_emit(db, provisioner) + # Entra and Okta ignore a PATCH body; 204 avoids serializing large groups. + return Response(status_code=204) + + +@router.delete("/Groups/{group_id}") +async def delete_group(group_id: str, connection: Connection, db: DbSession) -> Response: + provisioner = Provisioner(db, connection) + group = await _group(provisioner, group_id) + try: + await provisioner.delete_group(group) + except ProvisioningError as exc: + raise scim.from_provisioning_error(exc) from None + await _commit_and_emit(db, provisioner) + return Response(status_code=204) diff --git a/api/src/services/access_list.py b/api/src/services/access_list.py index 4895608682..16550400e1 100644 --- a/api/src/services/access_list.py +++ b/api/src/services/access_list.py @@ -243,6 +243,19 @@ class available (`personal` / `own_private_agent` / `table_policy` before AccessEntry(method='GET', path='/api/chat/conversations/{conversation_id}/messages', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='chat.execute', boundary='organization', operation_id=None, reason="User's own chat conversation, message, or artifact.", intended_change=None), AccessEntry(method='POST', path='/api/chat/conversations/{conversation_id}/messages', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='chat.execute', boundary='organization', operation_id=None, reason='Original-user chat ownership; no caller flag widens it.', intended_change=None), AccessEntry(method='GET', path='/api/chat/conversations/{conversation_id}/state', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='chat.execute', boundary='organization', operation_id=None, reason="User's own chat conversation, message, or artifact.", intended_change=None), + AccessEntry(method='GET', path='/api/identity-providers', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.read', boundary='organization', operation_id='identityproviders.list', reason='List identity provider connections. Decided by the evaluator (src.services.authorization.enforce): Filtered to organizations where the caller holds identityproviders.read.', intended_change=None), + AccessEntry(method='POST', path='/api/identity-providers', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.readwrite', boundary='organization', operation_id='identityproviders.create', reason='Create an identity provider connection and its first token. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the organization.', intended_change=None), + AccessEntry(method='GET', path='/api/identity-providers/domains', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.read', boundary='organization', operation_id='identityproviders.domains.list', reason="List an organization's claimed email domains. Decided by the evaluator (src.services.authorization.enforce): identityproviders.read at the organization.", intended_change=None), + AccessEntry(method='POST', path='/api/identity-providers/domains', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.readwrite', boundary='organization', operation_id='identityproviders.domains.create', reason='Claim an email domain for an organization. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the organization.', intended_change=None), + AccessEntry(method='DELETE', path='/api/identity-providers/domains/{claim_id}', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.readwrite', boundary='organization', operation_id='identityproviders.domains.delete', reason="Remove a domain claim. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the claim's organization.", intended_change=None), + AccessEntry(method='POST', path='/api/identity-providers/domains/{claim_id}/verify', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.readwrite', boundary='organization', operation_id='identityproviders.domains.verify', reason="Verify a domain claim through its DNS TXT record. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the claim's organization.", intended_change=None), + AccessEntry(method='DELETE', path='/api/identity-providers/{identity_provider_id}', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.readwrite', boundary='organization', operation_id='identityproviders.delete', reason="Delete an identity provider connection. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the connection's organization.", intended_change=None), + AccessEntry(method='GET', path='/api/identity-providers/{identity_provider_id}', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.read', boundary='organization', operation_id='identityproviders.get', reason="Read an identity provider connection. Decided by the evaluator (src.services.authorization.enforce): identityproviders.read at the connection's organization.", intended_change=None), + AccessEntry(method='PATCH', path='/api/identity-providers/{identity_provider_id}', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.readwrite', boundary='organization', operation_id='identityproviders.update', reason="Update an identity provider connection. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the connection's organization.", intended_change=None), + AccessEntry(method='GET', path='/api/identity-providers/{identity_provider_id}/groups', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.read', boundary='organization', operation_id='identityproviders.groups.list', reason="List the groups an identity provider pushed. Decided by the evaluator (src.services.authorization.enforce): identityproviders.read at the connection's organization.", intended_change=None), + AccessEntry(method='DELETE', path='/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.readwrite', boundary='organization', operation_id='identityproviders.groups.unmap', reason="Remove an identity provider group mapping. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the connection's organization.", intended_change=None), + AccessEntry(method='PUT', path='/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.readwrite', boundary='organization', operation_id='identityproviders.groups.map', reason="Map an identity provider group to roles. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the connection's organization, plus the manual role-assignment checks for each mapped role.", intended_change=None), + AccessEntry(method='POST', path='/api/identity-providers/{identity_provider_id}/token', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='identityproviders.readwrite', boundary='organization', operation_id='identityproviders.token.rotate', reason="Issue a new identity provider token. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the connection's organization.", intended_change=None), AccessEntry(method='GET', path='/api/model-catalog/names', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.AUTHENTICATED, permission='settings.readbasic', boundary='organization', operation_id=None, reason='Reads public catalog display names for model ids already visible to the caller.', intended_change=None), AccessEntry(method='GET', path='/api/chat/model-profiles', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='chat.execute', boundary='organization', operation_id=None, reason="Reads the org's shared AI model-profile catalog.", intended_change=None), AccessEntry(method='POST', path='/api/chat/runs', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, inline_checks=('is_provider_org', 'is_superuser'), inline_effect=InlineEffect.NO_CALLER_EFFECT, permission='chat.execute', boundary='organization', operation_id=None, reason="Starts a chat run, which runs the conversation's agent.", intended_change=None), @@ -601,6 +614,23 @@ class available (`personal` / `own_private_agent` / `table_policy` before AccessEntry(method='POST', path='/api/roles/{role_id}/workflows', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.EVALUATOR, permission='workflows.readwrite', boundary='organization', operation_id='roles.workflows.assign', reason="Shares workflows with a role or stops sharing them; an entity-sharing write in that entity's own domain, not role authoring.", intended_change=None), AccessEntry(method='GET', path='/api/run-retention', access_class=AccessClass.PERSONAL, current_gate=CurrentGate.AUTHENTICATED, permission=None, boundary=None, operation_id=None, reason='Retention window for removed-run wording.', intended_change=None), AccessEntry(method='POST', path='/api/schedules/validate', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.AUTHENTICATED, permission='workflows.read', boundary='organization', operation_id=None, reason='Current original-user permission and reach; resource sharing, parent-resource launch, and Solution boundaries remain independent.', intended_change=None), + AccessEntry(method='GET', path='/api/scim/v2/{identity_provider_id}/Groups', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='POST', path='/api/scim/v2/{identity_provider_id}/Groups', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='DELETE', path='/api/scim/v2/{identity_provider_id}/Groups/{group_id}', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='GET', path='/api/scim/v2/{identity_provider_id}/Groups/{group_id}', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='PATCH', path='/api/scim/v2/{identity_provider_id}/Groups/{group_id}', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='PUT', path='/api/scim/v2/{identity_provider_id}/Groups/{group_id}', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='GET', path='/api/scim/v2/{identity_provider_id}/ResourceTypes', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='GET', path='/api/scim/v2/{identity_provider_id}/ResourceTypes/{type_id}', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='GET', path='/api/scim/v2/{identity_provider_id}/Schemas', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='GET', path='/api/scim/v2/{identity_provider_id}/Schemas/{schema_id}', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='GET', path='/api/scim/v2/{identity_provider_id}/ServiceProviderConfig', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='GET', path='/api/scim/v2/{identity_provider_id}/Users', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='POST', path='/api/scim/v2/{identity_provider_id}/Users', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='DELETE', path='/api/scim/v2/{identity_provider_id}/Users/{user_id}', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='GET', path='/api/scim/v2/{identity_provider_id}/Users/{user_id}', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='PATCH', path='/api/scim/v2/{identity_provider_id}/Users/{user_id}', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), + AccessEntry(method='PUT', path='/api/scim/v2/{identity_provider_id}/Users/{user_id}', access_class=AccessClass.PUBLIC, current_gate=CurrentGate.NONE, permission=None, boundary=None, operation_id=None, reason="SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session.", intended_change=None), AccessEntry(method='POST', path='/api/sdk/ai/complete', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.AUTHENTICATED, permission='ai.execute', boundary='organization', operation_id=None, reason='Runtime authority and original-user permission/reach; no caller flag bypass.', intended_change=None), AccessEntry(method='GET', path='/api/sdk/ai/info', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.AUTHENTICATED, permission='ai.read', boundary='organization', operation_id=None, reason='Lists AI models and profiles for the SDK; returns no secrets.', intended_change=None), AccessEntry(method='POST', path='/api/sdk/ai/stream', access_class=AccessClass.PERMISSION, current_gate=CurrentGate.AUTHENTICATED, permission='ai.execute', boundary='organization', operation_id=None, reason='Runtime authority and original-user permission/reach; no caller flag bypass.', intended_change=None), diff --git a/api/src/services/events/builtins.py b/api/src/services/events/builtins.py index 06ce77d61d..8591d98a0b 100644 --- a/api/src/services/events/builtins.py +++ b/api/src/services/events/builtins.py @@ -455,3 +455,41 @@ async def emit_event_delivery_retry_exhausted( body, organization_id=organization_id, ) + + +def identity_provider_user_body( + *, + organization_id: str | UUID, + identity_provider: dict[str, str | None], + user: dict[str, Any], +) -> dict[str, Any]: + """Body for the ``user.*`` topics identity provisioning emits; the actor + is the identity provider connection.""" + return { + **_base_body( + organization_id=organization_id, + actor={ + "type": "identity_provider", + "id": identity_provider["id"], + "email": None, + "name": identity_provider["name"], + }, + ), + "identity_provider": identity_provider, + "user": user, + } + + +async def emit_identity_provider_event( + topic: str, + body: dict[str, Any], + *, + organization_id: str | UUID, + identity_provider_id: str | UUID, +) -> None: + await _emit( + topic, + body, + organization_id=organization_id, + triggered_by=f"identity_provider:{identity_provider_id}", + ) diff --git a/api/src/services/events/registry.py b/api/src/services/events/registry.py index f44c0ae02c..f112874506 100644 --- a/api/src/services/events/registry.py +++ b/api/src/services/events/registry.py @@ -21,6 +21,31 @@ "name": "Admin User", } +_IDENTITY_PROVIDER = { + "id": "550e8400-e29b-41d4-a716-446655440050", + "name": "Contoso Entra ID", + "vendor": "entra", +} + +_IDENTITY_PROVIDER_ACTOR = { + "type": "identity_provider", + "id": _IDENTITY_PROVIDER["id"], + "email": None, + "name": _IDENTITY_PROVIDER["name"], +} + +_PROVISIONED_USER = { + "id": "550e8400-e29b-41d4-a716-446655440051", + "email": "alice@contoso.com", + "name": "Alice Example", + "is_active": True, + "external_id": "6f1c2d3e-0000-4000-8000-000000000001", + "attributes": { + "userName": "alice@contoso.com", + "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User": {"department": "Support"}, + }, +} + _BASE_BODY = { "schema_version": 1, "occurred_at": "2026-05-28T12:34:56Z", @@ -56,6 +81,62 @@ def _body(*, actor: dict, **fields: object) -> dict: }, ), }, + { + "topic": "user.provisioned", + "description": "Fired when an identity provider creates a user or adopts an existing one.", + "category": "Users", + "emitted_by": "Identity provisioning", + "example_body": _body( + actor=_IDENTITY_PROVIDER_ACTOR, + identity_provider=_IDENTITY_PROVIDER, + user=_PROVISIONED_USER, + adopted=False, + ), + }, + { + "topic": "user.updated", + "description": "Fired when an identity provider changes a user it manages.", + "category": "Users", + "emitted_by": "Identity provisioning", + "example_body": _body( + actor=_IDENTITY_PROVIDER_ACTOR, + identity_provider=_IDENTITY_PROVIDER, + user=_PROVISIONED_USER, + ), + }, + { + "topic": "user.deprovisioned", + "description": ( + "Fired when an identity provider deactivates or deletes a user, or the " + "user loses their last mapped group." + ), + "category": "Users", + "emitted_by": "Identity provisioning", + "example_body": _body( + actor=_IDENTITY_PROVIDER_ACTOR, + identity_provider=_IDENTITY_PROVIDER, + user={**_PROVISIONED_USER, "is_active": False}, + deprovision={"reason": "inactive", "account_removed": False}, + ), + }, + { + "topic": "user.groups_changed", + "description": "Fired when an identity provider adds a user to a group or removes them.", + "category": "Users", + "emitted_by": "Identity provisioning", + "example_body": _body( + actor=_IDENTITY_PROVIDER_ACTOR, + identity_provider=_IDENTITY_PROVIDER, + user=_PROVISIONED_USER, + group={ + "id": "550e8400-e29b-41d4-a716-446655440052", + "display_name": "Support Techs", + "change": "added", + }, + roles_added=["550e8400-e29b-41d4-a716-446655440053"], + roles_removed=[], + ), + }, { "topic": "workflow.failed", "description": "Fired when a workflow execution attempt fails.", diff --git a/api/src/services/identity_providers.py b/api/src/services/identity_providers.py new file mode 100644 index 0000000000..f08a558015 --- /dev/null +++ b/api/src/services/identity_providers.py @@ -0,0 +1,578 @@ +"""Administration of identity provider connections and domain claims. + +Every function takes the evaluator ``Caller`` the router loaded and checks +the organization-boundary permission itself, so a caller only ever sees or +changes connections in organizations where they hold ``identityproviders``. +Mapping a group to a role is a role grant, so it also passes the same checks +a manual role assignment does. +""" + +from __future__ import annotations + +from uuid import UUID + +from fastapi import HTTPException +from sqlalchemy import func, select +from sqlalchemy.ext.asyncio import AsyncSession + +from src.config import get_settings +from src.models.contracts.identity_providers import ( + IdentityProviderCreate, + IdentityProviderCreated, + IdentityProviderGroupResponse, + IdentityProviderResponse, + IdentityProviderTokenInfo, + IdentityProviderTokenIssued, + IdentityProviderUpdate, + OrganizationDomainResponse, + RoleSummary, +) +from src.models.orm.identity_providers import ( + IdentityProvider, + IdentityProviderGroup, + IdentityProviderGroupMapping, + IdentityProviderGroupMember, + IdentityProviderGroupRole, + IdentityProviderToken, + IdentityProviderUser, + OrganizationDomain, +) +from src.models.orm.organizations import Organization +from src.models.orm.users import Role +from src.services.audit import emit_audit +from src.services.authorization.enforce import ( + Caller, + cross_org, + operation_reach, + permitted_organizations, + require_operation, + require_permission, +) +from src.services.identity_provisioning import domains, tokens +from src.services.identity_provisioning.core import UNMAPPABLE_ROLE_IDS, Provisioner + +def scim_base_url(identity_provider_id: UUID) -> str: + return f"{get_settings().public_url.rstrip('/')}/api/scim/v2/{identity_provider_id}" + + +# --------------------------------------------------------------------------- +# Domain claims +# --------------------------------------------------------------------------- + + +def domain_response(claim: OrganizationDomain) -> OrganizationDomainResponse: + return OrganizationDomainResponse( + id=claim.id, + organization_id=claim.organization_id, + domain=claim.domain, + verified_at=claim.verified_at, + last_checked_at=claim.last_checked_at, + txt_record_name=claim.domain, + txt_record_value=domains.txt_record_value(claim), + created_at=claim.created_at, + ) + + +async def _require_organization(session: AsyncSession, organization_id: UUID) -> Organization: + organization = await session.get(Organization, organization_id) + if organization is None: + raise HTTPException(404, "Organization not found") + return organization + + +async def list_domains( + session: AsyncSession, caller: Caller, organization_id: UUID +) -> list[OrganizationDomainResponse]: + require_operation(caller, "identityproviders.domains.list", cross_org(organization_id)) + return [domain_response(c) for c in await domains.list_claims(session, organization_id)] + + +async def add_domain( + session: AsyncSession, caller: Caller, organization_id: UUID, domain: str +) -> OrganizationDomainResponse: + require_operation(caller, "identityproviders.domains.create", cross_org(organization_id)) + await _require_organization(session, organization_id) + try: + claim = await domains.add_claim( + session, organization_id, domain, created_by=caller.principal.email + ) + except domains.DomainClaimError as exc: + raise HTTPException(exc.status_code, exc.detail) from None + await emit_audit( + session, + "identity_provider.domain_claimed", + resource_type="organization", + resource_id=organization_id, + details={"domain": claim.domain}, + ) + return domain_response(claim) + + +async def _domain_claim(session: AsyncSession, claim_id: UUID) -> OrganizationDomain: + claim = await session.get(OrganizationDomain, claim_id) + if claim is None: + raise HTTPException(404, "Domain claim not found") + return claim + + +async def verify_domain( + session: AsyncSession, caller: Caller, claim_id: UUID +) -> OrganizationDomainResponse: + claim = await _domain_claim(session, claim_id) + require_operation(caller, "identityproviders.domains.verify", cross_org(claim.organization_id)) + was_verified = claim.verified_at is not None + try: + claim = await domains.verify_claim(session, claim) + except domains.DomainClaimError as exc: + raise HTTPException(exc.status_code, exc.detail) from None + if claim.verified_at is not None and not was_verified: + await emit_audit( + session, + "identity_provider.domain_verified", + resource_type="organization", + resource_id=claim.organization_id, + details={"domain": claim.domain}, + ) + return domain_response(claim) + + +async def delete_domain(session: AsyncSession, caller: Caller, claim_id: UUID) -> None: + claim = await _domain_claim(session, claim_id) + require_operation(caller, "identityproviders.domains.delete", cross_org(claim.organization_id)) + await session.delete(claim) + await session.flush() + await emit_audit( + session, + "identity_provider.domain_removed", + resource_type="organization", + resource_id=claim.organization_id, + details={"domain": claim.domain}, + ) + + +# --------------------------------------------------------------------------- +# Connections +# --------------------------------------------------------------------------- + + +def _token_info(token: IdentityProviderToken | None) -> IdentityProviderTokenInfo | None: + if token is None: + return None + return IdentityProviderTokenInfo( + hint=token.token_hint, + expires_at=token.expires_at, + created_at=token.created_at, + last_used_at=token.last_used_at, + ) + + +async def _responses( + session: AsyncSession, connections: list[IdentityProvider] +) -> list[IdentityProviderResponse]: + if not connections: + return [] + ids = [c.id for c in connections] + org_ids = {c.organization_id for c in connections} + org_names = dict( + (await session.execute( + select(Organization.id, Organization.name).where(Organization.id.in_(org_ids)) + )).all() + ) + verified: dict[UUID, list[str]] = {} + for org_id, domain in ( + await session.execute( + select(OrganizationDomain.organization_id, OrganizationDomain.domain) + .where( + OrganizationDomain.organization_id.in_(org_ids), + OrganizationDomain.verified_at.is_not(None), + ) + .order_by(OrganizationDomain.domain) + ) + ).all(): + verified.setdefault(org_id, []).append(domain) + user_counts = dict( + (await session.execute( + select(IdentityProviderUser.identity_provider_id, func.count()) + .where(IdentityProviderUser.identity_provider_id.in_(ids)) + .group_by(IdentityProviderUser.identity_provider_id) + )).all() + ) + group_counts = dict( + (await session.execute( + select(IdentityProviderGroup.identity_provider_id, func.count()) + .where(IdentityProviderGroup.identity_provider_id.in_(ids)) + .group_by(IdentityProviderGroup.identity_provider_id) + )).all() + ) + mapped_counts = dict( + (await session.execute( + select(IdentityProviderGroup.identity_provider_id, func.count()) + .join( + IdentityProviderGroupMapping, + IdentityProviderGroupMapping.group_id == IdentityProviderGroup.id, + ) + .where(IdentityProviderGroup.identity_provider_id.in_(ids)) + .group_by(IdentityProviderGroup.identity_provider_id) + )).all() + ) + current_tokens = {c.id: await tokens.current_token(session, c.id) for c in connections} + return [ + IdentityProviderResponse( + id=c.id, + organization_id=c.organization_id, + organization_name=org_names.get(c.organization_id, ""), + name=c.name, + source_type=c.source_type, + vendor=c.vendor, # type: ignore[arg-type] # DB check constraint + is_enabled=c.is_enabled, + provision_as_external=c.provision_as_external, + scim_base_url=scim_base_url(c.id), + token=_token_info(current_tokens[c.id]), + verified_domains=verified.get(c.organization_id, []), + user_count=user_counts.get(c.id, 0), + group_count=group_counts.get(c.id, 0), + mapped_group_count=mapped_counts.get(c.id, 0), + last_request_at=c.last_request_at, + last_sync_at=c.last_sync_at, + created_at=c.created_at, + updated_at=c.updated_at, + ) + for c in connections + ] + + +async def _connection( + session: AsyncSession, caller: Caller, identity_provider_id: UUID, *, operation: str +) -> IdentityProvider: + connection = await session.get(IdentityProvider, identity_provider_id) + if connection is None: + raise HTTPException(404, "Identity provider not found") + require_operation(caller, operation, cross_org(connection.organization_id)) + return connection + + +async def list_connections( + session: AsyncSession, caller: Caller, organization_id: UUID | None +) -> list[IdentityProviderResponse]: + stmt = select(IdentityProvider).order_by(IdentityProvider.name) + if organization_id is not None: + require_operation(caller, "identityproviders.list", cross_org(organization_id)) + stmt = stmt.where(IdentityProvider.organization_id == organization_id) + else: + reach = operation_reach(caller, "identityproviders.list").where( + IdentityProvider.organization_id + ) + if reach is not None: + stmt = stmt.where(reach) + return await _responses(session, list((await session.scalars(stmt)).all())) + + +async def get_connection( + session: AsyncSession, caller: Caller, identity_provider_id: UUID +) -> IdentityProviderResponse: + connection = await _connection( + session, caller, identity_provider_id, operation="identityproviders.get" + ) + return (await _responses(session, [connection]))[0] + + +async def create_connection( + session: AsyncSession, caller: Caller, request: IdentityProviderCreate +) -> IdentityProviderCreated: + require_operation(caller, "identityproviders.create", cross_org(request.organization_id)) + await _require_organization(session, request.organization_id) + connection = IdentityProvider( + organization_id=request.organization_id, + name=request.name.strip(), + source_type=request.source_type, + vendor=request.vendor, + provision_as_external=request.provision_as_external, + created_by=caller.principal.email, + ) + session.add(connection) + await session.flush() + raw, token = await tokens.issue_token( + session, + connection, + lifetime_days=request.token_lifetime_days, + created_by=caller.principal.email, + ) + await emit_audit( + session, + "identity_provider.create", + resource_type="identity_provider", + resource_id=connection.id, + details={"name": connection.name, "organization_id": str(connection.organization_id)}, + ) + await session.refresh(connection) + (response,) = await _responses(session, [connection]) + issued = IdentityProviderTokenIssued( + token=raw, info=_token_info(token), previous_tokens_expire_at=None # type: ignore[arg-type] + ) + return IdentityProviderCreated(identity_provider=response, token=issued) + + +async def update_connection( + session: AsyncSession, + caller: Caller, + identity_provider_id: UUID, + request: IdentityProviderUpdate, +) -> IdentityProviderResponse: + connection = await _connection( + session, caller, identity_provider_id, operation="identityproviders.update" + ) + changes = request.model_dump(exclude_unset=True, exclude_none=True) + if "name" in changes: + changes["name"] = changes["name"].strip() + for key, value in changes.items(): + setattr(connection, key, value) + await session.flush() + await emit_audit( + session, + "identity_provider.update", + resource_type="identity_provider", + resource_id=connection.id, + details={"changes": changes}, + ) + await session.refresh(connection) + return (await _responses(session, [connection]))[0] + + +async def delete_connection( + session: AsyncSession, caller: Caller, identity_provider_id: UUID +) -> None: + """Remove a connection. Its users stay as they are but become Native, and + the roles it granted become manual assignments (the FK sets NULL).""" + connection = await _connection( + session, caller, identity_provider_id, operation="identityproviders.delete" + ) + await session.delete(connection) + await session.flush() + await emit_audit( + session, + "identity_provider.delete", + resource_type="identity_provider", + resource_id=identity_provider_id, + details={"name": connection.name}, + ) + + +async def rotate_token( + session: AsyncSession, caller: Caller, identity_provider_id: UUID, lifetime_days: int +) -> IdentityProviderTokenIssued: + connection = await _connection( + session, caller, identity_provider_id, operation="identityproviders.token.rotate" + ) + previous = await tokens.current_token(session, connection.id) + raw, token = await tokens.issue_token( + session, connection, lifetime_days=lifetime_days, created_by=caller.principal.email + ) + previous_expiry = None + if previous is not None: + await session.refresh(previous) + previous_expiry = previous.expires_at + await emit_audit( + session, + "identity_provider.token_rotated", + resource_type="identity_provider", + resource_id=connection.id, + details={"lifetime_days": lifetime_days}, + ) + info = _token_info(token) + assert info is not None + return IdentityProviderTokenIssued( + token=raw, info=info, previous_tokens_expire_at=previous_expiry + ) + + +# --------------------------------------------------------------------------- +# Groups and mappings +# --------------------------------------------------------------------------- + + +async def list_groups( + session: AsyncSession, caller: Caller, identity_provider_id: UUID +) -> list[IdentityProviderGroupResponse]: + connection = await _connection( + session, caller, identity_provider_id, operation="identityproviders.groups.list" + ) + groups = ( + await session.scalars( + select(IdentityProviderGroup) + .where(IdentityProviderGroup.identity_provider_id == connection.id) + .order_by(IdentityProviderGroup.display_name) + ) + ).all() + group_ids = [g.id for g in groups] + member_counts = dict( + (await session.execute( + select(IdentityProviderGroupMember.group_id, func.count()) + .where(IdentityProviderGroupMember.group_id.in_(group_ids)) + .group_by(IdentityProviderGroupMember.group_id) + )).all() + ) if group_ids else {} + mapped = set( + await session.scalars( + select(IdentityProviderGroupMapping.group_id).where( + IdentityProviderGroupMapping.group_id.in_(group_ids) + ) + ) + ) if group_ids else set() + roles: dict[UUID, list[RoleSummary]] = {} + if group_ids: + for group_id, role_id, role_name in ( + await session.execute( + select(IdentityProviderGroupRole.group_id, Role.id, Role.name) + .join(Role, Role.id == IdentityProviderGroupRole.role_id) + .where(IdentityProviderGroupRole.group_id.in_(group_ids)) + .order_by(Role.name) + ) + ).all(): + roles.setdefault(group_id, []).append(RoleSummary(id=role_id, name=role_name)) + return [ + IdentityProviderGroupResponse( + id=g.id, + display_name=g.display_name, + external_id=g.external_id, + member_count=member_counts.get(g.id, 0), + mapped=g.id in mapped, + roles=roles.get(g.id, []), + ) + for g in groups + ] + + +async def _group( + session: AsyncSession, connection: IdentityProvider, group_id: UUID +) -> IdentityProviderGroup: + group = await session.get(IdentityProviderGroup, group_id) + if group is None or group.identity_provider_id != connection.id: + raise HTTPException(404, "Group not found") + return group + + +async def _authorize_mapped_roles( + session: AsyncSession, caller: Caller, organization_id: UUID, role_ids: set[UUID] +) -> None: + """The checks a manual assignment of each role at this org would pass.""" + from src.services.user_role_assignments import ( + RoleAssignmentError, + check_boundaries, + check_holder, + check_role_change, + default_boundaries, + load_roles, + ) + + if not role_ids: + return + unmappable = role_ids & UNMAPPABLE_ROLE_IDS + if unmappable: + raise HTTPException(422, "Base roles and platform roles can't be mapped from a group") + roles = await load_roles(session, role_ids) + missing = role_ids - roles.keys() + if missing: + raise HTTPException(422, f"Role {sorted(missing, key=str)[0]} not found") + require_permission(caller, "roleassignments.readwrite", organization_id) + reach = permitted_organizations(caller, "roleassignments.readwrite") + boundaries = default_boundaries(organization_id) + try: + for role in roles.values(): + check_role_change(caller, role, frozenset(), organization_id) + check_holder(role, organization_id) + check_boundaries(caller, reach, role, boundaries) + except RoleAssignmentError as exc: + raise HTTPException(exc.status_code, exc.detail) from None + + +async def set_group_mapping( + session: AsyncSession, + caller: Caller, + identity_provider_id: UUID, + group_id: UUID, + role_ids: list[UUID], +) -> tuple[IdentityProviderGroupResponse, Provisioner]: + """Map a group; returns the provisioner whose events the caller sends + after committing.""" + connection = await _connection( + session, caller, identity_provider_id, operation="identityproviders.groups.map" + ) + group = await _group(session, connection, group_id) + wanted = set(role_ids) + await _authorize_mapped_roles(session, caller, connection.organization_id, wanted) + mapping = await session.get(IdentityProviderGroupMapping, group.id) + if mapping is None: + session.add( + IdentityProviderGroupMapping(group_id=group.id, created_by=caller.principal.email) + ) + await session.flush() + current = set( + await session.scalars( + select(IdentityProviderGroupRole.role_id).where( + IdentityProviderGroupRole.group_id == group.id + ) + ) + ) + for role_id in current - wanted: + await session.delete(await session.get(IdentityProviderGroupRole, (group.id, role_id))) + for role_id in wanted - current: + session.add(IdentityProviderGroupRole(group_id=group.id, role_id=role_id)) + await session.flush() + await emit_audit( + session, + "identity_provider.group_mapped", + resource_type="identity_provider", + resource_id=connection.id, + details={ + "group": group.display_name, + "role_ids": sorted(str(r) for r in wanted), + }, + ) + provisioner = Provisioner(session, connection) + await provisioner.reconcile_all() + (response,) = [g for g in await list_groups(session, caller, connection.id) if g.id == group.id] + return response, provisioner + + +async def delete_group_mapping( + session: AsyncSession, caller: Caller, identity_provider_id: UUID, group_id: UUID +) -> Provisioner: + connection = await _connection( + session, caller, identity_provider_id, operation="identityproviders.groups.unmap" + ) + group = await _group(session, connection, group_id) + mapping = await session.get(IdentityProviderGroupMapping, group.id) + if mapping is None: + raise HTTPException(404, "This group is not mapped") + await session.delete(mapping) + await session.flush() + await emit_audit( + session, + "identity_provider.group_unmapped", + resource_type="identity_provider", + resource_id=connection.id, + details={"group": group.display_name}, + ) + provisioner = Provisioner(session, connection) + await provisioner.reconcile_all() + return provisioner + + +async def identity_sources( + session: AsyncSession, user_ids: list[UUID] +) -> dict[UUID, tuple[UUID, str, str]]: + """``user_id -> (identity_provider_id, name, vendor)`` for linked users.""" + if not user_ids: + return {} + rows = await session.execute( + select( + IdentityProviderUser.user_id, + IdentityProvider.id, + IdentityProvider.name, + IdentityProvider.vendor, + ) + .join(IdentityProvider, IdentityProvider.id == IdentityProviderUser.identity_provider_id) + .where(IdentityProviderUser.user_id.in_(user_ids)) + ) + return {user_id: (idp_id, name, vendor) for user_id, idp_id, name, vendor in rows.all()} + diff --git a/api/src/services/identity_provisioning/__init__.py b/api/src/services/identity_provisioning/__init__.py new file mode 100644 index 0000000000..f80971e41c --- /dev/null +++ b/api/src/services/identity_provisioning/__init__.py @@ -0,0 +1,7 @@ +"""Identity provisioning: keep Bifrost users and role assignments in step +with an external directory. + +``core`` is protocol-independent. ``scim`` translates SCIM 2.0 requests into +core commands; a Google Directory sync feeds the same core with snapshots. +See ``docs/plans/2026-11-16-scim-identity-providers.md``. +""" diff --git a/api/src/services/identity_provisioning/core.py b/api/src/services/identity_provisioning/core.py new file mode 100644 index 0000000000..106836c614 --- /dev/null +++ b/api/src/services/identity_provisioning/core.py @@ -0,0 +1,708 @@ +"""Protocol-independent provisioning commands for one identity provider +connection. + +Every command runs inside the caller's transaction. The rules it enforces: + +- A connection only creates or changes users whose email domain is a + verified claim of its organization, and never touches a user whose home is + another organization. +- An existing user in the same organization is adopted by email; one in + another organization is a conflict. +- The identity provider owns name, email and active for linked users. +- Roles come from group mappings. Provisioning only removes the role + assignments it created (``UserRole.identity_provider_id``), so a manual + grant always survives. +- Once a connection has at least one group mapping, a user needs at least one + mapped group to stay active. +- Deleting a user disables and unlinks them, except a user this connection + created who has never signed in: that account has no history to keep and + is removed permanently. + +Audit rows are written in the transaction; platform events are collected in +``pending_events`` and sent with :meth:`Provisioner.emit_pending_events` +after the caller commits. +""" + +from __future__ import annotations + +import enum +import logging +from collections.abc import Iterable +from dataclasses import dataclass, field +from datetime import datetime, timezone +from typing import Any +from uuid import UUID + +from sqlalchemy import delete, or_, select +from sqlalchemy.ext.asyncio import AsyncSession + +from shared.builtin_roles import ( + BASE_ROLE_IDS, + DECRYPTION_ROLE_ID, + PLATFORM_ADMIN_ROLE_ID, + PLATFORM_OPERATOR_ROLE_ID, +) +from src.core.log_safety import log_safe +from src.models.orm.identity_providers import ( + IdentityProvider, + IdentityProviderGroup, + IdentityProviderGroupMapping, + IdentityProviderGroupMember, + IdentityProviderGroupRole, + IdentityProviderUser, +) +from src.models.orm.users import User, UserRole +from src.services.audit_context import ActorContext +from src.services.identity_provisioning.domains import email_domain, verified_domains + +logger = logging.getLogger(__name__) + +# Roles no group mapping may ever grant: base roles are held through +# ``User.base_role_id``, and the platform-wide builtin roles are assigned by a +# person, never by a directory. +UNMAPPABLE_ROLE_IDS = frozenset( + {*BASE_ROLE_IDS, PLATFORM_ADMIN_ROLE_ID, PLATFORM_OPERATOR_ROLE_ID, DECRYPTION_ROLE_ID} +) + + +class ProvisioningErrorKind(enum.Enum): + CONFLICT = "conflict" + INVALID = "invalid" + DOMAIN_NOT_CLAIMED = "domain_not_claimed" + NOT_FOUND = "not_found" + + +class ProvisioningError(Exception): + def __init__(self, kind: ProvisioningErrorKind, detail: str): + super().__init__(detail) + self.kind = kind + self.detail = detail + + +@dataclass(frozen=True) +class UserRecord: + """A directory user, independent of the protocol that delivered it.""" + + user_name: str + email: str + external_id: str | None + display_name: str | None + active: bool + attributes: dict[str, Any] + + +@dataclass(frozen=True) +class GroupRecord: + display_name: str + external_id: str | None + attributes: dict[str, Any] + + +@dataclass +class _AccessChange: + activated: bool = False + deactivated: bool = False + roles_added: set[UUID] = field(default_factory=set) + roles_removed: set[UUID] = field(default_factory=set) + + +def normalize_email(value: str) -> str: + return value.strip().lower() + + +class Provisioner: + def __init__(self, session: AsyncSession, connection: IdentityProvider): + self.session = session + self.connection = connection + self.pending_events: list[tuple[str, dict[str, Any]]] = [] + self._actor = ActorContext( + user_id=None, + organization_id=connection.organization_id, + email=None, + name=f"Identity provider: {connection.name}", + source="identity_provider", + surface="service", + ) + + # ------------------------------------------------------------------ + # Lookups + # ------------------------------------------------------------------ + + async def get_user_link(self, link_id: UUID) -> IdentityProviderUser | None: + return await self.session.scalar( + select(IdentityProviderUser).where( + IdentityProviderUser.id == link_id, + IdentityProviderUser.identity_provider_id == self.connection.id, + ) + ) + + async def get_group(self, group_id: UUID) -> IdentityProviderGroup | None: + return await self.session.scalar( + select(IdentityProviderGroup).where( + IdentityProviderGroup.id == group_id, + IdentityProviderGroup.identity_provider_id == self.connection.id, + ) + ) + + async def group_member_ids(self, group_id: UUID) -> set[UUID]: + rows = await self.session.scalars( + select(IdentityProviderGroupMember.identity_provider_user_id).where( + IdentityProviderGroupMember.group_id == group_id + ) + ) + return set(rows) + + # ------------------------------------------------------------------ + # Users + # ------------------------------------------------------------------ + + async def create_user(self, record: UserRecord) -> IdentityProviderUser: + await self._refuse_duplicate_link(record, exclude_link_id=None) + email = normalize_email(record.email) + await self._require_claimed_domain(email) + + user = await self.session.scalar(select(User).where(User.email == email)) + created_user = user is None + if user is None: + user = await self._new_user(email, record) + else: + await self._require_adoptable(user) + user.name = record.display_name or user.name + + link = IdentityProviderUser( + identity_provider_id=self.connection.id, + user_id=user.id, + external_id=record.external_id, + user_name=record.user_name, + active=record.active, + attributes=record.attributes, + created_user=created_user, + ) + self.session.add(link) + await self.session.flush() + await self._reconcile_access([link.id]) + await self._audit( + "identity_provider.user_provisioned", + user, + {"email": email, "adopted": not created_user}, + ) + self._queue_user_event("user.provisioned", user, link, adopted=not created_user) + return link + + async def replace_user( + self, link: IdentityProviderUser, record: UserRecord + ) -> IdentityProviderUser: + await self._refuse_duplicate_link(record, exclude_link_id=link.id) + user = await self.session.get(User, link.user_id, with_for_update=True) + assert user is not None + email = normalize_email(record.email) + if email != user.email: + await self._require_claimed_domain(email) + taken = await self.session.scalar( + select(User.id).where(User.email == email, User.id != user.id) + ) + if taken is not None: + raise ProvisioningError( + ProvisioningErrorKind.CONFLICT, "Another user already has this email" + ) + user.email = email + user.name = record.display_name or user.name + link.external_id = record.external_id + link.user_name = record.user_name + link.active = record.active + link.attributes = record.attributes + await self.session.flush() + await self._reconcile_access([link.id]) + await self._audit("identity_provider.user_updated", user, {"email": user.email}) + self._queue_user_event("user.updated", user, link) + return link + + async def delete_user(self, link: IdentityProviderUser) -> None: + user = await self.session.get(User, link.user_id, with_for_update=True) + assert user is not None + if link.created_user and user.last_login is None: + from shared.run_cancellation import cancel_unstarted_runs_for_deleted_user + + body = self._user_event_body(user, link) + await cancel_unstarted_runs_for_deleted_user(self.session, user.id) + await self.session.delete(user) + await self.session.flush() + await self._audit( + "identity_provider.user_deleted", None, {"email": body["user"]["email"]}, + resource_id=body["user"]["id"], + ) + self.pending_events.append( + ("user.deprovisioned", {**body, "deprovision": {"reason": "deleted", "account_removed": True}}) + ) + return + + await self._set_active(user, False) + removed = await self._remove_granted_roles(user.id, None) + await self.session.delete(link) + await self.session.flush() + for role_id in removed: + await self._invalidate_role(role_id) + await self._audit("identity_provider.user_unlinked", user, {"email": user.email}) + self.pending_events.append( + ( + "user.deprovisioned", + { + **self._user_event_body(user, None), + "deprovision": {"reason": "deleted", "account_removed": False}, + }, + ) + ) + + # ------------------------------------------------------------------ + # Groups + # ------------------------------------------------------------------ + + async def create_group( + self, record: GroupRecord, member_link_ids: Iterable[UUID] + ) -> IdentityProviderGroup: + await self._refuse_duplicate_group(record, exclude_group_id=None) + group = IdentityProviderGroup( + identity_provider_id=self.connection.id, + external_id=record.external_id, + display_name=record.display_name, + attributes=record.attributes, + ) + self.session.add(group) + await self.session.flush() + await self.set_group_members(group, member_link_ids) + return group + + async def replace_group( + self, group: IdentityProviderGroup, record: GroupRecord + ) -> IdentityProviderGroup: + await self._refuse_duplicate_group(record, exclude_group_id=group.id) + group.display_name = record.display_name + group.external_id = record.external_id + group.attributes = record.attributes + await self.session.flush() + return group + + async def set_group_members( + self, group: IdentityProviderGroup, member_link_ids: Iterable[UUID] + ) -> None: + wanted = await self._known_links(set(member_link_ids)) + current = await self.group_member_ids(group.id) + added, removed = wanted - current, current - wanted + if removed: + await self.session.execute( + delete(IdentityProviderGroupMember).where( + IdentityProviderGroupMember.group_id == group.id, + IdentityProviderGroupMember.identity_provider_user_id.in_(removed), + ) + ) + for link_id in added: + self.session.add( + IdentityProviderGroupMember(group_id=group.id, identity_provider_user_id=link_id) + ) + await self.session.flush() + if added or removed: + await self._reconcile_access(added | removed, group_changes={ + link_id: ("added" if link_id in added else "removed") for link_id in added | removed + }, group=group) + + async def delete_group(self, group: IdentityProviderGroup) -> None: + members = await self.group_member_ids(group.id) + await self.session.delete(group) + await self.session.flush() + if members: + await self._reconcile_access(members) + + # ------------------------------------------------------------------ + # Access reconciliation + # ------------------------------------------------------------------ + + async def reconcile_all(self) -> None: + """Recompute access for every linked user (after a mapping change).""" + link_ids = await self.session.scalars( + select(IdentityProviderUser.id).where( + IdentityProviderUser.identity_provider_id == self.connection.id + ) + ) + await self._reconcile_access(set(link_ids)) + + async def _reconcile_access( + self, + link_ids: Iterable[UUID], + *, + group_changes: dict[UUID, str] | None = None, + group: IdentityProviderGroup | None = None, + ) -> None: + link_ids = set(link_ids) + if not link_ids: + return + mapped_roles = await self._mapped_group_roles() + has_mappings = bool(mapped_roles) + memberships: dict[UUID, set[UUID]] = {} + for link_id, group_id in ( + await self.session.execute( + select( + IdentityProviderGroupMember.identity_provider_user_id, + IdentityProviderGroupMember.group_id, + ).where(IdentityProviderGroupMember.identity_provider_user_id.in_(link_ids)) + ) + ).all(): + memberships.setdefault(link_id, set()).add(group_id) + + links = ( + await self.session.scalars( + select(IdentityProviderUser).where(IdentityProviderUser.id.in_(link_ids)) + ) + ).all() + for link in links: + user = await self.session.get(User, link.user_id, with_for_update=True) + assert user is not None + mapped = memberships.get(link.id, set()) & mapped_roles.keys() + desired_roles: set[UUID] = set() + for group_id in mapped: + desired_roles |= mapped_roles[group_id] + desired_active = link.active and (not has_mappings or bool(mapped)) + change = _AccessChange() + if user.is_active != desired_active: + await self._set_active(user, desired_active) + change.activated = desired_active + change.deactivated = not desired_active + change.roles_removed = await self._remove_granted_roles(user.id, desired_roles) + change.roles_added = await self._grant_roles(user, desired_roles) + for role_id in change.roles_added | change.roles_removed: + await self._invalidate_role(role_id) + if change.roles_added or change.roles_removed: + from shared.role_cache import invalidate_user + + await invalidate_user(user.id) + if group is not None and group_changes and link.id in group_changes: + self.pending_events.append( + ( + "user.groups_changed", + { + **self._user_event_body(user, link), + "group": { + "id": str(group.id), + "display_name": group.display_name, + "change": group_changes[link.id], + }, + "roles_added": sorted(str(r) for r in change.roles_added), + "roles_removed": sorted(str(r) for r in change.roles_removed), + }, + ) + ) + if change.deactivated: + await self._audit( + "identity_provider.user_deactivated", user, {"email": user.email} + ) + reason = "inactive" if not link.active else "no_mapped_group" + self.pending_events.append( + ( + "user.deprovisioned", + { + **self._user_event_body(user, link), + "deprovision": {"reason": reason, "account_removed": False}, + }, + ) + ) + elif change.activated: + await self._audit( + "identity_provider.user_activated", user, {"email": user.email} + ) + + async def _mapped_group_roles(self) -> dict[UUID, set[UUID]]: + rows = ( + await self.session.execute( + select(IdentityProviderGroupMapping.group_id, IdentityProviderGroupRole.role_id) + .join( + IdentityProviderGroup, + IdentityProviderGroup.id == IdentityProviderGroupMapping.group_id, + ) + .outerjoin( + IdentityProviderGroupRole, + IdentityProviderGroupRole.group_id == IdentityProviderGroupMapping.group_id, + ) + .where(IdentityProviderGroup.identity_provider_id == self.connection.id) + ) + ).all() + mapped: dict[UUID, set[UUID]] = {} + for group_id, role_id in rows: + roles = mapped.setdefault(group_id, set()) + if role_id is not None and role_id not in UNMAPPABLE_ROLE_IDS: + roles.add(role_id) + return mapped + + async def _grant_roles(self, user: User, desired: set[UUID]) -> set[UUID]: + from shared.account_guards import AccountGuardError + from src.services.user_role_assignments import ( + RoleAssignmentError, + default_boundaries, + insert_assignment, + ) + + held = set( + await self.session.scalars(select(UserRole.role_id).where(UserRole.user_id == user.id)) + ) + added: set[UUID] = set() + for role_id in sorted(desired - held - {user.base_role_id}): + try: + await insert_assignment( + self.session, + user_id=user.id, + role_id=role_id, + boundaries=default_boundaries(user.organization_id), + assigned_by=f"identity-provider:{self.connection.name}"[:255], + identity_provider_id=self.connection.id, + ) + except (RoleAssignmentError, AccountGuardError) as exc: + # e.g. a privileged role mapped for an External user: skip the + # grant, keep provisioning the rest. + logger.warning( + "Identity provider %s skipped role %s for user %s: %s", + self.connection.id, + role_id, + user.id, + log_safe(str(exc)), + ) + continue + added.add(role_id) + return added + + async def _remove_granted_roles(self, user_id: UUID, desired: set[UUID] | None) -> set[UUID]: + """Delete assignments this connection made that are no longer wanted + (all of them when ``desired`` is None).""" + stmt = select(UserRole.role_id).where( + UserRole.user_id == user_id, + UserRole.identity_provider_id == self.connection.id, + ) + if desired: + stmt = stmt.where(UserRole.role_id.not_in(desired)) + removed = set(await self.session.scalars(stmt)) + if removed: + await self.session.execute( + delete(UserRole).where(UserRole.user_id == user_id, UserRole.role_id.in_(removed)) + ) + await self.session.flush() + return removed + + async def _set_active(self, user: User, active: bool) -> None: + if user.is_active == active: + return + if not active: + from shared.account_guards import ( + AccountGuardError, + refuse_last_active_platform_admin_removal, + ) + from shared.human_sessions import revoke_all_human_sessions_in_transaction + + try: + await refuse_last_active_platform_admin_removal(self.session, {user.id}) + except AccountGuardError as exc: + raise ProvisioningError(ProvisioningErrorKind.CONFLICT, exc.detail) from None + user.is_active = False + await self.session.flush() + await revoke_all_human_sessions_in_transaction(self.session, user.id) + else: + user.is_active = True + await self.session.flush() + + async def _invalidate_role(self, role_id: UUID) -> None: + from src.core.cache import invalidate_role_users + + await invalidate_role_users(None, str(role_id)) + + # ------------------------------------------------------------------ + # Guards + # ------------------------------------------------------------------ + + async def _require_claimed_domain(self, email: str) -> None: + domain = email_domain(email) + if domain is None or domain not in await verified_domains( + self.session, self.connection.organization_id + ): + raise ProvisioningError( + ProvisioningErrorKind.DOMAIN_NOT_CLAIMED, + f"The email domain '{domain}' is not a verified domain of this organization", + ) + + async def _require_adoptable(self, user: User) -> None: + if user.organization_id != self.connection.organization_id: + raise ProvisioningError( + ProvisioningErrorKind.CONFLICT, + "A user with this email belongs to another organization", + ) + if user.is_system or user.identity_kind is not None: + raise ProvisioningError( + ProvisioningErrorKind.CONFLICT, "This email belongs to a system account" + ) + linked = await self.session.scalar( + select(IdentityProviderUser.identity_provider_id).where( + IdentityProviderUser.user_id == user.id + ) + ) + if linked is not None: + raise ProvisioningError( + ProvisioningErrorKind.CONFLICT, + "This user is already managed by an identity provider", + ) + + async def _refuse_duplicate_link( + self, record: UserRecord, *, exclude_link_id: UUID | None + ) -> None: + clauses = [IdentityProviderUser.user_name == record.user_name] + if record.external_id is not None: + clauses.append(IdentityProviderUser.external_id == record.external_id) + stmt = select(IdentityProviderUser.id).where( + IdentityProviderUser.identity_provider_id == self.connection.id, or_(*clauses) + ) + if exclude_link_id is not None: + stmt = stmt.where(IdentityProviderUser.id != exclude_link_id) + if await self.session.scalar(stmt) is not None: + raise ProvisioningError( + ProvisioningErrorKind.CONFLICT, "A user with this userName or externalId exists" + ) + + async def _refuse_duplicate_group( + self, record: GroupRecord, *, exclude_group_id: UUID | None + ) -> None: + clauses = [IdentityProviderGroup.display_name == record.display_name] + if record.external_id is not None: + clauses.append(IdentityProviderGroup.external_id == record.external_id) + stmt = select(IdentityProviderGroup.id).where( + IdentityProviderGroup.identity_provider_id == self.connection.id, or_(*clauses) + ) + if exclude_group_id is not None: + stmt = stmt.where(IdentityProviderGroup.id != exclude_group_id) + if await self.session.scalar(stmt) is not None: + raise ProvisioningError( + ProvisioningErrorKind.CONFLICT, + "A group with this displayName or externalId exists", + ) + + async def _known_links(self, link_ids: set[UUID]) -> set[UUID]: + if not link_ids: + return set() + known = set( + await self.session.scalars( + select(IdentityProviderUser.id).where( + IdentityProviderUser.identity_provider_id == self.connection.id, + IdentityProviderUser.id.in_(link_ids), + ) + ) + ) + unknown = link_ids - known + if unknown: + raise ProvisioningError( + ProvisioningErrorKind.INVALID, + f"Unknown group member id(s): {', '.join(sorted(str(u) for u in unknown))}", + ) + return known + + # ------------------------------------------------------------------ + # Creation, audit and events + # ------------------------------------------------------------------ + + async def _new_user(self, email: str, record: UserRecord) -> User: + from shared.base_role_defaults import BaseRoleDefaultError, resolve_base_role + from shared.identities import is_identity_email + from shared.sdk_users import set_user_base_role + + if is_identity_email(email): + raise ProvisioningError(ProvisioningErrorKind.INVALID, "This email domain is reserved") + try: + base_role = await resolve_base_role( + self.session, organization_id=self.connection.organization_id + ) + except BaseRoleDefaultError as exc: + raise ProvisioningError(ProvisioningErrorKind.INVALID, exc.detail) from None + now = datetime.now(timezone.utc) + user = User( + email=email, + name=record.display_name, + hashed_password="", + # Activation is decided by reconciliation right after linking. + is_active=False, + is_superuser=False, + is_external=self.connection.provision_as_external, + # The identity provider vouches for the address; the person signs + # in through SSO, which marks the account registered. + is_verified=True, + is_registered=False, + organization_id=self.connection.organization_id, + created_at=now, + updated_at=now, + ) + await set_user_base_role(self.session, user, base_role.id) + self.session.add(user) + await self.session.flush() + return user + + async def _audit( + self, + action: str, + user: User | None, + details: dict[str, Any], + *, + resource_id: Any = None, + ) -> None: + from src.services.audit import emit_audit + + await emit_audit( + self.session, + action, + resource_type="user", + resource_id=user.id if user is not None else resource_id, + details={ + **details, + "identity_provider_id": str(self.connection.id), + "identity_provider_name": self.connection.name, + }, + actor_override=self._actor, + ) + + def _user_event_body( + self, user: User, link: IdentityProviderUser | None + ) -> dict[str, Any]: + from src.services.events.builtins import identity_provider_user_body + + return identity_provider_user_body( + organization_id=self.connection.organization_id, + identity_provider={ + "id": str(self.connection.id), + "name": self.connection.name, + "vendor": self.connection.vendor, + }, + user={ + "id": str(user.id), + "email": user.email, + "name": user.name, + "is_active": user.is_active, + "external_id": link.external_id if link else None, + "attributes": link.attributes if link else None, + }, + ) + + def _queue_user_event( + self, + topic: str, + user: User, + link: IdentityProviderUser, + *, + adopted: bool | None = None, + ) -> None: + body = self._user_event_body(user, link) + if adopted is not None: + body["adopted"] = adopted + self.pending_events.append((topic, body)) + + async def emit_pending_events(self) -> None: + """Send collected platform events. Call only after commit.""" + from src.services.events.builtins import emit_identity_provider_event + + events, self.pending_events = self.pending_events, [] + for topic, body in events: + await emit_identity_provider_event( + topic, + body, + organization_id=self.connection.organization_id, + identity_provider_id=self.connection.id, + ) diff --git a/api/src/services/identity_provisioning/domains.py b/api/src/services/identity_provisioning/domains.py new file mode 100644 index 0000000000..62402feb55 --- /dev/null +++ b/api/src/services/identity_provisioning/domains.py @@ -0,0 +1,155 @@ +"""Organization domain claims, proven by a DNS TXT record. + +An organization adds a claim and receives a verification token. It publishes +``bifrost-verification=`` as a TXT record on the domain itself (the +same apex convention Microsoft 365 uses), then asks Bifrost to verify. A +domain can be verified by only one organization; the partial unique index on +``organization_domains`` enforces that even under concurrent verification. +""" + +from __future__ import annotations + +import re +import secrets +from datetime import datetime, timezone +from uuid import UUID + +import dns.asyncresolver +import dns.exception +import dns.resolver +from sqlalchemy import select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.ext.asyncio import AsyncSession + +from src.models.orm.identity_providers import OrganizationDomain + +TXT_PREFIX = "bifrost-verification=" +_DNS_TIMEOUT_SECONDS = 5.0 +# RFC 1035 labels; at least two labels; no trailing dot. +_DOMAIN_RE = re.compile( + r"^(?=.{1,253}$)(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z][a-z0-9-]{0,61}[a-z0-9]$" +) + + +class DomainClaimError(Exception): + def __init__(self, status_code: int, detail: str): + super().__init__(detail) + self.status_code = status_code + self.detail = detail + + +def normalize_domain(value: str) -> str: + domain = value.strip().lower().rstrip(".") + if domain.startswith("@"): + domain = domain[1:] + try: + domain = domain.encode("idna").decode("ascii") + except UnicodeError: + raise DomainClaimError(422, "Enter a valid domain name") from None + if not _DOMAIN_RE.match(domain): + raise DomainClaimError(422, "Enter a valid domain name") + return domain + + +def email_domain(email: str) -> str | None: + _, at, domain = email.rpartition("@") + if not at or not domain: + return None + return domain.lower() + + +def txt_record_value(claim: OrganizationDomain) -> str: + return f"{TXT_PREFIX}{claim.verification_token}" + + +async def verified_domains(session: AsyncSession, organization_id: UUID) -> set[str]: + rows = await session.scalars( + select(OrganizationDomain.domain).where( + OrganizationDomain.organization_id == organization_id, + OrganizationDomain.verified_at.is_not(None), + ) + ) + return set(rows) + + +async def list_claims(session: AsyncSession, organization_id: UUID) -> list[OrganizationDomain]: + rows = await session.scalars( + select(OrganizationDomain) + .where(OrganizationDomain.organization_id == organization_id) + .order_by(OrganizationDomain.domain) + ) + return list(rows) + + +async def get_claim( + session: AsyncSession, organization_id: UUID, claim_id: UUID +) -> OrganizationDomain: + claim = await session.scalar( + select(OrganizationDomain).where( + OrganizationDomain.id == claim_id, + OrganizationDomain.organization_id == organization_id, + ) + ) + if claim is None: + raise DomainClaimError(404, "Domain claim not found") + return claim + + +async def add_claim( + session: AsyncSession, organization_id: UUID, domain: str, *, created_by: str +) -> OrganizationDomain: + domain = normalize_domain(domain) + existing = await session.scalar( + select(OrganizationDomain).where( + OrganizationDomain.organization_id == organization_id, + OrganizationDomain.domain == domain, + ) + ) + if existing is not None: + raise DomainClaimError(409, f"This organization already claims {domain}") + claim = OrganizationDomain( + organization_id=organization_id, + domain=domain, + verification_token=secrets.token_urlsafe(24), + created_by=created_by, + ) + session.add(claim) + await session.flush() + return claim + + +async def lookup_txt_records(domain: str) -> list[str]: + """TXT strings published on ``domain``; empty when there are none.""" + resolver = dns.asyncresolver.Resolver() + try: + answer = await resolver.resolve(domain, "TXT", lifetime=_DNS_TIMEOUT_SECONDS) + except (dns.resolver.NXDOMAIN, dns.resolver.NoAnswer): + return [] + except (dns.resolver.NoNameservers, dns.exception.Timeout): + raise DomainClaimError( + 503, f"DNS lookup for {domain} failed; try again in a few minutes" + ) from None + return [ + b"".join(rdata.strings).decode("utf-8", errors="replace") + for rdata in answer + ] + + +async def verify_claim(session: AsyncSession, claim: OrganizationDomain) -> OrganizationDomain: + """Check DNS and mark the claim verified when the record is present.""" + now = datetime.now(timezone.utc) + records = await lookup_txt_records(claim.domain) + claim.last_checked_at = now + if txt_record_value(claim) not in {record.strip() for record in records}: + await session.flush() + return claim + if claim.verified_at is None: + claim.verified_at = now + try: + async with session.begin_nested(): + await session.flush() + except IntegrityError: + raise DomainClaimError( + 409, f"{claim.domain} is already verified by another organization" + ) from None + return claim diff --git a/api/src/services/identity_provisioning/scim.py b/api/src/services/identity_provisioning/scim.py new file mode 100644 index 0000000000..0ee403ac30 --- /dev/null +++ b/api/src/services/identity_provisioning/scim.py @@ -0,0 +1,570 @@ +"""SCIM 2.0 (RFC 7643 / 7644) translation for the provisioning core. + +``scim2-models`` validates resources, applies PatchOp requests and parses +filters. Real identity providers are looser than the RFC, so requests are +normalized first: + +- Entra sends ``manager`` as a bare id string and omits its ``$ref``. +- Okta (and Entra without its 2020 compatibility flag) removes group members + with ``{"op": "remove", "path": "members", "value": [{"value": id}]}``. +- Vendor extensions Bifrost has no schema for are kept verbatim ("keep all + the data") but are not validated or exposed through ``/Schemas``. + +Responses are built from the stored attributes, so whatever the identity +provider sent round-trips. +""" + +from __future__ import annotations + +import copy +from dataclasses import dataclass +from datetime import datetime +from typing import Any +from uuid import UUID + +from pydantic import ValidationError +from scim2_models import ( + Context, + EnterpriseUser, + Group, + PatchOp, + ResourceType, + SCIMException, + ScimFilter, + User, +) +from scim2_models.path.expressions import Comparison, CompareOperator + +from src.models.orm.identity_providers import IdentityProviderGroup, IdentityProviderUser +from src.services.identity_provisioning.core import ( + GroupRecord, + ProvisioningError, + ProvisioningErrorKind, + UserRecord, +) + +ScimUser = User[EnterpriseUser] + +SCIM_MEDIA_TYPE = "application/scim+json" +USER_SCHEMA = "urn:ietf:params:scim:schemas:core:2.0:User" +GROUP_SCHEMA = "urn:ietf:params:scim:schemas:core:2.0:Group" +ENTERPRISE_SCHEMA = "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User" +PATCH_SCHEMA = "urn:ietf:params:scim:api:messages:2.0:PatchOp" +LIST_SCHEMA = "urn:ietf:params:scim:api:messages:2.0:ListResponse" +ERROR_SCHEMA = "urn:ietf:params:scim:api:messages:2.0:Error" +KNOWN_USER_SCHEMAS = frozenset({USER_SCHEMA, ENTERPRISE_SCHEMA}) +MAX_RESULTS = 200 + +# Attributes Bifrost assigns or never stores. +_SERVER_OWNED = ("id", "meta", "schemas") +_NEVER_STORED = ("password",) + +USER_FILTER_ATTRIBUTES = ("userName", "externalId", "id") +GROUP_FILTER_ATTRIBUTES = ("displayName", "externalId", "id") + + +class ScimError(Exception): + """An RFC 7644 §3.12 error response.""" + + def __init__(self, status: int, detail: str, scim_type: str | None = None): + super().__init__(detail) + self.status = status + self.detail = detail + self.scim_type = scim_type + + def body(self) -> dict[str, Any]: + body: dict[str, Any] = { + "schemas": [ERROR_SCHEMA], + "status": str(self.status), + "detail": self.detail, + } + if self.scim_type: + body["scimType"] = self.scim_type + return body + + +def from_provisioning_error(exc: ProvisioningError) -> ScimError: + if exc.kind is ProvisioningErrorKind.CONFLICT: + return ScimError(409, exc.detail, "uniqueness") + if exc.kind is ProvisioningErrorKind.NOT_FOUND: + return ScimError(404, exc.detail) + return ScimError(400, exc.detail, "invalidValue") + + +def _from_validation_error(exc: ValidationError) -> ScimError: + first = exc.errors()[0] + if first["type"].startswith("scim_"): + ctx = first.get("ctx", {}) + return ScimError(int(ctx.get("status", 400)), first["msg"], ctx.get("scim_type")) + location = ".".join(str(part) for part in first["loc"]) + detail = f"{first['msg']}: {location}" if location else first["msg"] + return ScimError(400, detail, "invalidValue") + + +def _from_scim_exception(exc: SCIMException) -> ScimError: + return ScimError(exc.status, exc.detail, exc.scim_type or None) + + +# --------------------------------------------------------------------------- +# Request normalization +# --------------------------------------------------------------------------- + + +def _split_unknown_extensions( + body: dict[str, Any], known: frozenset[str] +) -> tuple[dict[str, Any], dict[str, Any]]: + """Separate extension objects Bifrost has no schema for.""" + kept: dict[str, Any] = {} + unknown: dict[str, Any] = {} + for key, value in body.items(): + if key.startswith("urn:") and key not in known: + unknown[key] = value + else: + kept[key] = value + if "schemas" in kept and isinstance(kept["schemas"], list): + kept["schemas"] = [s for s in kept["schemas"] if s in known] + return kept, unknown + + +def _normalize_enterprise(body: dict[str, Any]) -> None: + enterprise = body.get(ENTERPRISE_SCHEMA) + if isinstance(enterprise, dict) and isinstance(enterprise.get("manager"), str): + enterprise["manager"] = {"value": enterprise["manager"]} + + +def _require_object(body: Any) -> dict[str, Any]: + if not isinstance(body, dict): + raise ScimError(400, "The request body must be a JSON object", "invalidSyntax") + return body + + +def parse_user(body: Any) -> tuple[Any, dict[str, Any]]: + """Validate a POST/PUT User body; returns the resource and the unknown + extensions.""" + raw = copy.deepcopy(_require_object(body)) + for key in (*_SERVER_OWNED[:2], *_NEVER_STORED): + raw.pop(key, None) + kept, unknown = _split_unknown_extensions(raw, KNOWN_USER_SCHEMAS) + kept.setdefault("schemas", [USER_SCHEMA]) + _normalize_enterprise(kept) + try: + resource = ScimUser.model_validate(kept) + except ValidationError as exc: + raise _from_validation_error(exc) from None + if not resource.user_name: + raise ScimError(400, "userName is required", "invalidValue") + return resource, unknown + + +def parse_group(body: Any) -> Group: + raw = copy.deepcopy(_require_object(body)) + for key in _SERVER_OWNED[:2]: + raw.pop(key, None) + raw.setdefault("schemas", [GROUP_SCHEMA]) + try: + resource = Group.model_validate(raw) + except ValidationError as exc: + raise _from_validation_error(exc) from None + if not resource.display_name: + raise ScimError(400, "displayName is required", "invalidValue") + return resource + + +def _patch_operations(body: Any) -> list[dict[str, Any]]: + body = _require_object(body) + operations = body.get("Operations") + if not isinstance(operations, list) or not operations: + raise ScimError(400, "A PatchOp request needs at least one operation", "invalidSyntax") + return [_require_object(op) for op in operations] + + +def _patch_op_message(operations: list[dict[str, Any]]) -> dict[str, Any]: + return {"schemas": [PATCH_SCHEMA], "Operations": operations} + + +def _apply_unknown_extension_ops( + operations: list[dict[str, Any]], unknown: dict[str, Any] +) -> list[dict[str, Any]]: + """Apply operations aimed at unknown extensions directly to their stored + objects; return the operations left for the validated model.""" + remaining: list[dict[str, Any]] = [] + for op in operations: + verb = str(op.get("op", "")).lower() + path = op.get("path") + value = op.get("value") + if isinstance(path, str) and path.startswith("urn:") and not any( + path == schema or path.startswith(f"{schema}:") for schema in KNOWN_USER_SCHEMAS + ): + schema, _, attribute = path.rpartition(":") + if path in unknown or not schema: + schema, attribute = path, "" + target = unknown.setdefault(schema, {}) + if verb == "remove": + if attribute: + target.pop(attribute, None) + else: + unknown.pop(schema, None) + elif attribute: + target[attribute] = value + elif isinstance(value, dict): + target.update(value) + continue + if not path and isinstance(value, dict): + known_value = {} + for key, item in value.items(): + if key.startswith("urn:") and key not in KNOWN_USER_SCHEMAS: + if isinstance(item, dict): + unknown.setdefault(key, {}).update(item) + else: + known_value[key] = item + if not known_value: + continue + op = {**op, "value": known_value} + remaining.append(op) + return remaining + + +def _normalize_user_ops(operations: list[dict[str, Any]]) -> list[dict[str, Any]]: + normalized = [] + manager_path = f"{ENTERPRISE_SCHEMA}:manager" + for op in operations: + if op.get("path") == manager_path and isinstance(op.get("value"), str): + op = {**op, "value": {"value": op["value"]}} + value = op.get("value") + if not op.get("path") and isinstance(value, dict) and isinstance( + value.get(ENTERPRISE_SCHEMA), dict + ): + value = copy.deepcopy(value) + _normalize_enterprise(value) + op = {**op, "value": value} + normalized.append(op) + return normalized + + +def _normalize_group_ops(operations: list[dict[str, Any]]) -> list[dict[str, Any]]: + normalized: list[dict[str, Any]] = [] + for op in operations: + if ( + str(op.get("op", "")).lower() == "remove" + and op.get("path") == "members" + and isinstance(op.get("value"), list) + ): + for member in op["value"]: + member_id = member.get("value") if isinstance(member, dict) else None + if not isinstance(member_id, str): + raise ScimError(400, "Each removed member needs a value", "invalidValue") + normalized.append( + {"op": "remove", "path": f'members[value eq {ScimFilter.quote(member_id)}]'} + ) + continue + normalized.append(op) + return normalized + + +def apply_user_patch(link: IdentityProviderUser, body: Any) -> tuple[Any, dict[str, Any]]: + current = stored_user_resource(link) + known, unknown = _split_unknown_extensions(current, KNOWN_USER_SCHEMAS) + _normalize_enterprise(known) + operations = _apply_unknown_extension_ops(_patch_operations(body), unknown) + operations = _normalize_user_ops(operations) + try: + resource = ScimUser.model_validate(known) + if operations: + patch = PatchOp[ScimUser].model_validate( + _patch_op_message(operations), scim_ctx=Context.RESOURCE_PATCH_REQUEST + ) + patch.patch(resource) + except ValidationError as exc: + raise _from_validation_error(exc) from None + except SCIMException as exc: + raise _from_scim_exception(exc) from None + return resource, unknown + + +def apply_group_patch( + group: IdentityProviderGroup, member_ids: set[UUID], body: Any +) -> Group: + current = { + **stored_group_attributes(group), + "schemas": [GROUP_SCHEMA], + "id": str(group.id), + "members": [{"value": str(member)} for member in sorted(member_ids, key=str)], + } + operations = _normalize_group_ops(_patch_operations(body)) + try: + resource = Group.model_validate(current) + patch = PatchOp[Group].model_validate( + _patch_op_message(operations), scim_ctx=Context.RESOURCE_PATCH_REQUEST + ) + patch.patch(resource) + except ValidationError as exc: + raise _from_validation_error(exc) from None + except SCIMException as exc: + raise _from_scim_exception(exc) from None + if not resource.display_name: + raise ScimError(400, "displayName is required", "invalidValue") + return resource + + +# --------------------------------------------------------------------------- +# Resource -> core records +# --------------------------------------------------------------------------- + + +def _attributes(resource: Any, unknown: dict[str, Any] | None = None) -> dict[str, Any]: + data = resource.model_dump(mode="json", exclude_none=True, by_alias=True) + for key in (*_SERVER_OWNED, *_NEVER_STORED): + data.pop(key, None) + if unknown: + data.update(copy.deepcopy(unknown)) + return data + + +def _primary_email(resource: Any) -> str: + emails = [e for e in (resource.emails or []) if e.value] + chosen = ( + next((e for e in emails if e.primary), None) + or next((e for e in emails if (e.type or "").lower() == "work"), None) + or (emails[0] if emails else None) + ) + if chosen is not None: + return str(chosen.value) + if "@" in resource.user_name: + return resource.user_name + raise ScimError(400, "The user needs an email address or an email userName", "invalidValue") + + +def _display_name(resource: Any) -> str | None: + if resource.display_name: + return resource.display_name + name = resource.name + if name is None: + return None + if name.formatted: + return name.formatted + parts = [part for part in (name.given_name, name.family_name) if part] + return " ".join(parts) or None + + +def user_record(resource: Any, unknown: dict[str, Any]) -> UserRecord: + return UserRecord( + user_name=resource.user_name, + email=_primary_email(resource), + external_id=resource.external_id, + display_name=_display_name(resource), + active=True if resource.active is None else bool(resource.active), + attributes=_attributes(resource, unknown), + ) + + +def group_record(resource: Group) -> GroupRecord: + attributes = _attributes(resource) + attributes.pop("members", None) + return GroupRecord( + display_name=str(resource.display_name), + external_id=resource.external_id, + attributes=attributes, + ) + + +def group_member_ids(resource: Group) -> set[UUID]: + ids: set[UUID] = set() + for member in resource.members or []: + try: + ids.add(UUID(str(member.value))) + except ValueError: + raise ScimError( + 400, f"Group member '{member.value}' is not a known user id", "invalidValue" + ) from None + return ids + + +# --------------------------------------------------------------------------- +# Core rows -> responses +# --------------------------------------------------------------------------- + + +def _meta(resource_type: str, location: str, created: datetime, modified: datetime) -> dict: + return { + "resourceType": resource_type, + "created": created.isoformat(), + "lastModified": modified.isoformat(), + "location": location, + } + + +def stored_user_resource(link: IdentityProviderUser) -> dict[str, Any]: + data = copy.deepcopy(link.attributes or {}) + schemas = [USER_SCHEMA] + schemas.extend(key for key in data if key.startswith("urn:")) + return { + **data, + "schemas": schemas, + "id": str(link.id), + "userName": link.user_name, + **({"externalId": link.external_id} if link.external_id else {}), + "active": link.active, + } + + +def stored_group_attributes(group: IdentityProviderGroup) -> dict[str, Any]: + data = copy.deepcopy(group.attributes or {}) + data["displayName"] = group.display_name + if group.external_id: + data["externalId"] = group.external_id + return data + + +def user_response(link: IdentityProviderUser, base_url: str) -> dict[str, Any]: + location = f"{base_url}/Users/{link.id}" + return { + **stored_user_resource(link), + "meta": _meta("User", location, link.created_at, link.updated_at), + } + + +def group_response( + group: IdentityProviderGroup, + members: list[tuple[UUID, str]] | None, + base_url: str, +) -> dict[str, Any]: + location = f"{base_url}/Groups/{group.id}" + body: dict[str, Any] = { + "schemas": [GROUP_SCHEMA], + "id": str(group.id), + **stored_group_attributes(group), + "meta": _meta("Group", location, group.created_at, group.updated_at), + } + if members is not None: + body["members"] = [ + {"value": str(link_id), "display": user_name, "$ref": f"{base_url}/Users/{link_id}"} + for link_id, user_name in members + ] + return body + + +def project( + body: dict[str, Any], attributes: str | None, excluded_attributes: str | None +) -> dict[str, Any]: + """Apply the ``attributes`` / ``excludedAttributes`` query parameters at + top-level attribute granularity. ``id`` and ``schemas`` always return.""" + + def names(value: str | None) -> set[str]: + return { + part.strip().split(".")[0].lower() + for part in (value or "").split(",") + if part.strip() + } + + included, excluded = names(attributes), names(excluded_attributes) + always = {"id", "schemas"} + if included: + return {k: v for k, v in body.items() if k in always or k.lower() in included} + return {k: v for k, v in body.items() if k in always or k.lower() not in excluded} + + +def list_response(resources: list[dict[str, Any]], total: int, start_index: int) -> dict: + return { + "schemas": [LIST_SCHEMA], + "totalResults": total, + "startIndex": start_index, + "itemsPerPage": len(resources), + "Resources": resources, + } + + +# --------------------------------------------------------------------------- +# Filters +# --------------------------------------------------------------------------- + + +@dataclass(frozen=True) +class EqualityFilter: + attribute: str + value: str + + +def parse_filter(expression: str | None, allowed: tuple[str, ...]) -> EqualityFilter | None: + """Accept only `` eq ""`` on the attributes identity + providers look resources up by; anything else is ``invalidFilter``.""" + if not expression: + return None + try: + node = ScimFilter(expression).ast + except (SCIMException, ValueError) as exc: + raise ScimError(400, f"Invalid filter: {exc}", "invalidFilter") from None + if ( + not isinstance(node, Comparison) + or node.op is not CompareOperator.eq + or node.attr_path.sub_attr is not None + or node.attr_path.uri not in (None, "") + or not isinstance(node.value, str) + ): + raise ScimError( + 400, + f"Only ' eq \"value\"' filters on {', '.join(allowed)} are supported", + "invalidFilter", + ) + attribute = next((a for a in allowed if a.lower() == node.attr_path.attr.lower()), None) + if attribute is None: + raise ScimError( + 400, f"Filtering on '{node.attr_path.attr}' is not supported", "invalidFilter" + ) + return EqualityFilter(attribute, node.value) + + +# --------------------------------------------------------------------------- +# Discovery +# --------------------------------------------------------------------------- + + +def service_provider_config(base_url: str) -> dict[str, Any]: + return { + "schemas": ["urn:ietf:params:scim:schemas:core:2.0:ServiceProviderConfig"], + "documentationUri": "https://docs.gobifrost.com", + "patch": {"supported": True}, + "bulk": {"supported": False, "maxOperations": 0, "maxPayloadSize": 0}, + "filter": {"supported": True, "maxResults": MAX_RESULTS}, + "changePassword": {"supported": False}, + "sort": {"supported": False}, + "etag": {"supported": False}, + "authenticationSchemes": [ + { + "type": "oauthbearertoken", + "name": "Bearer token", + "description": "A connection token generated in Bifrost", + "primary": True, + } + ], + "meta": { + "resourceType": "ServiceProviderConfig", + "location": f"{base_url}/ServiceProviderConfig", + }, + } + + +def _dump(model: Any) -> dict[str, Any]: + return model.model_dump(scim_ctx=Context.RESOURCE_QUERY_RESPONSE) + + +def resource_types(base_url: str) -> list[dict[str, Any]]: + types = [] + for model, endpoint in ((ScimUser, "/Users"), (Group, "/Groups")): + resource_type = ResourceType.from_resource(model) + body = _dump(resource_type) + body["endpoint"] = endpoint + body["meta"] = { + "resourceType": "ResourceType", + "location": f"{base_url}/ResourceTypes/{body['id']}", + } + types.append(body) + return types + + +def schemas(base_url: str) -> list[dict[str, Any]]: + result = [] + for model in (User, EnterpriseUser, Group): + body = _dump(model.to_schema()) + body["meta"] = {"resourceType": "Schema", "location": f"{base_url}/Schemas/{body['id']}"} + result.append(body) + return result diff --git a/api/src/services/identity_provisioning/tokens.py b/api/src/services/identity_provisioning/tokens.py new file mode 100644 index 0000000000..aa053cdaf8 --- /dev/null +++ b/api/src/services/identity_provisioning/tokens.py @@ -0,0 +1,124 @@ +"""Bearer credentials for identity provider connections. + +Only the SHA-256 digest of a token is stored; the raw token is returned once, +when it is generated. Rotation issues a new token and cuts the previous +token's remaining life to a short overlap, so the identity provider can be +updated without a provisioning gap. +""" + +from __future__ import annotations + +import hashlib +import hmac +import secrets +from datetime import datetime, timedelta, timezone +from uuid import UUID + +from sqlalchemy import select, update +from sqlalchemy.ext.asyncio import AsyncSession + +from src.models.orm.identity_providers import IdentityProvider, IdentityProviderToken + +TOKEN_PREFIX = "bfidp_" +ROTATION_OVERLAP = timedelta(hours=1) +DEFAULT_LIFETIME_DAYS = 365 +ALLOWED_LIFETIME_DAYS = (90, 180, 365, 730) +# Bookkeeping writes (last_used_at / last_request_at) at most this often per +# token, so a busy provisioning cycle doesn't rewrite the row on every call. +_USAGE_WRITE_INTERVAL = timedelta(minutes=1) + + +def hash_token(raw: str) -> str: + return hashlib.sha256(raw.encode("utf-8")).hexdigest() + + +def _generate() -> tuple[str, str]: + raw = f"{TOKEN_PREFIX}{secrets.token_urlsafe(32)}" + return raw, hash_token(raw) + + +async def current_token( + session: AsyncSession, identity_provider_id: UUID +) -> IdentityProviderToken | None: + """The newest unrevoked token, expired or not (for display).""" + return await session.scalar( + select(IdentityProviderToken) + .where( + IdentityProviderToken.identity_provider_id == identity_provider_id, + IdentityProviderToken.revoked_at.is_(None), + ) + .order_by(IdentityProviderToken.created_at.desc()) + .limit(1) + ) + + +async def issue_token( + session: AsyncSession, + connection: IdentityProvider, + *, + lifetime_days: int, + created_by: str, +) -> tuple[str, IdentityProviderToken]: + """Create a new token; earlier live tokens keep working for the overlap.""" + if lifetime_days not in ALLOWED_LIFETIME_DAYS: + raise ValueError(f"lifetime_days must be one of {ALLOWED_LIFETIME_DAYS}") + now = datetime.now(timezone.utc) + overlap_end = now + ROTATION_OVERLAP + await session.execute( + update(IdentityProviderToken) + .where( + IdentityProviderToken.identity_provider_id == connection.id, + IdentityProviderToken.revoked_at.is_(None), + IdentityProviderToken.expires_at > overlap_end, + ) + .values(expires_at=overlap_end) + ) + raw, digest = _generate() + token = IdentityProviderToken( + identity_provider_id=connection.id, + token_hash=digest, + token_hint=raw[-4:], + expires_at=now + timedelta(days=lifetime_days), + created_by=created_by, + ) + session.add(token) + await session.flush() + return raw, token + + +async def authenticate( + session: AsyncSession, identity_provider_id: UUID, raw: str +) -> IdentityProvider | None: + """The enabled connection ``raw`` is a live token for, or None.""" + if not raw.startswith(TOKEN_PREFIX): + return None + digest = hash_token(raw) + now = datetime.now(timezone.utc) + row = ( + await session.execute( + select(IdentityProviderToken, IdentityProvider) + .join( + IdentityProvider, + IdentityProvider.id == IdentityProviderToken.identity_provider_id, + ) + .where( + IdentityProviderToken.token_hash == digest, + IdentityProviderToken.identity_provider_id == identity_provider_id, + IdentityProviderToken.revoked_at.is_(None), + IdentityProviderToken.expires_at > now, + IdentityProvider.is_enabled.is_(True), + ) + ) + ).one_or_none() + if row is None: + return None + token, connection = row + # The lookup is by digest; compare again in constant time so a future + # change to the query can't turn it into a prefix or partial match. + if not hmac.compare_digest(token.token_hash, digest): + return None + if token.last_used_at is None or now - token.last_used_at > _USAGE_WRITE_INTERVAL: + token.last_used_at = now + connection.last_request_at = now + await session.flush() + return connection diff --git a/api/src/services/operation_catalog.py b/api/src/services/operation_catalog.py index 22e1f8a358..78d6fe3e51 100644 --- a/api/src/services/operation_catalog.py +++ b/api/src/services/operation_catalog.py @@ -3844,6 +3844,197 @@ "native_builder": "Browser-editor renames are not a coding target.", }, ), + OperationDefinition( + operation_id="identityproviders.domains.list", + summary="List an organization's claimed email domains", + target_kind=OperationTargetKind.COLLECTION, + rest=RestOperationBinding(method="GET", path="/api/identity-providers/domains", response_model="OrganizationDomainResponse"), + action_scopes=("identityproviders.read",), + authorization_resolver="identityproviders.read at the organization", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.domains.create", + summary="Claim an email domain for an organization", + target_kind=OperationTargetKind.COLLECTION, + rest=RestOperationBinding(method="POST", path="/api/identity-providers/domains", response_model="OrganizationDomainResponse", request_model="OrganizationDomainCreate"), + action_scopes=("identityproviders.readwrite",), + authorization_resolver="identityproviders.readwrite at the organization", + audit_event="identity_provider.domain_claimed", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.domains.verify", + summary="Verify a domain claim through its DNS TXT record", + target_kind=OperationTargetKind.RESOURCE, + rest=RestOperationBinding(method="POST", path="/api/identity-providers/domains/{claim_id}/verify", response_model="OrganizationDomainResponse"), + action_scopes=("identityproviders.readwrite",), + authorization_resolver="identityproviders.readwrite at the claim's organization", + audit_event="identity_provider.domain_verified", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.domains.delete", + summary="Remove a domain claim", + target_kind=OperationTargetKind.RESOURCE, + rest=RestOperationBinding(method="DELETE", path="/api/identity-providers/domains/{claim_id}"), + action_scopes=("identityproviders.readwrite",), + authorization_resolver="identityproviders.readwrite at the claim's organization", + audit_event="identity_provider.domain_removed", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.list", + summary="List identity provider connections", + target_kind=OperationTargetKind.COLLECTION, + rest=RestOperationBinding(method="GET", path="/api/identity-providers", response_model="IdentityProviderResponse"), + action_scopes=("identityproviders.read",), + authorization_resolver="Filtered to organizations where the caller holds identityproviders.read", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.create", + summary="Create an identity provider connection and its first token", + target_kind=OperationTargetKind.COLLECTION, + rest=RestOperationBinding(method="POST", path="/api/identity-providers", response_model="IdentityProviderCreated", request_model="IdentityProviderCreate"), + action_scopes=("identityproviders.readwrite",), + authorization_resolver="identityproviders.readwrite at the organization", + audit_event="identity_provider.create", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.get", + summary="Read an identity provider connection", + target_kind=OperationTargetKind.RESOURCE, + rest=RestOperationBinding(method="GET", path="/api/identity-providers/{identity_provider_id}", response_model="IdentityProviderResponse"), + action_scopes=("identityproviders.read",), + authorization_resolver="identityproviders.read at the connection's organization", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.update", + summary="Update an identity provider connection", + target_kind=OperationTargetKind.RESOURCE, + rest=RestOperationBinding(method="PATCH", path="/api/identity-providers/{identity_provider_id}", response_model="IdentityProviderResponse", request_model="IdentityProviderUpdate"), + action_scopes=("identityproviders.readwrite",), + authorization_resolver="identityproviders.readwrite at the connection's organization", + audit_event="identity_provider.update", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.delete", + summary="Delete an identity provider connection", + target_kind=OperationTargetKind.RESOURCE, + rest=RestOperationBinding(method="DELETE", path="/api/identity-providers/{identity_provider_id}"), + action_scopes=("identityproviders.readwrite",), + authorization_resolver="identityproviders.readwrite at the connection's organization", + audit_event="identity_provider.delete", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.token.rotate", + summary="Issue a new identity provider token", + target_kind=OperationTargetKind.RESOURCE, + rest=RestOperationBinding(method="POST", path="/api/identity-providers/{identity_provider_id}/token", response_model="IdentityProviderTokenIssued", request_model="IdentityProviderTokenRotate"), + action_scopes=("identityproviders.readwrite",), + authorization_resolver="identityproviders.readwrite at the connection's organization", + audit_event="identity_provider.token_rotated", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.groups.list", + summary="List the groups an identity provider pushed", + target_kind=OperationTargetKind.RESOURCE, + rest=RestOperationBinding(method="GET", path="/api/identity-providers/{identity_provider_id}/groups", response_model="IdentityProviderGroupResponse"), + action_scopes=("identityproviders.read",), + authorization_resolver="identityproviders.read at the connection's organization", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.groups.map", + summary="Map an identity provider group to roles", + target_kind=OperationTargetKind.RESOURCE, + rest=RestOperationBinding(method="PUT", path="/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping", response_model="IdentityProviderGroupResponse", request_model="IdentityProviderGroupMappingUpdate"), + action_scopes=("identityproviders.readwrite", "roleassignments.readwrite",), + authorization_resolver="identityproviders.readwrite at the connection's organization, plus the manual role-assignment checks for each mapped role", + audit_event="identity_provider.group_mapped", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), + OperationDefinition( + operation_id="identityproviders.groups.unmap", + summary="Remove an identity provider group mapping", + target_kind=OperationTargetKind.RESOURCE, + rest=RestOperationBinding(method="DELETE", path="/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping"), + action_scopes=("identityproviders.readwrite",), + authorization_resolver="identityproviders.readwrite at the connection's organization", + audit_event="identity_provider.group_unmapped", + exclusions={ + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + }, + ), ) diff --git a/api/src/services/operation_inventory.py b/api/src/services/operation_inventory.py index 67e816fded..cc87d22855 100644 --- a/api/src/services/operation_inventory.py +++ b/api/src/services/operation_inventory.py @@ -99,6 +99,15 @@ def _rest_disposition(method: str, path: str) -> dict[str, str]: "owns no entity state." ), } + if path.startswith("/api/scim/v2/{identity_provider_id}/"): + return { + "status": OperationSurfaceStatus.TRANSPORT_ONLY.value, + "reason": ( + "SCIM 2.0 wire protocol an external identity provider calls with " + "a connection token; writes go through the identity provisioning " + "core, administered by the identityproviders operations." + ), + } if (method, path) == ("POST", "/api/solutions/{solution_id}/dev-context"): return { "status": OperationSurfaceStatus.TRANSPORT_ONLY.value, diff --git a/api/src/services/user_role_assignments.py b/api/src/services/user_role_assignments.py index 628ce2017e..0bfa002546 100644 --- a/api/src/services/user_role_assignments.py +++ b/api/src/services/user_role_assignments.py @@ -144,8 +144,13 @@ async def insert_assignment( role_id: UUID, boundaries: Iterable[Boundary], assigned_by: str, + identity_provider_id: UUID | None = None, ) -> None: - """Add a ``user_roles`` row and its boundary rows.""" + """Add a ``user_roles`` row and its boundary rows. + + ``identity_provider_id`` marks an assignment that identity provisioning + made from a group mapping (it is the only writer that passes one). + """ from shared.account_guards import ( AccountGuardError, assert_role_assignment_allowed_for_user, @@ -163,6 +168,7 @@ async def insert_assignment( role_id=role_id, assigned_by=assigned_by, assigned_at=datetime.now(timezone.utc), + identity_provider_id=identity_provider_id, ) ) # The boundary rows reference the assignment by a composite foreign key. diff --git a/api/tests/e2e/api/test_scim_provisioning.py b/api/tests/e2e/api/test_scim_provisioning.py new file mode 100644 index 0000000000..1fb222d06d --- /dev/null +++ b/api/tests/e2e/api/test_scim_provisioning.py @@ -0,0 +1,218 @@ +"""SCIM provisioning through a live API: an Entra/Okta-shaped lifecycle, and +the scim2-tester RFC checks that don't depend on the domain-claim rule.""" + +import uuid +from datetime import datetime, timezone +from typing import Any + +import httpx +import pytest +from sqlalchemy import select, update + +from src.models.orm.identity_providers import OrganizationDomain +from src.models.orm.users import User, UserRole +from tests.e2e.conftest import E2E_API_URL + +SCIM_JSON = "application/scim+json" +ENTERPRISE = "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User" +PATCH = "urn:ietf:params:scim:api:messages:2.0:PatchOp" + + +def _ok(response: httpx.Response, status: int = 200) -> Any: + assert response.status_code == status, response.text + return response.json() if response.content else None + + +@pytest.fixture +def scim_connection(e2e_client, platform_admin, db_session): + """An org with a verified domain and a SCIM connection.""" + admin = platform_admin.headers + tag = uuid.uuid4().hex[:8] + organization = _ok( + e2e_client.post("/api/organizations", headers=admin, json={"name": f"SCIM {tag}"}), 201 + ) + domain = f"scim-{tag}.example" + claim = _ok( + e2e_client.post( + "/api/identity-providers/domains", + headers=admin, + json={"organization_id": organization["id"], "domain": domain}, + ), + 201, + ) + assert claim["txt_record_value"].startswith("bifrost-verification=") + created = _ok( + e2e_client.post( + "/api/identity-providers", + headers=admin, + json={"organization_id": organization["id"], "name": f"Entra {tag}", "vendor": "entra"}, + ), + 201, + ) + yield { + "organization": organization, + "domain": domain, + "claim_id": claim["id"], + "connection": created["identity_provider"], + "token": created["token"]["token"], + "path": f"/api/scim/v2/{created['identity_provider']['id']}", + } + e2e_client.delete( + f"/api/identity-providers/{created['identity_provider']['id']}", headers=admin + ) + e2e_client.delete(f"/api/organizations/{organization['id']}", headers=admin) + + +async def _verify_domain_out_of_band(db_session, claim_id: str) -> None: + # Real verification needs public DNS; mark it as the verify step would. + await db_session.execute( + update(OrganizationDomain) + .where(OrganizationDomain.id == uuid.UUID(claim_id)) + .values(verified_at=datetime.now(timezone.utc)) + ) + await db_session.commit() + + +@pytest.mark.e2e +@pytest.mark.asyncio +async def test_scim_user_and_group_lifecycle(e2e_client, platform_admin, db_session, scim_connection): + admin = platform_admin.headers + path, domain = scim_connection["path"], scim_connection["domain"] + scim = {"Authorization": f"Bearer {scim_connection['token']}", "Content-Type": SCIM_JSON} + + refused = e2e_client.get(f"{path}/Users", headers={"Authorization": "Bearer bfidp_wrong"}) + assert refused.status_code == 401 + assert refused.headers["content-type"].startswith(SCIM_JSON) + assert refused.json()["schemas"] == ["urn:ietf:params:scim:api:messages:2.0:Error"] + + email = f"pat@{domain}" + entra_user = { + "schemas": ["urn:ietf:params:scim:schemas:core:2.0:User", ENTERPRISE], + "userName": email, + "externalId": "entra-object-1", + "active": "True", + "displayName": "Pat Example", + "emails": [{"value": email, "type": "work", "primary": True}], + ENTERPRISE: {"department": "Support"}, + } + unclaimed = {**entra_user, "userName": "pat@unclaimed.example", "externalId": "x", + "emails": [{"value": "pat@unclaimed.example", "primary": True}]} + await _verify_domain_out_of_band(db_session, scim_connection["claim_id"]) + assert e2e_client.post(f"{path}/Users", headers=scim, json=unclaimed).status_code == 400 + + created = e2e_client.post(f"{path}/Users", headers=scim, json=entra_user) + body = _ok(created, 201) + assert created.headers["location"].endswith(f"/Users/{body['id']}") + assert body["active"] is True and body[ENTERPRISE]["department"] == "Support" + assert _ok(e2e_client.post(f"{path}/Users", headers=scim, json=entra_user), 409)["scimType"] == "uniqueness" + + found = _ok(e2e_client.get(f"{path}/Users", headers=scim, params={"filter": f'userName eq "{email.upper()}"'})) + assert [r["id"] for r in found["Resources"]] == [body["id"]] + + users = _ok(e2e_client.get("/api/users", headers=admin, params={"organization_id": scim_connection["organization"]["id"]})) + user = next(u for u in users if u["email"] == email) + assert user["identity_source"]["name"] == scim_connection["connection"]["name"] + + group = _ok(e2e_client.post(f"{path}/Groups", headers=scim, json={ + "schemas": ["urn:ietf:params:scim:schemas:core:2.0:Group"], + "displayName": "Support Techs", "externalId": "entra-group-1", + "members": [{"value": body["id"]}], + }), 201) + role = _ok(e2e_client.post("/api/roles", headers=admin, json={"name": f"Support Tech {uuid.uuid4().hex[:6]}"}), 201) + try: + mapped = _ok(e2e_client.put( + f"/api/identity-providers/{scim_connection['connection']['id']}/groups/{group['id']}/mapping", + headers=admin, json={"role_ids": [role["id"]]}, + )) + assert mapped["mapped"] and mapped["member_count"] == 1 + user_id = uuid.UUID(user["id"]) + db_session.expire_all() + grant = await db_session.scalar( + select(UserRole).where(UserRole.user_id == user_id, UserRole.role_id == uuid.UUID(role["id"])) + ) + assert grant is not None + assert str(grant.identity_provider_id) == scim_connection["connection"]["id"] + + # Okta removes members with a value list rather than a path filter. + removed = e2e_client.patch(f"{path}/Groups/{group['id']}", headers=scim, json={ + "schemas": [PATCH], "Operations": [{"op": "remove", "path": "members", "value": [{"value": body["id"]}]}], + }) + assert removed.status_code == 204, removed.text + db_session.expire_all() + account = await db_session.get(User, user_id) + assert account is not None and not account.is_active + assert await db_session.scalar( + select(UserRole).where(UserRole.user_id == user_id, UserRole.role_id == uuid.UUID(role["id"])) + ) is None + + rotated = _ok(e2e_client.post( + f"/api/identity-providers/{scim_connection['connection']['id']}/token", headers=admin, json={"lifetime_days": 90}, + )) + assert e2e_client.get(f"{path}/Users/{body['id']}", headers=scim).status_code == 200 + new_scim = {**scim, "Authorization": f"Bearer {rotated['token']}"} + patched = _ok(e2e_client.patch(f"{path}/Users/{body['id']}", headers=new_scim, json={ + "schemas": [PATCH], "Operations": [{"op": "Replace", "path": "active", "value": "False"}], + })) + assert patched["active"] is False + + assert e2e_client.delete(f"{path}/Users/{body['id']}", headers=new_scim).status_code == 204 + db_session.expire_all() + assert await db_session.get(User, user_id) is None + finally: + e2e_client.delete(f"/api/roles/{role['id']}", headers=admin) + + +class _HttpxScimClient: + """scim2-client engine over plain httpx (the bundled one needs httpx2).""" + + @staticmethod + def build(base_url: str, token: str): + from scim2_client import BaseSyncSCIMClient + + class Client(BaseSyncSCIMClient): + def __init__(self) -> None: + super().__init__() + self.http = httpx.Client( + base_url=base_url, + headers={"Authorization": f"Bearer {token}"}, + timeout=30.0, + ) + + def _stays_under_base_url(self, endpoint: str) -> bool: + return endpoint.startswith(base_url) or not endpoint.startswith("http") + + def _parse_json(self, response): + return response.json() + + def request(self, method: str, url: str, **kwargs): + return self.http.request(method, url, **kwargs) + + return Client() + + +@pytest.mark.e2e +def test_scim2_tester_discovery_and_group_crud(scim_connection): + from scim2_tester import Status, check_server + + client = _HttpxScimClient.build(f"{E2E_API_URL}{scim_connection['path']}", scim_connection["token"]) + client.discover() + results = check_server(client, include_tags={"discovery", "crud"}, resource_types=["Group"]) + check_server( + client, include_tags={"discovery"} + ) + + # Some Group checks first create member Users from generated data: a + # non-email userName or a random email domain. Refusing those is the + # domain-claim rule working, so only those two refusals are expected. + provisioning_refusals = ( + "The user needs an email address or an email userName", + "is not a verified domain of this organization", + ) + failures = [ + f"{r.resource_type}: {r.title}: {r.reason}" + for r in results + if r.status in (Status.ERROR, Status.CRITICAL) + and not any(refusal in (r.reason or "") for refusal in provisioning_refusals) + ] + assert not failures, "\n".join(failures) + passed = {r.title for r in results if r.status is Status.SUCCESS} + assert {"service_provider_config_endpoint", "object_creation", "object_deletion"} <= passed, passed diff --git a/api/tests/unit/services/test_identity_provisioning_core.py b/api/tests/unit/services/test_identity_provisioning_core.py new file mode 100644 index 0000000000..7458ed2f9a --- /dev/null +++ b/api/tests/unit/services/test_identity_provisioning_core.py @@ -0,0 +1,242 @@ +"""The protocol-independent provisioning rules, against the database.""" + +from datetime import datetime, timezone +from uuid import uuid4 + +import pytest +import pytest_asyncio +from sqlalchemy import select + +from src.models.orm.identity_providers import ( + IdentityProvider, + IdentityProviderGroupMapping, + IdentityProviderGroupRole, + IdentityProviderUser, + OrganizationDomain, +) +from src.models.orm.organizations import Organization +from src.models.orm.users import Role, User, UserRole +from src.services.identity_provisioning.core import ( + GroupRecord, + ProvisioningError, + ProvisioningErrorKind, + Provisioner, + UserRecord, +) + + +def _record(email: str, *, external_id: str | None = None, active: bool = True) -> UserRecord: + return UserRecord( + user_name=email, + email=email, + external_id=external_id or f"ext-{email}", + display_name="Pat Example", + active=active, + attributes={"userName": email}, + ) + + +@pytest_asyncio.fixture +async def setup(db_session): + suffix = uuid4().hex[:10] + domain = f"contoso-{suffix}.example" + organization = Organization(name=f"Contoso {suffix}", created_by="test") + other = Organization(name=f"Fabrikam {suffix}", created_by="test") + db_session.add_all([organization, other]) + await db_session.flush() + db_session.add( + OrganizationDomain( + organization_id=organization.id, + domain=domain, + verification_token="t", + verified_at=datetime.now(timezone.utc), + created_by="test", + ) + ) + connection = IdentityProvider( + organization_id=organization.id, + name="Contoso Entra", + source_type="scim", + vendor="entra", + created_by="test", + ) + role = Role(name=f"Support Tech {suffix}", created_by="test") + db_session.add_all([connection, role]) + await db_session.flush() + return { + "session": db_session, + "organization": organization, + "other": other, + "domain": domain, + "connection": connection, + "role": role, + } + + +async def _user(session, link: IdentityProviderUser) -> User: + user = await session.get(User, link.user_id) + await session.refresh(user) + return user + + +async def _map(session, group_id, role_ids=()): + session.add(IdentityProviderGroupMapping(group_id=group_id, created_by="test")) + await session.flush() + for role_id in role_ids: + session.add(IdentityProviderGroupRole(group_id=group_id, role_id=role_id)) + await session.flush() + + +@pytest.mark.asyncio +async def test_creates_an_active_verified_user_in_the_connection_org(setup): + provisioner = Provisioner(setup["session"], setup["connection"]) + + link = await provisioner.create_user(_record(f"pat@{setup['domain']}")) + + user = await _user(setup["session"], link) + assert user.organization_id == setup["organization"].id + assert user.is_active and user.is_verified and not user.is_registered + assert link.created_user + assert [topic for topic, _ in provisioner.pending_events] == ["user.provisioned"] + + +@pytest.mark.asyncio +async def test_refuses_an_email_domain_the_org_has_not_verified(setup): + provisioner = Provisioner(setup["session"], setup["connection"]) + + with pytest.raises(ProvisioningError) as raised: + await provisioner.create_user(_record("pat@unclaimed.example")) + + assert raised.value.kind is ProvisioningErrorKind.DOMAIN_NOT_CLAIMED + + +@pytest.mark.asyncio +async def test_adopts_an_existing_user_in_the_same_org(setup): + session = setup["session"] + email = f"invited@{setup['domain']}" + existing = User(email=email, organization_id=setup["organization"].id) + session.add(existing) + await session.flush() + + link = await Provisioner(session, setup["connection"]).create_user(_record(email)) + + assert link.user_id == existing.id + assert not link.created_user + + +@pytest.mark.asyncio +async def test_refuses_to_take_a_user_from_another_org(setup): + session = setup["session"] + email = f"elsewhere@{setup['domain']}" + session.add(User(email=email, organization_id=setup["other"].id)) + await session.flush() + + with pytest.raises(ProvisioningError) as raised: + await Provisioner(session, setup["connection"]).create_user(_record(email)) + + assert raised.value.kind is ProvisioningErrorKind.CONFLICT + + +@pytest.mark.asyncio +async def test_mapped_group_grants_tagged_roles_and_removing_it_revokes_access(setup): + session, role = setup["session"], setup["role"] + provisioner = Provisioner(session, setup["connection"]) + link = await provisioner.create_user(_record(f"tech@{setup['domain']}")) + group = await provisioner.create_group( + GroupRecord(display_name="Techs", external_id="g1", attributes={}), [link.id] + ) + await _map(session, group.id, [role.id]) + await provisioner.reconcile_all() + + assignment = await session.scalar( + select(UserRole).where(UserRole.user_id == link.user_id, UserRole.role_id == role.id) + ) + assert assignment is not None + assert assignment.identity_provider_id == setup["connection"].id + + await provisioner.set_group_members(group, []) + + user = await _user(session, link) + assert not user.is_active + assert ( + await session.scalar( + select(UserRole).where(UserRole.user_id == user.id, UserRole.role_id == role.id) + ) + is None + ) + + +@pytest.mark.asyncio +async def test_a_manual_grant_of_a_mapped_role_survives_unmapping(setup): + session, role = setup["session"], setup["role"] + provisioner = Provisioner(session, setup["connection"]) + link = await provisioner.create_user(_record(f"manual@{setup['domain']}")) + session.add(UserRole(user_id=link.user_id, role_id=role.id, assigned_by="admin@test")) + await session.flush() + group = await provisioner.create_group( + GroupRecord(display_name="Techs", external_id="g1", attributes={}), [link.id] + ) + await _map(session, group.id, [role.id]) + await provisioner.reconcile_all() + + await provisioner.delete_group(group) + + manual = await session.scalar( + select(UserRole).where(UserRole.user_id == link.user_id, UserRole.role_id == role.id) + ) + assert manual is not None and manual.identity_provider_id is None + + +@pytest.mark.asyncio +async def test_a_connection_without_mappings_keeps_ungrouped_users_active(setup): + provisioner = Provisioner(setup["session"], setup["connection"]) + + link = await provisioner.create_user(_record(f"solo@{setup['domain']}")) + + assert (await _user(setup["session"], link)).is_active + + +@pytest.mark.asyncio +async def test_inactive_record_disables_the_user(setup): + provisioner = Provisioner(setup["session"], setup["connection"]) + email = f"leaver@{setup['domain']}" + link = await provisioner.create_user(_record(email)) + + await provisioner.replace_user(link, _record(email, external_id=link.external_id, active=False)) + + assert not (await _user(setup["session"], link)).is_active + assert "user.deprovisioned" in [topic for topic, _ in provisioner.pending_events] + + +@pytest.mark.asyncio +async def test_deleting_a_never_signed_in_user_it_created_removes_the_account(setup): + session = setup["session"] + provisioner = Provisioner(session, setup["connection"]) + link = await provisioner.create_user(_record(f"probe@{setup['domain']}")) + user_id = link.user_id + + await provisioner.delete_user(link) + + session.expunge_all() + assert await session.get(User, user_id) is None + + +@pytest.mark.asyncio +async def test_deleting_a_user_who_signed_in_disables_and_unlinks(setup): + session = setup["session"] + provisioner = Provisioner(session, setup["connection"]) + link = await provisioner.create_user(_record(f"veteran@{setup['domain']}")) + user = await _user(session, link) + user.last_login = datetime.now(timezone.utc) + await session.flush() + + await provisioner.delete_user(link) + + await session.refresh(user) + assert not user.is_active + assert ( + await session.scalar( + select(IdentityProviderUser).where(IdentityProviderUser.user_id == user.id) + ) + is None + ) diff --git a/api/tests/unit/services/test_identity_provisioning_scim.py b/api/tests/unit/services/test_identity_provisioning_scim.py new file mode 100644 index 0000000000..8050b625ad --- /dev/null +++ b/api/tests/unit/services/test_identity_provisioning_scim.py @@ -0,0 +1,150 @@ +"""SCIM request translation, including the identity-provider quirks it +normalizes.""" + +from datetime import datetime, timezone +from uuid import uuid4 + +import pytest + +from src.models.orm.identity_providers import IdentityProviderGroup, IdentityProviderUser +from src.services.identity_provisioning import scim +from src.services.identity_provisioning.scim import ScimError + +ENTERPRISE = scim.ENTERPRISE_SCHEMA +CUSTOM = "urn:ietf:params:scim:schemas:extension:CustomExt:2.0:User" + + +def _patch(*operations: dict) -> dict: + return {"schemas": [scim.PATCH_SCHEMA], "Operations": list(operations)} + + +def _link(**attributes) -> IdentityProviderUser: + now = datetime.now(timezone.utc) + return IdentityProviderUser( + id=uuid4(), + user_id=uuid4(), + external_id="ext-1", + user_name="pat@contoso.example", + active=True, + attributes={ + "userName": "pat@contoso.example", + "emails": [{"value": "pat@contoso.example", "type": "work", "primary": True}], + **attributes, + }, + created_at=now, + updated_at=now, + ) + + +def test_entra_create_keeps_extensions_and_drops_the_password(): + resource, unknown = scim.parse_user( + { + "schemas": [scim.USER_SCHEMA, ENTERPRISE, CUSTOM], + "userName": "Pat@Contoso.example", + "active": "True", + "externalId": "ext-1", + "password": "never-stored", + "emails": [{"value": "pat@contoso.example", "type": "work", "primary": True}], + ENTERPRISE: {"department": "Support", "manager": "manager-id"}, + CUSTOM: {"costCenter": "42"}, + } + ) + record = scim.user_record(resource, unknown) + + assert record.active is True + assert record.email == "pat@contoso.example" + assert record.attributes[ENTERPRISE]["manager"] == {"value": "manager-id"} + assert record.attributes[CUSTOM] == {"costCenter": "42"} + assert "password" not in record.attributes + + +def test_user_without_email_or_email_username_is_refused(): + resource, unknown = scim.parse_user({"userName": "pat"}) + + with pytest.raises(ScimError) as raised: + scim.user_record(resource, unknown) + + assert raised.value.status == 400 + + +def test_entra_patch_quirks_apply(): + resource, unknown = scim.apply_user_patch( + _link(), + _patch( + {"op": "Replace", "path": "active", "value": "False"}, + {"op": "Add", "path": f"{ENTERPRISE}:manager", "value": "manager-2"}, + {"op": "replace", "path": 'emails[type eq "work"].value', "value": "pat2@contoso.example"}, + {"op": "Replace", "value": {"displayName": "Pat Two"}}, + ), + ) + record = scim.user_record(resource, unknown) + + assert record.active is False + assert record.email == "pat2@contoso.example" + assert record.display_name == "Pat Two" + assert record.attributes[ENTERPRISE]["manager"]["value"] == "manager-2" + + +def test_patch_on_an_unknown_extension_is_kept_verbatim(): + resource, unknown = scim.apply_user_patch( + _link(**{CUSTOM: {"costCenter": "42"}}), + _patch({"op": "replace", "path": f"{CUSTOM}:costCenter", "value": "43"}), + ) + + assert scim.user_record(resource, unknown).attributes[CUSTOM] == {"costCenter": "43"} + + +def test_patch_cannot_change_the_id(): + with pytest.raises(ScimError) as raised: + scim.apply_user_patch(_link(), _patch({"op": "replace", "path": "id", "value": "x"})) + + assert raised.value.scim_type == "mutability" + + +def test_okta_member_removal_with_a_value_list_is_normalized(): + now = datetime.now(timezone.utc) + group = IdentityProviderGroup( + id=uuid4(), display_name="Techs", attributes={}, created_at=now, updated_at=now + ) + keep, drop, add = uuid4(), uuid4(), uuid4() + + resource = scim.apply_group_patch( + group, + {keep, drop}, + _patch( + {"op": "remove", "path": "members", "value": [{"value": str(drop)}]}, + {"op": "add", "path": "members", "value": [{"value": str(add)}]}, + ), + ) + + assert scim.group_member_ids(resource) == {keep, add} + + +@pytest.mark.parametrize( + ("expression", "attribute", "value"), + [ + ('userName eq "Pat@Contoso.example"', "userName", "Pat@Contoso.example"), + ('externalId eq "ext-1"', "externalId", "ext-1"), + ], +) +def test_supported_filters(expression, attribute, value): + parsed = scim.parse_filter(expression, scim.USER_FILTER_ATTRIBUTES) + + assert parsed == scim.EqualityFilter(attribute, value) + + +@pytest.mark.parametrize( + "expression", + ['userName sw "pat"', 'userName eq "a" and active eq true', 'title eq "x"', "garbage(("], +) +def test_unsupported_filters_are_invalid_filter_errors(expression): + with pytest.raises(ScimError) as raised: + scim.parse_filter(expression, scim.USER_FILTER_ATTRIBUTES) + + assert (raised.value.status, raised.value.scim_type) == (400, "invalidFilter") + + +def test_excluded_attributes_keep_id_and_schemas(): + body = {"schemas": ["s"], "id": "1", "displayName": "G", "members": [{"value": "x"}]} + + assert scim.project(body, None, "members") == {"schemas": ["s"], "id": "1", "displayName": "G"} diff --git a/api/tests/unit/services/test_operation_catalog.py b/api/tests/unit/services/test_operation_catalog.py index 1c4aa62de5..bd14aebf26 100644 --- a/api/tests/unit/services/test_operation_catalog.py +++ b/api/tests/unit/services/test_operation_catalog.py @@ -370,6 +370,46 @@ ), } +IDENTITY_PROVIDER_OPERATIONS = { + "identityproviders.domains.list": ("GET", "/api/identity-providers/domains", None, None, False), + "identityproviders.domains.create": ("POST", "/api/identity-providers/domains", None, None, False), + "identityproviders.domains.verify": ( + "POST", "/api/identity-providers/domains/{claim_id}/verify", None, None, False, + ), + "identityproviders.domains.delete": ( + "DELETE", "/api/identity-providers/domains/{claim_id}", None, None, False, + ), + "identityproviders.list": ("GET", "/api/identity-providers", None, None, False), + "identityproviders.create": ("POST", "/api/identity-providers", None, None, False), + "identityproviders.get": ("GET", "/api/identity-providers/{identity_provider_id}", None, None, False), + "identityproviders.update": ( + "PATCH", "/api/identity-providers/{identity_provider_id}", None, None, False, + ), + "identityproviders.delete": ( + "DELETE", "/api/identity-providers/{identity_provider_id}", None, None, False, + ), + "identityproviders.token.rotate": ( + "POST", "/api/identity-providers/{identity_provider_id}/token", None, None, False, + ), + "identityproviders.groups.list": ( + "GET", "/api/identity-providers/{identity_provider_id}/groups", None, None, False, + ), + "identityproviders.groups.map": ( + "PUT", + "/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping", + None, + None, + False, + ), + "identityproviders.groups.unmap": ( + "DELETE", + "/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping", + None, + None, + False, + ), +} + PRODUCT_UPDATES_OPERATIONS = { "productupdates.get": ("GET", "/api/product-updates", None, None, False), "productupdates.receipts.create": ( @@ -1202,6 +1242,7 @@ **PLATFORM_JOB_OPERATIONS, **SOLUTION_OPERATIONS, **PRODUCT_UPDATES_OPERATIONS, + **IDENTITY_PROVIDER_OPERATIONS, **ROLE_RESOURCE_ASSIGNMENT_OPERATIONS, } diff --git a/api/tests/unit/services/test_operation_inventory.py b/api/tests/unit/services/test_operation_inventory.py index d65561858a..fa82b29c53 100644 --- a/api/tests/unit/services/test_operation_inventory.py +++ b/api/tests/unit/services/test_operation_inventory.py @@ -25,7 +25,7 @@ def test_every_observed_surface_is_classified_with_a_reason() -> None: "cli": 180, # Includes solution git sources. "manifest": 16, "mcp": 101, # Includes inline Execute Code. - "rest": 724, # Includes App setup, four Solution-source routes, and bulk Organization tags. + "rest": 754, # Includes App setup, four Solution-source routes, bulk Organization tags, and identity providers (13 admin + 17 SCIM). "sdk": 21, } for surface, rows in inventory["uncataloged"].items(): diff --git a/api/tests/unit/test_org_scoping_enforcement.py b/api/tests/unit/test_org_scoping_enforcement.py index 85ff77ec9d..ca88fd6dac 100644 --- a/api/tests/unit/test_org_scoping_enforcement.py +++ b/api/tests/unit/test_org_scoping_enforcement.py @@ -259,6 +259,11 @@ def test_allowlist_entries_still_exist(self) -> None: # A memory store is an owner boundary looked up by its user/organization # identity. It is never name-resolved through the org-to-global cascade. "MemoryStore", + # Identity provider connections and the organization's email-domain claims + # are looked up by id within their organization (or by verified domain), + # never resolved by name through the org-to-global cascade. + "IdentityProvider", + "OrganizationDomain", } diff --git a/api/tests/unit/test_permissions.py b/api/tests/unit/test_permissions.py index d9eb7cf04d..9ecbc41e91 100644 --- a/api/tests/unit/test_permissions.py +++ b/api/tests/unit/test_permissions.py @@ -44,6 +44,7 @@ "filepolicies.read", "filepolicies.readwrite", "forms.read", "forms.readbasic", "forms.readwrite", "home.read", "home.read.all", "home.readwrite", "home.readwrite.all", + "identityproviders.read", "identityproviders.readwrite", "integrations.read", "integrations.readwrite", "knowledge.read", "knowledge.readwrite", "mcp.read", "mcp.readbasic", "mcp.readwrite", diff --git a/docs/generated/access-list.json b/docs/generated/access-list.json index 2d7d4b4ef5..4fc68844bc 100644 --- a/docs/generated/access-list.json +++ b/docs/generated/access-list.json @@ -4331,6 +4331,188 @@ "permission": "userlifecycle.readwrite", "reason": "Renames a custom identity; default identities are never renamed. Decided by the evaluator (src.services.authorization.enforce) at the identity's organization; privileged identities only for a Platform Admin." }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": "identityproviders.list", + "path": "/api/identity-providers", + "permission": "identityproviders.read", + "reason": "List identity provider connections. Decided by the evaluator (src.services.authorization.enforce): Filtered to organizations where the caller holds identityproviders.read." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "POST", + "operation_id": "identityproviders.create", + "path": "/api/identity-providers", + "permission": "identityproviders.readwrite", + "reason": "Create an identity provider connection and its first token. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the organization." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": "identityproviders.domains.list", + "path": "/api/identity-providers/domains", + "permission": "identityproviders.read", + "reason": "List an organization's claimed email domains. Decided by the evaluator (src.services.authorization.enforce): identityproviders.read at the organization." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "POST", + "operation_id": "identityproviders.domains.create", + "path": "/api/identity-providers/domains", + "permission": "identityproviders.readwrite", + "reason": "Claim an email domain for an organization. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the organization." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "DELETE", + "operation_id": "identityproviders.domains.delete", + "path": "/api/identity-providers/domains/{claim_id}", + "permission": "identityproviders.readwrite", + "reason": "Remove a domain claim. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the claim's organization." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "POST", + "operation_id": "identityproviders.domains.verify", + "path": "/api/identity-providers/domains/{claim_id}/verify", + "permission": "identityproviders.readwrite", + "reason": "Verify a domain claim through its DNS TXT record. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the claim's organization." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "DELETE", + "operation_id": "identityproviders.delete", + "path": "/api/identity-providers/{identity_provider_id}", + "permission": "identityproviders.readwrite", + "reason": "Delete an identity provider connection. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the connection's organization." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": "identityproviders.get", + "path": "/api/identity-providers/{identity_provider_id}", + "permission": "identityproviders.read", + "reason": "Read an identity provider connection. Decided by the evaluator (src.services.authorization.enforce): identityproviders.read at the connection's organization." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "PATCH", + "operation_id": "identityproviders.update", + "path": "/api/identity-providers/{identity_provider_id}", + "permission": "identityproviders.readwrite", + "reason": "Update an identity provider connection. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the connection's organization." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": "identityproviders.groups.list", + "path": "/api/identity-providers/{identity_provider_id}/groups", + "permission": "identityproviders.read", + "reason": "List the groups an identity provider pushed. Decided by the evaluator (src.services.authorization.enforce): identityproviders.read at the connection's organization." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "DELETE", + "operation_id": "identityproviders.groups.unmap", + "path": "/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping", + "permission": "identityproviders.readwrite", + "reason": "Remove an identity provider group mapping. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the connection's organization." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "PUT", + "operation_id": "identityproviders.groups.map", + "path": "/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping", + "permission": "identityproviders.readwrite", + "reason": "Map an identity provider group to roles. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the connection's organization, plus the manual role-assignment checks for each mapped role." + }, + { + "access_class": "permission", + "boundary": "organization", + "current_gate": "evaluator", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "POST", + "operation_id": "identityproviders.token.rotate", + "path": "/api/identity-providers/{identity_provider_id}/token", + "permission": "identityproviders.readwrite", + "reason": "Issue a new identity provider token. Decided by the evaluator (src.services.authorization.enforce): identityproviders.readwrite at the connection's organization." + }, { "access_class": "permission", "boundary": "organization", @@ -7055,6 +7237,244 @@ "permission": "workflows.read", "reason": "Current original-user permission and reach; resource sharing, parent-resource launch, and Solution boundaries remain independent." }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Groups", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "POST", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Groups", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "DELETE", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Groups/{group_id}", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Groups/{group_id}", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "PATCH", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Groups/{group_id}", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "PUT", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Groups/{group_id}", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/ResourceTypes", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/ResourceTypes/{type_id}", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Schemas", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Schemas/{schema_id}", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/ServiceProviderConfig", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Users", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "POST", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Users", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "DELETE", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Users/{user_id}", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "GET", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Users/{user_id}", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "PATCH", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Users/{user_id}", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, + { + "access_class": "public", + "boundary": null, + "current_gate": "none", + "inline_checks": [], + "inline_effect": null, + "intended_change": null, + "mcp_tool": null, + "method": "PUT", + "operation_id": null, + "path": "/api/scim/v2/{identity_provider_id}/Users/{user_id}", + "permission": null, + "reason": "SCIM 2.0 provisioning endpoint called by an external identity provider; authenticated by the connection's bearer token (src.services.identity_provisioning.tokens), not a user session." + }, { "access_class": "permission", "boundary": "organization", diff --git a/docs/generated/operation-surface-inventory.json b/docs/generated/operation-surface-inventory.json index 660c89bdc1..d3d4f7cd5b 100644 --- a/docs/generated/operation-surface-inventory.json +++ b/docs/generated/operation-surface-inventory.json @@ -10741,13 +10741,690 @@ "summary": "Rename or move a file or folder in the browser editor", "target_kind": "workspace" } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.domains.list", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.read" + ], + "async_policy": "synchronous", + "audit_event": null, + "authorization_resolver": "identityproviders.read at the organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.domains.list", + "rest": { + "method": "GET", + "path": "/api/identity-providers/domains", + "request_model": null, + "response_model": "OrganizationDomainResponse" + }, + "side_effects": [], + "summary": "List an organization's claimed email domains", + "target_kind": "collection" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.domains.create", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.readwrite" + ], + "async_policy": "synchronous", + "audit_event": "identity_provider.domain_claimed", + "authorization_resolver": "identityproviders.readwrite at the organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.domains.create", + "rest": { + "method": "POST", + "path": "/api/identity-providers/domains", + "request_model": "OrganizationDomainCreate", + "response_model": "OrganizationDomainResponse" + }, + "side_effects": [], + "summary": "Claim an email domain for an organization", + "target_kind": "collection" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.domains.verify", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.readwrite" + ], + "async_policy": "synchronous", + "audit_event": "identity_provider.domain_verified", + "authorization_resolver": "identityproviders.readwrite at the claim's organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.domains.verify", + "rest": { + "method": "POST", + "path": "/api/identity-providers/domains/{claim_id}/verify", + "request_model": null, + "response_model": "OrganizationDomainResponse" + }, + "side_effects": [], + "summary": "Verify a domain claim through its DNS TXT record", + "target_kind": "resource" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.domains.delete", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.readwrite" + ], + "async_policy": "synchronous", + "audit_event": "identity_provider.domain_removed", + "authorization_resolver": "identityproviders.readwrite at the claim's organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.domains.delete", + "rest": { + "method": "DELETE", + "path": "/api/identity-providers/domains/{claim_id}", + "request_model": null, + "response_model": null + }, + "side_effects": [], + "summary": "Remove a domain claim", + "target_kind": "resource" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.list", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.read" + ], + "async_policy": "synchronous", + "audit_event": null, + "authorization_resolver": "Filtered to organizations where the caller holds identityproviders.read", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.list", + "rest": { + "method": "GET", + "path": "/api/identity-providers", + "request_model": null, + "response_model": "IdentityProviderResponse" + }, + "side_effects": [], + "summary": "List identity provider connections", + "target_kind": "collection" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.create", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.readwrite" + ], + "async_policy": "synchronous", + "audit_event": "identity_provider.create", + "authorization_resolver": "identityproviders.readwrite at the organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.create", + "rest": { + "method": "POST", + "path": "/api/identity-providers", + "request_model": "IdentityProviderCreate", + "response_model": "IdentityProviderCreated" + }, + "side_effects": [], + "summary": "Create an identity provider connection and its first token", + "target_kind": "collection" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.get", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.read" + ], + "async_policy": "synchronous", + "audit_event": null, + "authorization_resolver": "identityproviders.read at the connection's organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.get", + "rest": { + "method": "GET", + "path": "/api/identity-providers/{identity_provider_id}", + "request_model": null, + "response_model": "IdentityProviderResponse" + }, + "side_effects": [], + "summary": "Read an identity provider connection", + "target_kind": "resource" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.update", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.readwrite" + ], + "async_policy": "synchronous", + "audit_event": "identity_provider.update", + "authorization_resolver": "identityproviders.readwrite at the connection's organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.update", + "rest": { + "method": "PATCH", + "path": "/api/identity-providers/{identity_provider_id}", + "request_model": "IdentityProviderUpdate", + "response_model": "IdentityProviderResponse" + }, + "side_effects": [], + "summary": "Update an identity provider connection", + "target_kind": "resource" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.delete", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.readwrite" + ], + "async_policy": "synchronous", + "audit_event": "identity_provider.delete", + "authorization_resolver": "identityproviders.readwrite at the connection's organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.delete", + "rest": { + "method": "DELETE", + "path": "/api/identity-providers/{identity_provider_id}", + "request_model": null, + "response_model": null + }, + "side_effects": [], + "summary": "Delete an identity provider connection", + "target_kind": "resource" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.token.rotate", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.readwrite" + ], + "async_policy": "synchronous", + "audit_event": "identity_provider.token_rotated", + "authorization_resolver": "identityproviders.readwrite at the connection's organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.token.rotate", + "rest": { + "method": "POST", + "path": "/api/identity-providers/{identity_provider_id}/token", + "request_model": "IdentityProviderTokenRotate", + "response_model": "IdentityProviderTokenIssued" + }, + "side_effects": [], + "summary": "Issue a new identity provider token", + "target_kind": "resource" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.groups.list", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.read" + ], + "async_policy": "synchronous", + "audit_event": null, + "authorization_resolver": "identityproviders.read at the connection's organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.groups.list", + "rest": { + "method": "GET", + "path": "/api/identity-providers/{identity_provider_id}/groups", + "request_model": null, + "response_model": "IdentityProviderGroupResponse" + }, + "side_effects": [], + "summary": "List the groups an identity provider pushed", + "target_kind": "resource" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.groups.map", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.readwrite", + "roleassignments.readwrite" + ], + "async_policy": "synchronous", + "audit_event": "identity_provider.group_mapped", + "authorization_resolver": "identityproviders.readwrite at the connection's organization, plus the manual role-assignment checks for each mapped role", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.groups.map", + "rest": { + "method": "PUT", + "path": "/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping", + "request_model": "IdentityProviderGroupMappingUpdate", + "response_model": "IdentityProviderGroupResponse" + }, + "side_effects": [], + "summary": "Map an identity provider group to roles", + "target_kind": "resource" + } + }, + { + "observed": { + "cli": { + "path": null, + "status": "intentionally_unsupported" + }, + "manifest": { + "entity": null, + "status": "intentionally_unsupported" + }, + "mcp": { + "name": null, + "status": "intentionally_unsupported" + }, + "rest": { + "operation_id": "identityproviders.groups.unmap", + "status": "exact_parity" + }, + "sdk": { + "binding": null, + "status": "missing_surface" + } + }, + "operation": { + "action_scopes": [ + "identityproviders.readwrite" + ], + "async_policy": "synchronous", + "audit_event": "identity_provider.group_unmapped", + "authorization_resolver": "identityproviders.readwrite at the connection's organization", + "cli": null, + "exclusions": { + "cli": "Identity provider administration is available through the administrative UI and REST API.", + "manifest": "Identity provider connections, tokens and domain claims are deployment-local identity state.", + "mcp": "Identity provider administration is intentionally excluded from coding-harness MCP tools.", + "native_builder": "Identity provider administration is not a coding target." + }, + "manifest": null, + "mcp": null, + "native_builder": false, + "operation_id": "identityproviders.groups.unmap", + "rest": { + "method": "DELETE", + "path": "/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping", + "request_model": null, + "response_model": null + }, + "side_effects": [], + "summary": "Remove an identity provider group mapping", + "target_kind": "resource" + } } ], "counts": { "cli": 180, "manifest": 16, "mcp": 101, - "rest": 724, + "rest": 754, "sdk": 21 }, "schema_version": 1, @@ -14300,6 +14977,125 @@ "reason": "REST operation has not entered the canonical operation catalog yet.", "status": "missing_surface" }, + { + "method": "GET", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Groups", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "POST", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Groups", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "DELETE", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Groups/{group_id}", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "GET", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Groups/{group_id}", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "PATCH", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Groups/{group_id}", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "PUT", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Groups/{group_id}", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "GET", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/ResourceTypes", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "GET", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/ResourceTypes/{type_id}", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "GET", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Schemas", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "GET", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Schemas/{schema_id}", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "GET", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/ServiceProviderConfig", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "GET", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Users", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "POST", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Users", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "DELETE", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Users/{user_id}", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "GET", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Users/{user_id}", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "PATCH", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Users/{user_id}", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, + { + "method": "PUT", + "operation_id": "", + "path": "/api/scim/v2/{identity_provider_id}/Users/{user_id}", + "reason": "SCIM 2.0 wire protocol an external identity provider calls with a connection token; writes go through the identity provisioning core, administered by the identityproviders operations.", + "status": "transport_only" + }, { "method": "POST", "operation_id": "", diff --git a/docs/plans/2026-11-16-scim-identity-providers.md b/docs/plans/2026-11-16-scim-identity-providers.md index d05867271f..69bcadd66a 100644 --- a/docs/plans/2026-11-16-scim-identity-providers.md +++ b/docs/plans/2026-11-16-scim-identity-providers.md @@ -78,7 +78,7 @@ Out of scope for v1: | 8 | One active bearer token per connection, stored hashed, with an expiry. Rotation creates a new token and shortens the previous token's expiry to **one hour** from the rotation, so the old and new tokens overlap briefly. | | 9 | Keep all the data. The full SCIM payload, enterprise extension included, is stored on the user's link. Odd vendor extensions are normalized or ignored as they come up. | | 10 | Provision / update / deprovision / group-membership changes emit platform events. | -| 11 | External (portal) users: a per-connection **Provision as External users** setting, off by default. An External user (`User.is_external`) keeps the same org cascade but is not covered by `access_level="authenticated"` sharing; they see only what a role or an explicit grant gives them. | +| 11 | External (portal) users: a per-connection **Provision as External users** setting, off by default. It applies to users the connection creates; an adopted user keeps their current setting. An External user (`User.is_external`) keeps the same org cascade but is not covered by `access_level="authenticated"` sharing; they see only what a role or an explicit grant gives them. | | 12 | The provisioning core is protocol-independent. Connections have `source_type` = `scim` \| `google_directory`. The UI is the same for both: groups seen, "N/N mapped", last sync, credential expiry. | ## Data model @@ -167,11 +167,17 @@ not be expired or revoked, and the connection must be enabled. Responses use (`src/core/rate_limit.py`) and a body-size cap. Libraries: -- `scim2-models` (Apache-2.0, Pydantic v2, pinned) for resource and message - validation and serialization. +- `scim2-models` (Apache-2.0, Pydantic v2, pinned to `0.12.x`) validates + resources, applies `PatchOp` requests (`PatchOp.patch`) and parses filters + (`ScimFilter`, which brings in `lark`). Requests are normalized first for + quirks the library rejects: Entra's bare-string `manager`, and + Okta's `remove members` with a value list. Vendor extensions with no + schema are kept verbatim and patched directly. - `dnspython`, now a direct dependency (it was transitive), for TXT lookups. -- `scim2-tester` runs RFC compliance checks in the e2e suite. -- No filter-grammar dependency; the supported subset is small and explicit. +- `scim2-tester` (dev only) runs the RFC discovery checks and the Group CRUD + checks in the e2e suite. User CRUD is covered by an Entra/Okta-shaped + lifecycle test instead, because the tester invents random email domains, + which the domain-claim rule correctly refuses. ## Management API and permissions diff --git a/pyproject.toml b/pyproject.toml index 7487529097..f5015e6b5b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -62,6 +62,10 @@ dependencies = [ "webauthn>=2.7.1", # WebAuthn/Passkeys support "authlib>=1.9.5", # OIDC relying party: JWKS + id_token claim validation "altcha>=2.1.0", # Self-hosted proof-of-work protection for public forms + # SCIM 2.0 (RFC 7643/7644) resource, PatchOp and filter models for the + # identity-provider provisioning endpoint. Pre-1.0: pinned to a minor line. + "scim2-models>=0.12.4,<0.13", + "dnspython>=2.8.0", # DNS TXT verification of organization domain claims # ========================================================================= # Configuration @@ -202,4 +206,6 @@ dev = [ # still declares watchdog for laptop installs. "debugpy", # Remote debugging (VS Code attach) "watchdog", # File watching for hot reload (watchmedo command) + # RFC 7643/7644 compliance checks against the SCIM endpoint (e2e suite) + "scim2-tester>=0.6.4,<0.7", ] diff --git a/requirements-dev.lock b/requirements-dev.lock index b9e2fc01bc..d6dd2ebb89 100644 --- a/requirements-dev.lock +++ b/requirements-dev.lock @@ -889,7 +889,9 @@ distro==1.9.0 \ dnspython==2.8.0 \ --hash=sha256:01d9bbc4a2d76bf0db7c1f729812ded6d912bd318d3b1cf81d30c0f845dbf3af \ --hash=sha256:181d3c6996452cb1189c4046c61599b84a5a86e099562ffde77d26984ff26d0f - # via email-validator + # via + # bifrost-api (pyproject.toml) + # email-validator docstring-parser==0.18.0 \ --hash=sha256:292510982205c12b1248696f44959db3cdd1740237a968ea1e2e7a900eeb2015 \ --hash=sha256:b3fcbed555c47d8479be0796ef7e19c2670d428d72e96da63f3a40122860374b @@ -897,7 +899,9 @@ docstring-parser==0.18.0 \ email-validator==2.3.0 \ --hash=sha256:80f13f623413e6b197ae73bb10bf4eb0908faf509ad8362c5edeb0be7fd450b4 \ --hash=sha256:9fc05c37f2f6cf439ff414f8fc46d917929974a82244c20eb10231ba60c54426 - # via bifrost-api (pyproject.toml) + # via + # bifrost-api (pyproject.toml) + # pydantic et-xmlfile==2.0.0 \ --hash=sha256:7a91720bc756843502c3b7504c77b8fe44217c85c537d85037f0f536151b2caa \ --hash=sha256:dab3f4764309081ce75662649be815c4c9081e88f0837825f90fd28317d4da54 @@ -1490,6 +1494,10 @@ keyring==25.7.0 \ --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b # via bifrost-api (pyproject.toml) +lark==1.3.1 \ + --hash=sha256:b426a7a6d6d53189d318f2b6236ab5d6429eaf09259f1ca33eb716eed10d2905 \ + --hash=sha256:c629b661023a014c37da873b4ff58a817398d12635d3bbb2c5a03be7fe5d1e12 + # via scim2-models libcst==1.9.0 \ --hash=sha256:02ee2dbdb5c218116f16a021350fc267a14be205b50d81c33f5d73857ab6fbf7 \ --hash=sha256:03165a264653bb77f6a11b412ae09c08bdb0c25864f3b8d42b816ac64b9d4b9e \ @@ -2390,7 +2398,7 @@ pycparser==3.0 \ --hash=sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29 \ --hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992 # via cffi -pydantic==2.14.3 \ +pydantic[email]==2.14.3 \ --hash=sha256:1054f237f4d4733e5a30e1e2333bd33a2004ab9ab1510bd053a9a31bb48e232d \ --hash=sha256:3f7b347782ab7e4cf4c798496ae3a82787d9f21ec12676a9c564355eeef8daa8 # via @@ -2405,6 +2413,7 @@ pydantic==2.14.3 \ # pydantic-ai-slim # pydantic-graph # pydantic-settings + # scim2-models pydantic-ai-harness==0.67.0 \ --hash=sha256:5149401a3f61b52598324e9f3458f57b757ff1606ed45bb21560945ce438768b \ --hash=sha256:68749d5e6c096c6ea7d2a12791f821eb0f08ee77405f3de493fb9f464ef6ed2e @@ -3159,6 +3168,21 @@ ruff==0.15.12 \ --hash=sha256:fb129f40f114f089ebe0ca56c0d251cf2061b17651d464bb6478dc01e69f11f5 \ --hash=sha256:fe87510d000220aa1ed530d4448a7c696a0cae1213e5ec30e5874287b66557b5 # via bifrost-api (pyproject.toml) +scim2-client==0.14.1 \ + --hash=sha256:6f60caa7f3df6657a28885bd7b3dbbe5773ffb747bdc3cee0eaf35cb02452bc7 \ + --hash=sha256:bf7c3f3cf9da7cb8e813eaf560cfeab8037f6c1b5ab81b56cf137d8f94d3b7cd + # via scim2-tester +scim2-models==0.12.4 \ + --hash=sha256:d219558358557a674e0a5b85f6d11cb2cd8cdf3182c4942df0956891be052e8e \ + --hash=sha256:d3cfaafbe49d6daa61b63afe0f0536774e7543218eaa222164ccb95ecb8871aa + # via + # bifrost-api (pyproject.toml) + # scim2-client + # scim2-tester +scim2-tester==0.6.4 \ + --hash=sha256:a4d5e522f228d2d2afaa147d66c641bc5659dcdde5967411fb4b830b08198a8a \ + --hash=sha256:df8c582d391511ae7f0d35e0d7bfe4916c79571af22768ab96cc43599765b144 + # via bifrost-api (pyproject.toml) secretstorage==3.5.0 \ --hash=sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137 \ --hash=sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be diff --git a/requirements.lock b/requirements.lock index 2939ce6d15..2749ed4c10 100644 --- a/requirements.lock +++ b/requirements.lock @@ -749,7 +749,9 @@ distro==1.9.0 \ dnspython==2.8.0 \ --hash=sha256:01d9bbc4a2d76bf0db7c1f729812ded6d912bd318d3b1cf81d30c0f845dbf3af \ --hash=sha256:181d3c6996452cb1189c4046c61599b84a5a86e099562ffde77d26984ff26d0f - # via email-validator + # via + # bifrost-api (pyproject.toml) + # email-validator docstring-parser==0.18.0 \ --hash=sha256:292510982205c12b1248696f44959db3cdd1740237a968ea1e2e7a900eeb2015 \ --hash=sha256:b3fcbed555c47d8479be0796ef7e19c2670d428d72e96da63f3a40122860374b @@ -757,7 +759,9 @@ docstring-parser==0.18.0 \ email-validator==2.3.0 \ --hash=sha256:80f13f623413e6b197ae73bb10bf4eb0908faf509ad8362c5edeb0be7fd450b4 \ --hash=sha256:9fc05c37f2f6cf439ff414f8fc46d917929974a82244c20eb10231ba60c54426 - # via bifrost-api (pyproject.toml) + # via + # bifrost-api (pyproject.toml) + # pydantic et-xmlfile==2.0.0 \ --hash=sha256:7a91720bc756843502c3b7504c77b8fe44217c85c537d85037f0f536151b2caa \ --hash=sha256:dab3f4764309081ce75662649be815c4c9081e88f0837825f90fd28317d4da54 @@ -1346,6 +1350,10 @@ keyring==25.7.0 \ --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b # via bifrost-api (pyproject.toml) +lark==1.3.1 \ + --hash=sha256:b426a7a6d6d53189d318f2b6236ab5d6429eaf09259f1ca33eb716eed10d2905 \ + --hash=sha256:c629b661023a014c37da873b4ff58a817398d12635d3bbb2c5a03be7fe5d1e12 + # via scim2-models libcst==1.9.0 \ --hash=sha256:02ee2dbdb5c218116f16a021350fc267a14be205b50d81c33f5d73857ab6fbf7 \ --hash=sha256:03165a264653bb77f6a11b412ae09c08bdb0c25864f3b8d42b816ac64b9d4b9e \ @@ -2238,7 +2246,7 @@ pycparser==3.0 \ --hash=sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29 \ --hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992 # via cffi -pydantic==2.14.3 \ +pydantic[email]==2.14.3 \ --hash=sha256:1054f237f4d4733e5a30e1e2333bd33a2004ab9ab1510bd053a9a31bb48e232d \ --hash=sha256:3f7b347782ab7e4cf4c798496ae3a82787d9f21ec12676a9c564355eeef8daa8 # via @@ -2253,6 +2261,7 @@ pydantic==2.14.3 \ # pydantic-ai-slim # pydantic-graph # pydantic-settings + # scim2-models pydantic-ai-harness==0.67.0 \ --hash=sha256:5149401a3f61b52598324e9f3458f57b757ff1606ed45bb21560945ce438768b \ --hash=sha256:68749d5e6c096c6ea7d2a12791f821eb0f08ee77405f3de493fb9f464ef6ed2e @@ -2981,6 +2990,10 @@ ruff==0.15.12 \ --hash=sha256:fb129f40f114f089ebe0ca56c0d251cf2061b17651d464bb6478dc01e69f11f5 \ --hash=sha256:fe87510d000220aa1ed530d4448a7c696a0cae1213e5ec30e5874287b66557b5 # via bifrost-api (pyproject.toml) +scim2-models==0.12.4 \ + --hash=sha256:d219558358557a674e0a5b85f6d11cb2cd8cdf3182c4942df0956891be052e8e \ + --hash=sha256:d3cfaafbe49d6daa61b63afe0f0536774e7543218eaa222164ccb95ecb8871aa + # via bifrost-api (pyproject.toml) secretstorage==3.5.0 \ --hash=sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137 \ --hash=sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be From f8146c89fe9e9925de96d6f9ef50ea193f6059ca Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Mon, 16 Nov 2026 13:36:05 -0500 Subject: [PATCH 03/26] Regenerate client types for identity provider connections Rename the group-mapping role model to MappedRole: as RoleSummary it collided with the role-assignment schema of the same name, and OpenAPI silently kept only one of them. Co-Authored-By: Claude Opus 5.5 --- .../models/contracts/identity_providers.py | 4 +- api/src/services/identity_providers.py | 6 +- client/src/lib/v1.d.ts | 788 ++++++++++++++++++ 3 files changed, 793 insertions(+), 5 deletions(-) diff --git a/api/src/models/contracts/identity_providers.py b/api/src/models/contracts/identity_providers.py index 9c4e5e63d6..98f3bd9378 100644 --- a/api/src/models/contracts/identity_providers.py +++ b/api/src/models/contracts/identity_providers.py @@ -103,7 +103,7 @@ class IdentityProviderCreated(BaseModel): token: IdentityProviderTokenIssued -class RoleSummary(BaseModel): +class MappedRole(BaseModel): id: UUID name: str @@ -114,7 +114,7 @@ class IdentityProviderGroupResponse(BaseModel): external_id: str | None member_count: int mapped: bool - roles: list[RoleSummary] + roles: list[MappedRole] class IdentityProviderGroupMappingUpdate(BaseModel): diff --git a/api/src/services/identity_providers.py b/api/src/services/identity_providers.py index f08a558015..a916b1934b 100644 --- a/api/src/services/identity_providers.py +++ b/api/src/services/identity_providers.py @@ -25,7 +25,7 @@ IdentityProviderTokenIssued, IdentityProviderUpdate, OrganizationDomainResponse, - RoleSummary, + MappedRole, ) from src.models.orm.identity_providers import ( IdentityProvider, @@ -418,7 +418,7 @@ async def list_groups( ) ) ) if group_ids else set() - roles: dict[UUID, list[RoleSummary]] = {} + roles: dict[UUID, list[MappedRole]] = {} if group_ids: for group_id, role_id, role_name in ( await session.execute( @@ -428,7 +428,7 @@ async def list_groups( .order_by(Role.name) ) ).all(): - roles.setdefault(group_id, []).append(RoleSummary(id=role_id, name=role_name)) + roles.setdefault(group_id, []).append(MappedRole(id=role_id, name=role_name)) return [ IdentityProviderGroupResponse( id=g.id, diff --git a/client/src/lib/v1.d.ts b/client/src/lib/v1.d.ts index 3ae434096b..bbb5b585ea 100644 --- a/client/src/lib/v1.d.ts +++ b/client/src/lib/v1.d.ts @@ -9296,6 +9296,147 @@ export interface paths { patch?: never; trace?: never; }; + "/api/identity-providers/domains": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + /** List Domains */ + get: operations["identityproviders.domains.list"]; + put?: never; + /** Add Domain */ + post: operations["identityproviders.domains.create"]; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; + "/api/identity-providers/domains/{claim_id}/verify": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + get?: never; + put?: never; + /** Verify Domain */ + post: operations["identityproviders.domains.verify"]; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; + "/api/identity-providers/domains/{claim_id}": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + get?: never; + put?: never; + post?: never; + /** Delete Domain */ + delete: operations["identityproviders.domains.delete"]; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; + "/api/identity-providers": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + /** List Identity Providers */ + get: operations["identityproviders.list"]; + put?: never; + /** Create Identity Provider */ + post: operations["identityproviders.create"]; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; + "/api/identity-providers/{identity_provider_id}": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + /** Get Identity Provider */ + get: operations["identityproviders.get"]; + put?: never; + post?: never; + /** Delete Identity Provider */ + delete: operations["identityproviders.delete"]; + options?: never; + head?: never; + /** Update Identity Provider */ + patch: operations["identityproviders.update"]; + trace?: never; + }; + "/api/identity-providers/{identity_provider_id}/token": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + get?: never; + put?: never; + /** Rotate Identity Provider Token */ + post: operations["identityproviders.token.rotate"]; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; + "/api/identity-providers/{identity_provider_id}/groups": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + /** List Identity Provider Groups */ + get: operations["identityproviders.groups.list"]; + put?: never; + post?: never; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; + "/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + get?: never; + /** Map Identity Provider Group */ + put: operations["identityproviders.groups.map"]; + post?: never; + /** Unmap Identity Provider Group */ + delete: operations["identityproviders.groups.unmap"]; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; "/api/tables": { parameters: { query?: never; @@ -21394,6 +21535,170 @@ export interface components { */ base_role_id?: string | null; }; + /** IdentityProviderCreate */ + IdentityProviderCreate: { + /** + * Organization Id + * Format: uuid + */ + organization_id: string; + /** Name */ + name: string; + /** + * Vendor + * @enum {string} + */ + vendor: "entra" | "okta" | "jumpcloud" | "google" | "other"; + /** + * Source Type + * @default scim + * @constant + */ + source_type: "scim"; + /** + * Provision As External + * @default false + */ + provision_as_external: boolean; + /** + * Token Lifetime Days + * @default 365 + * @enum {integer} + */ + token_lifetime_days: 90 | 180 | 365 | 730; + }; + /** IdentityProviderCreated */ + IdentityProviderCreated: { + identity_provider: components["schemas"]["IdentityProviderResponse"]; + token: components["schemas"]["IdentityProviderTokenIssued"]; + }; + /** + * IdentityProviderGroupMappingUpdate + * @description Map a group. An empty ``role_ids`` maps it for access only. + */ + IdentityProviderGroupMappingUpdate: { + /** Role Ids */ + role_ids?: string[]; + }; + /** IdentityProviderGroupResponse */ + IdentityProviderGroupResponse: { + /** + * Id + * Format: uuid + */ + id: string; + /** Display Name */ + display_name: string; + /** External Id */ + external_id: string | null; + /** Member Count */ + member_count: number; + /** Mapped */ + mapped: boolean; + /** Roles */ + roles: components["schemas"]["MappedRole"][]; + }; + /** IdentityProviderResponse */ + IdentityProviderResponse: { + /** + * Id + * Format: uuid + */ + id: string; + /** + * Organization Id + * Format: uuid + */ + organization_id: string; + /** Organization Name */ + organization_name: string; + /** Name */ + name: string; + /** Source Type */ + source_type: string; + /** + * Vendor + * @enum {string} + */ + vendor: "entra" | "okta" | "jumpcloud" | "google" | "other"; + /** Is Enabled */ + is_enabled: boolean; + /** Provision As External */ + provision_as_external: boolean; + /** Scim Base Url */ + scim_base_url: string; + token: components["schemas"]["IdentityProviderTokenInfo"] | null; + /** Verified Domains */ + verified_domains: string[]; + /** User Count */ + user_count: number; + /** Group Count */ + group_count: number; + /** Mapped Group Count */ + mapped_group_count: number; + /** Last Request At */ + last_request_at: string | null; + /** Last Sync At */ + last_sync_at: string | null; + /** + * Created At + * Format: date-time + */ + created_at: string; + /** + * Updated At + * Format: date-time + */ + updated_at: string; + }; + /** IdentityProviderTokenInfo */ + IdentityProviderTokenInfo: { + /** Hint */ + hint: string; + /** + * Expires At + * Format: date-time + */ + expires_at: string; + /** + * Created At + * Format: date-time + */ + created_at: string; + /** Last Used At */ + last_used_at: string | null; + }; + /** + * IdentityProviderTokenIssued + * @description The raw token is returned only here, once. + */ + IdentityProviderTokenIssued: { + /** Token */ + token: string; + info: components["schemas"]["IdentityProviderTokenInfo"]; + /** Previous Tokens Expire At */ + previous_tokens_expire_at: string | null; + }; + /** IdentityProviderTokenRotate */ + IdentityProviderTokenRotate: { + /** + * Lifetime Days + * @default 365 + * @enum {integer} + */ + lifetime_days: 90 | 180 | 365 | 730; + }; + /** IdentityProviderUpdate */ + IdentityProviderUpdate: { + /** Name */ + name?: string | null; + /** Vendor */ + vendor?: ("entra" | "okta" | "jumpcloud" | "google" | "other") | null; + /** Is Enabled */ + is_enabled?: boolean | null; + /** Provision As External */ + provision_as_external?: boolean | null; + }; /** IdentityPublic */ IdentityPublic: { /** @@ -21439,6 +21744,24 @@ export interface components { /** Boundaries */ boundaries: components["schemas"]["RoleBoundaryPublic"][]; }; + /** + * IdentitySource + * @description Where a user's identity comes from (None on a user means Native). + */ + IdentitySource: { + /** + * Identity Provider Id + * Format: uuid + */ + identity_provider_id: string; + /** Name */ + name: string; + /** + * Vendor + * @enum {string} + */ + vendor: "entra" | "okta" | "jumpcloud" | "google" | "other"; + }; /** IdentityUpdate */ IdentityUpdate: { /** Name */ @@ -23645,6 +23968,16 @@ export interface components { */ last_reindex?: string | null; }; + /** MappedRole */ + MappedRole: { + /** + * Id + * Format: uuid + */ + id: string; + /** Name */ + name: string; + }; /** * MappingAuthorizeRequest * @description Request to begin OAuth authorize flow for a specific mapping. @@ -24769,6 +25102,44 @@ export interface components { */ tags?: string[]; }; + /** OrganizationDomainCreate */ + OrganizationDomainCreate: { + /** + * Organization Id + * Format: uuid + */ + organization_id: string; + /** Domain */ + domain: string; + }; + /** OrganizationDomainResponse */ + OrganizationDomainResponse: { + /** + * Id + * Format: uuid + */ + id: string; + /** + * Organization Id + * Format: uuid + */ + organization_id: string; + /** Domain */ + domain: string; + /** Verified At */ + verified_at: string | null; + /** Last Checked At */ + last_checked_at: string | null; + /** Txt Record Name */ + txt_record_name: string; + /** Txt Record Value */ + txt_record_value: string; + /** + * Created At + * Format: date-time + */ + created_at: string; + }; /** * OrganizationMetricsResponse * @description Response for organization metrics breakdown. @@ -31118,6 +31489,8 @@ export interface components { * @default false */ is_protected: boolean; + /** @description The identity provider connection that manages this user; null for a Native user. */ + identity_source?: components["schemas"]["IdentitySource"] | null; }; /** * UserResponse @@ -49826,6 +50199,421 @@ export interface operations { }; }; }; + "identityproviders.domains.list": { + parameters: { + query: { + organization_id: string; + }; + header?: never; + path?: never; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["OrganizationDomainResponse"][]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.domains.create": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + requestBody: { + content: { + "application/json": components["schemas"]["OrganizationDomainCreate"]; + }; + }; + responses: { + /** @description Successful Response */ + 201: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["OrganizationDomainResponse"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.domains.verify": { + parameters: { + query?: never; + header?: never; + path: { + claim_id: string; + }; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["OrganizationDomainResponse"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.domains.delete": { + parameters: { + query?: never; + header?: never; + path: { + claim_id: string; + }; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 204: { + headers: { + [name: string]: unknown; + }; + content?: never; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.list": { + parameters: { + query?: { + organization_id?: string | null; + }; + header?: never; + path?: never; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["IdentityProviderResponse"][]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.create": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + requestBody: { + content: { + "application/json": components["schemas"]["IdentityProviderCreate"]; + }; + }; + responses: { + /** @description Successful Response */ + 201: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["IdentityProviderCreated"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.get": { + parameters: { + query?: never; + header?: never; + path: { + identity_provider_id: string; + }; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["IdentityProviderResponse"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.delete": { + parameters: { + query?: never; + header?: never; + path: { + identity_provider_id: string; + }; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 204: { + headers: { + [name: string]: unknown; + }; + content?: never; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.update": { + parameters: { + query?: never; + header?: never; + path: { + identity_provider_id: string; + }; + cookie?: never; + }; + requestBody: { + content: { + "application/json": components["schemas"]["IdentityProviderUpdate"]; + }; + }; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["IdentityProviderResponse"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.token.rotate": { + parameters: { + query?: never; + header?: never; + path: { + identity_provider_id: string; + }; + cookie?: never; + }; + requestBody: { + content: { + "application/json": components["schemas"]["IdentityProviderTokenRotate"]; + }; + }; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["IdentityProviderTokenIssued"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.groups.list": { + parameters: { + query?: never; + header?: never; + path: { + identity_provider_id: string; + }; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["IdentityProviderGroupResponse"][]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.groups.map": { + parameters: { + query?: never; + header?: never; + path: { + identity_provider_id: string; + group_id: string; + }; + cookie?: never; + }; + requestBody: { + content: { + "application/json": components["schemas"]["IdentityProviderGroupMappingUpdate"]; + }; + }; + responses: { + /** @description Successful Response */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["IdentityProviderGroupResponse"]; + }; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; + "identityproviders.groups.unmap": { + parameters: { + query?: never; + header?: never; + path: { + identity_provider_id: string; + group_id: string; + }; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description Successful Response */ + 204: { + headers: { + [name: string]: unknown; + }; + content?: never; + }; + /** @description Validation Error */ + 422: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["HTTPValidationError"]; + }; + }; + }; + }; "tables.list": { parameters: { query?: { From 59fb08853f7b049f2b2da2d0f3a6d5adb4f97cbf Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Mon, 16 Nov 2026 13:51:55 -0500 Subject: [PATCH 04/26] Plan multi-connection behavior, sign-in claims and the RFC order Record how several connections in one organization behave and what v1 deliberately leaves open for enterprise IT. Following the community thread, add the Access-for-SaaS sign-in check, feed sign-in group claims into the same mappings, and lead the upstream RFC with per-organization sign-in. Co-Authored-By: Claude Opus 5.5 --- .../2026-11-16-scim-identity-providers.md | 79 +++++++++++++++++-- 1 file changed, 72 insertions(+), 7 deletions(-) diff --git a/docs/plans/2026-11-16-scim-identity-providers.md b/docs/plans/2026-11-16-scim-identity-providers.md index 69bcadd66a..77101310fa 100644 --- a/docs/plans/2026-11-16-scim-identity-providers.md +++ b/docs/plans/2026-11-16-scim-identity-providers.md @@ -220,10 +220,15 @@ Topics `user.provisioned`, `user.updated`, `user.deprovisioned` and ## Phases -1. **Backend foundation:** migration and models, domain claims and DNS - verification, connection and token management, the provisioning core, the - SCIM endpoint, events, audit, the permission domain, and tests (unit, e2e, - scim2-tester). +1. **Backend foundation (done):** migration and models, domain claims and + DNS verification, connection and token management, the provisioning core, + the SCIM endpoint, events, audit, the permission domain, and tests (unit, + e2e, scim2-tester). + - **Sign-in check (no code):** test Cloudflare Access for SaaS end to end + as Bifrost's generic OIDC provider, with Prefer SSO on. If it works, + that removes the double sign-in behind Access without trusting proxy + headers, and the result goes in the docs. Proxy header trust stays out + of scope: it isn't a standard, and every proxy does it differently. 2. **UI:** list, detail and edit dialog, the domains card, the token reveal, the mapping editor, and the Identity Source column; vitest plus one Playwright happy path. @@ -240,9 +245,19 @@ Topics `user.provisioned`, `user.updated`, `user.deprovisioned` and beta (this also covers secondary domains), and feeds the same core with the snapshot guards. This is the only way to get Google group membership, because Google's OIDC ID tokens carry no groups claim. -5. **Upstream RFC:** per-IdP sign-in consolidation, retiring or keeping the - legacy `Organization.domain` login auto-join in favour of verified claims, - and whether customer orgs may self-manage connections. +5. **Sign-in group claims:** once the platform emits `user.signed_in` with + the verified sign-in claims, groups in those claims feed the same core and + the same group mappings. A connection can then cover a tenant without + SCIM, and the mappings stay the one source of truth for group-to-role + rules. On its own this only acts when someone signs in. Revoking access + from someone who never signs in again still needs SCIM or a Directory sync. +6. **Upstream RFC**, per-organization sign-in first. Sign-in is + instance-wide today: one provider of each kind, one Prefer SSO default and + one callback URL. That blocks the common split of staff on one IdP and + client organizations on plain Microsoft SSO. The RFC also covers retiring + or keeping the legacy `Organization.domain` login auto-join in favour of + verified claims, and whether customer organizations may manage their own + connections. ## IdP notes @@ -250,6 +265,9 @@ Topics `user.provisioned`, `user.updated`, `user.deprovisioned` and - Provisioning cycles run about every 40 minutes; use "Provision on demand" for testing. - It sends the PATCH quirks above. + - It doesn't provision nested group membership: a user in a group nested + inside an assigned group isn't sent. Map the groups people are directly + in. - Validate with Microsoft's SCIM Validator. - **Okta:** validate with Okta's SCIM test suite. - **JumpCloud:** custom SCIM app. @@ -263,6 +281,53 @@ Topics `user.provisioned`, `user.updated`, `user.deprovisioned` and `201`. The test user must pass the domain claim. The never-signed-in `DELETE` rule in decision 5 keeps it from leaving disabled users behind. +## Multiple connections per organization + +An organization can have several connections. Each one has its own token, +groups and mappings, and revokes only the roles it granted. A user belongs to +exactly one connection: the first connection to provision or adopt them owns +them, and a second connection that sends the same person gets +`409 uniqueness`. Domain claims belong to the organization, so any connection +in the organization can provision any of its verified domains. + +This works for disjoint populations, such as an organization migrating +between tenants or a merger where each tenant covers different people. It does +not merge identities that appear in more than one directory. + +## Future direction (noted, not prioritized) + +Design goal: stay usable for large organizations whose internal IT looks like +an MSP, such as an enterprise managing many Microsoft 365 tenants or business +units. Nothing below is scheduled. Each item is recorded so v1 choices do not +close it off. + +1. **Scope a connection to domains.** Give each connection an optional + allowlist drawn from its organization's verified domains, so two IdPs in + one organization cannot compete for the same users. +2. **Hand a user off between connections.** Add an explicit "move to + connection B" action that transfers the link and the roles the connection + granted. Today a move depends on a `DELETE` from A followed by adoption in + B, and that path is not specified or tested. +3. **Test two connections in one organization.** Cover the `409` for an + overlapping user, and confirm that revocation by A leaves B's roles alone. +4. **One directory serving several organizations.** An enterprise may run one + Entra tenant while modelling business units as separate Bifrost + organizations. v1 binds a connection to one organization and lets only one + organization verify a domain, so this would need routing from one + connection to several organizations (by group or attribute) and a domain + model that allows it. Revisit together with per-IdP sign-in in the RFC. + +To keep these options open: + +- Keep connection ownership explicit on every row that provisioning writes + (links, groups, `user_roles.identity_provider_id`). Do not infer it from the + organization. +- Keep the provisioning core protocol-neutral and per connection, so routing + or merging can be layered over it later. +- Treat the unique `user_id` on `identity_provider_users` (one identity source + per user) as a deliberate v1 constraint, not a permanent one. Multi-source + identity would relax it. + ## Open questions (for the RFC) - Deleting a connection: the proposal is to unlink its users (they become From cb9f0b794c20802d4ca5a3e9f2dba0c07a532c2d Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Mon, 16 Nov 2026 14:18:15 -0500 Subject: [PATCH 05/26] Add the Identity Providers admin UI Users gains an Identity Providers view, modeled on NinjaOne's: - The list shows each connection's organization, provider, verified domains, provisioning status, token expiry, users and groups mapped. - The detail page has the SCIM card (Tenant URL to copy, token hint and expiry, regenerate with a one-time reveal), the organization's email domains with the TXT record to publish and a Verify action, and the group mapping table (group, mapping status, roles; a mapping with no roles grants access only). - An expired or expiring token, or no verified domain, raises a banner. - Editing uses a dialog with a left-hand section nav (General, Provisioning). - The Users list gains an Identity Source column: the managing connection, or Native. Covered by service and page vitest suites and a Playwright happy path that creates a connection, pushes a group over SCIM with the revealed token, and maps it to a role. Co-Authored-By: Claude Opus 5.5 --- client/e2e/identity-providers.admin.spec.ts | 111 ++++++ client/src/App.tsx | 30 ++ .../identity-providers/CopyField.tsx | 52 +++ .../identity-providers/DomainClaimsCard.tsx | 231 ++++++++++++ .../EditIdentityProviderDialog.tsx | 223 ++++++++++++ .../identity-providers/GroupMappingCard.tsx | 274 ++++++++++++++ .../IdentityProviderBadges.tsx | 53 +++ .../NewIdentityProviderDialog.tsx | 245 +++++++++++++ .../identity-providers/ProvisioningCard.tsx | 237 ++++++++++++ client/src/components/users/UsersViewTabs.tsx | 13 +- .../src/pages/IdentityProviderDetail.test.tsx | 244 +++++++++++++ client/src/pages/IdentityProviderDetail.tsx | 261 ++++++++++++++ client/src/pages/IdentityProviders.test.tsx | 230 ++++++++++++ client/src/pages/IdentityProviders.tsx | 337 ++++++++++++++++++ client/src/pages/Users.tsx | 17 +- .../src/services/identity-providers.test.ts | 144 ++++++++ client/src/services/identity-providers.ts | 221 ++++++++++++ 17 files changed, 2921 insertions(+), 2 deletions(-) create mode 100644 client/e2e/identity-providers.admin.spec.ts create mode 100644 client/src/components/identity-providers/CopyField.tsx create mode 100644 client/src/components/identity-providers/DomainClaimsCard.tsx create mode 100644 client/src/components/identity-providers/EditIdentityProviderDialog.tsx create mode 100644 client/src/components/identity-providers/GroupMappingCard.tsx create mode 100644 client/src/components/identity-providers/IdentityProviderBadges.tsx create mode 100644 client/src/components/identity-providers/NewIdentityProviderDialog.tsx create mode 100644 client/src/components/identity-providers/ProvisioningCard.tsx create mode 100644 client/src/pages/IdentityProviderDetail.test.tsx create mode 100644 client/src/pages/IdentityProviderDetail.tsx create mode 100644 client/src/pages/IdentityProviders.test.tsx create mode 100644 client/src/pages/IdentityProviders.tsx create mode 100644 client/src/services/identity-providers.test.ts create mode 100644 client/src/services/identity-providers.ts diff --git a/client/e2e/identity-providers.admin.spec.ts b/client/e2e/identity-providers.admin.spec.ts new file mode 100644 index 0000000000..26c641dfc1 --- /dev/null +++ b/client/e2e/identity-providers.admin.spec.ts @@ -0,0 +1,111 @@ +/** + * Identity Providers — admin happy path. + * + * A Platform Admin adds a SCIM connection for an organization and copies its + * one-time token and Tenant URL. The identity provider pushes a group with + * that token, and the admin maps the group to a role. + */ + +import { randomUUID } from "node:crypto"; + +import { test, expect } from "./fixtures/api-fixture"; + +const SUFFIX = randomUUID().slice(0, 8); +const ORG_NAME = `Contoso ${SUFFIX}`; +const PROVIDER_NAME = `Contoso Entra ${SUFFIX}`; +const ROLE_NAME = `L1 Techs ${SUFFIX}`; +const GROUP_NAME = `Ops-L1 ${SUFFIX}`; + +test.describe("Identity Providers", () => { + let organizationId: string; + let roleId: string; + + test.beforeAll(async ({ api }) => { + const org = await api.post("/api/organizations", { + data: { name: ORG_NAME }, + }); + expect(org.ok(), `create organization: ${await org.text()}`).toBe(true); + organizationId = ((await org.json()) as { id: string }).id; + const role = await api.post("/api/roles", { data: { name: ROLE_NAME } }); + expect(role.ok(), `create role: ${await role.text()}`).toBe(true); + roleId = ((await role.json()) as { id: string }).id; + }); + + test.afterAll(async ({ api }) => { + if (organizationId) { + const listed = await api.get("/api/identity-providers", { + params: { organization_id: organizationId }, + }); + for (const provider of (await listed.json()) as { id: string }[]) { + await api.delete(`/api/identity-providers/${provider.id}`); + } + await api.delete(`/api/organizations/${organizationId}`); + } + if (roleId) await api.delete(`/api/roles/${roleId}`); + }); + + test("IDP-01 adds a connection, receives a group and maps it to a role", async ({ + page, + api, + }) => { + await page.goto("/users"); + await page + .getByRole("navigation", { name: "User views" }) + .getByRole("link", { name: "Identity Providers" }) + .click(); + await expect(page).toHaveURL(/\/users\/identity-providers$/); + + await page.getByRole("button", { name: "Add Provider" }).click(); + const dialog = page.getByRole("dialog", { name: "New Identity Provider" }); + await dialog + .getByRole("textbox", { name: "Display Name" }) + .fill(PROVIDER_NAME); + await dialog.getByRole("combobox", { name: "Organization" }).click(); + await page.getByPlaceholder("Search organizations...").fill(ORG_NAME); + await page.getByRole("option", { name: ORG_NAME }).click(); + await dialog.getByRole("button", { name: "Create Connection" }).click(); + + await expect( + page.getByRole("heading", { level: 1, name: PROVIDER_NAME }), + ).toBeVisible(); + const token = ( + await page.getByLabel("Secret value").textContent() + )?.trim(); + expect(token).toMatch(/^bfidp_/); + const tenantUrl = await page + .getByRole("textbox", { name: "Tenant URL" }) + .inputValue(); + await page.getByRole("button", { name: "Dismiss secret" }).click(); + await expect(page.getByLabel("Secret value")).toHaveCount(0); + + // The identity provider's first provisioning cycle pushes a group. + const pushed = await api.post(`${new URL(tenantUrl).pathname}/Groups`, { + headers: { + Authorization: `Bearer ${token}`, + "Content-Type": "application/scim+json", + }, + data: { + schemas: ["urn:ietf:params:scim:schemas:core:2.0:Group"], + displayName: GROUP_NAME, + externalId: `entra-${SUFFIX}`, + }, + }); + expect(pushed.status(), await pushed.text()).toBe(201); + + await page.reload(); + await page + .getByRole("button", { name: `Edit mapping for ${GROUP_NAME}` }) + .click(); + const mapping = page.getByRole("dialog", { name: `Map ${GROUP_NAME}` }); + await mapping.getByRole("combobox", { name: "Access only" }).click(); + await page.getByRole("option", { name: ROLE_NAME }).click(); + // Close the role list without dismissing the dialog. + await mapping.getByRole("heading", { name: `Map ${GROUP_NAME}` }).click(); + await mapping.getByRole("button", { name: "Save Mapping" }).click(); + await expect(mapping).toHaveCount(0); + + const row = page.getByRole("row", { name: new RegExp(GROUP_NAME) }); + await expect(row.getByText("Mapped", { exact: true })).toBeVisible(); + await expect(row.getByText(ROLE_NAME)).toBeVisible(); + }); +}); diff --git a/client/src/App.tsx b/client/src/App.tsx index 6e2c1480a2..65e60a26f5 100644 --- a/client/src/App.tsx +++ b/client/src/App.tsx @@ -67,6 +67,16 @@ const Users = lazyWithReload(() => const Identities = lazyWithReload(() => import("@/pages/Identities").then((m) => ({ default: m.Identities })), ); +const IdentityProviders = lazyWithReload(() => + import("@/pages/IdentityProviders").then((m) => ({ + default: m.IdentityProviders, + })), +); +const IdentityProviderDetail = lazyWithReload(() => + import("@/pages/IdentityProviderDetail").then((m) => ({ + default: m.IdentityProviderDetail, + })), +); const UserAccessPage = lazyWithReload(() => import("@/pages/UserAccessPage").then((m) => ({ default: m.UserAccessPage, @@ -590,6 +600,26 @@ const routeElements = ( } /> + + + + } + /> + + + + } + /> { + if (await copyToClipboard(value)) { + setCopied(true); + window.setTimeout(() => setCopied(false), 2000); + } + }; + return ( +
+ event.currentTarget.select()} + /> + +
+ ); +} diff --git a/client/src/components/identity-providers/DomainClaimsCard.tsx b/client/src/components/identity-providers/DomainClaimsCard.tsx new file mode 100644 index 0000000000..61a1b9bbe6 --- /dev/null +++ b/client/src/components/identity-providers/DomainClaimsCard.tsx @@ -0,0 +1,231 @@ +import { useState } from "react"; +import { CheckCircle2, Clock, Loader2, Plus, Trash2 } from "lucide-react"; +import { toast } from "sonner"; + +import { CopyField } from "@/components/identity-providers/CopyField"; +import { Badge } from "@/components/ui/badge"; +import { Button } from "@/components/ui/button"; +import { + Card, + CardContent, + CardDescription, + CardHeader, + CardTitle, +} from "@/components/ui/card"; +import { Input } from "@/components/ui/input"; +import { Label } from "@/components/ui/label"; +import { Skeleton } from "@/components/ui/skeleton"; +import { getErrorMessage } from "@/lib/api-error"; +import { + useAddOrganizationDomain, + useDeleteOrganizationDomain, + useOrganizationDomains, + useVerifyOrganizationDomain, + type OrganizationDomain, +} from "@/services/identity-providers"; + +function ClaimRow({ + claim, + canWrite, +}: { + claim: OrganizationDomain; + canWrite: boolean; +}) { + const verify = useVerifyOrganizationDomain(); + const remove = useDeleteOrganizationDomain(); + const verified = claim.verified_at !== null; + const onVerify = async () => { + try { + const result = await verify.mutateAsync({ + params: { path: { claim_id: claim.id } }, + }); + if (result.verified_at) { + toast.success(`${claim.domain} verified`); + } else { + toast.error( + `The TXT record for ${claim.domain} wasn't found yet`, + { + description: + "DNS changes can take a while to propagate. Try again shortly.", + }, + ); + } + } catch (cause) { + toast.error( + getErrorMessage(cause, "The domain could not be verified."), + ); + } + }; + const onRemove = async () => { + try { + await remove.mutateAsync({ + params: { path: { claim_id: claim.id } }, + }); + } catch (cause) { + toast.error( + getErrorMessage(cause, "The domain could not be removed."), + ); + } + }; + return ( +
  • +
    + + @{claim.domain} + + {verified ? ( + + + ) : ( + + + )} + {canWrite && ( +
    + {!verified && ( + + )} + +
    + )} +
    + {!verified && ( +
    +

    + Add a TXT record on{" "} + {claim.txt_record_name} with this + value, then select Verify. +

    + +
    + )} +
  • + ); +} + +/** + * The organization's email domains. A connection only provisions users on + * verified domains, and each domain is verified by one organization. + */ +export function DomainClaimsCard({ + organizationId, + organizationName, + canWrite, +}: { + organizationId: string; + organizationName: string; + canWrite: boolean; +}) { + const query = useOrganizationDomains(organizationId); + const add = useAddOrganizationDomain(); + const [domain, setDomain] = useState(""); + const onAdd = async (event: React.FormEvent) => { + event.preventDefault(); + if (!domain.trim()) return; + try { + await add.mutateAsync({ + body: { + organization_id: organizationId, + domain: domain.trim(), + }, + }); + setDomain(""); + } catch (cause) { + toast.error( + getErrorMessage(cause, "The domain could not be added."), + ); + } + }; + const claims = query.data ?? []; + return ( + + + Email Domains + + {organizationName}'s domains. Users are only provisioned on + verified domains; they apply to every connection in the + organization. + + + + {query.isLoading ? ( + + ) : claims.length === 0 ? ( +

    + No domains yet. Add one and verify it before the + identity provider sends users. +

    + ) : ( +
      + {claims.map((claim) => ( + + ))} +
    + )} + {canWrite && ( +
    +
    + + + setDomain(event.target.value) + } + placeholder="contoso.com" + maxLength={253} + /> +
    + +
    + )} +
    +
    + ); +} diff --git a/client/src/components/identity-providers/EditIdentityProviderDialog.tsx b/client/src/components/identity-providers/EditIdentityProviderDialog.tsx new file mode 100644 index 0000000000..b52d86c52f --- /dev/null +++ b/client/src/components/identity-providers/EditIdentityProviderDialog.tsx @@ -0,0 +1,223 @@ +import { useState } from "react"; +import { Loader2 } from "lucide-react"; +import { toast } from "sonner"; + +import { Button } from "@/components/ui/button"; +import { + Dialog, + DialogContent, + DialogDescription, + DialogFooter, + DialogHeader, + DialogTitle, +} from "@/components/ui/dialog"; +import { Input } from "@/components/ui/input"; +import { Label } from "@/components/ui/label"; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from "@/components/ui/select"; +import { Switch } from "@/components/ui/switch"; +import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"; +import { getErrorMessage } from "@/lib/api-error"; +import { + VENDORS, + useUpdateIdentityProvider, + vendorLabel, + type IdentityProvider, + type IdentityProviderVendor, +} from "@/services/identity-providers"; + +function SwitchRow({ + id, + label, + description, + checked, + onCheckedChange, +}: { + id: string; + label: string; + description: string; + checked: boolean; + onCheckedChange: (checked: boolean) => void; +}) { + return ( +
    + +
    + +

    {description}

    +
    +
    + ); +} + +function EditContent({ + provider, + onOpenChange, +}: { + provider: IdentityProvider; + onOpenChange: (open: boolean) => void; +}) { + const update = useUpdateIdentityProvider(); + const [name, setName] = useState(provider.name); + const [vendor, setVendor] = useState( + provider.vendor, + ); + const [enabled, setEnabled] = useState(provider.is_enabled); + const [external, setExternal] = useState(provider.provision_as_external); + const busy = update.isPending; + const save = async () => { + try { + await update.mutateAsync({ + params: { path: { identity_provider_id: provider.id } }, + body: { + name: name.trim(), + vendor, + is_enabled: enabled, + provision_as_external: external, + }, + }); + toast.success("Identity provider saved"); + onOpenChange(false); + } catch (cause) { + toast.error( + getErrorMessage(cause, "The connection could not be saved."), + ); + } + }; + return ( + + + Configure Identity Provider + + Changes apply to the next request the identity provider + sends. + + +
    + + + + General + + + Provisioning + + + +
    + + + setName(event.target.value) + } + maxLength={255} + /> +
    +
    + + +
    +
    + + + + +
    +
    + + + + +
    + ); +} + +export function EditIdentityProviderDialog({ + provider, + open, + onOpenChange, +}: { + provider: IdentityProvider; + open: boolean; + onOpenChange: (open: boolean) => void; +}) { + return ( + + {open && ( + + )} + + ); +} diff --git a/client/src/components/identity-providers/GroupMappingCard.tsx b/client/src/components/identity-providers/GroupMappingCard.tsx new file mode 100644 index 0000000000..a1df1c465e --- /dev/null +++ b/client/src/components/identity-providers/GroupMappingCard.tsx @@ -0,0 +1,274 @@ +import { useState } from "react"; +import { ArrowRight, CheckCircle2, CircleDashed, Loader2 } from "lucide-react"; +import { toast } from "sonner"; + +import { Badge } from "@/components/ui/badge"; +import { Button } from "@/components/ui/button"; +import { + Card, + CardContent, + CardDescription, + CardHeader, + CardTitle, +} from "@/components/ui/card"; +import { + DataTable, + DataTableBody, + DataTableCell, + DataTableHead, + DataTableHeader, + DataTableRow, +} from "@/components/ui/data-table"; +import { + Dialog, + DialogContent, + DialogDescription, + DialogFooter, + DialogHeader, + DialogTitle, +} from "@/components/ui/dialog"; +import { Label } from "@/components/ui/label"; +import { MultiCombobox } from "@/components/ui/multi-combobox"; +import { Skeleton } from "@/components/ui/skeleton"; +import { useRoles } from "@/hooks/useRoles"; +import { getErrorMessage } from "@/lib/api-error"; +import { + useIdentityProviderGroups, + useMapIdentityProviderGroup, + useUnmapIdentityProviderGroup, + type IdentityProvider, + type IdentityProviderGroup, +} from "@/services/identity-providers"; + +function MappingDialog({ + provider, + group, + onOpenChange, +}: { + provider: IdentityProvider; + group: IdentityProviderGroup; + onOpenChange: (open: boolean) => void; +}) { + const roles = useRoles(); + const map = useMapIdentityProviderGroup(); + const unmap = useUnmapIdentityProviderGroup(); + const [roleIds, setRoleIds] = useState(group.roles.map((role) => role.id)); + const busy = map.isPending || unmap.isPending; + const path = { identity_provider_id: provider.id, group_id: group.id }; + const save = async () => { + try { + await map.mutateAsync({ + params: { path }, + body: { role_ids: roleIds }, + }); + onOpenChange(false); + } catch (cause) { + toast.error( + getErrorMessage(cause, "The mapping could not be saved."), + ); + } + }; + const remove = async () => { + try { + await unmap.mutateAsync({ params: { path } }); + onOpenChange(false); + } catch (cause) { + toast.error( + getErrorMessage(cause, "The mapping could not be removed."), + ); + } + }; + return ( + + + + Map {group.display_name} + + Members get these roles in {provider.organization_name}. + A mapping with no roles grants access only. Once any + group is mapped, users need a mapped group to stay + active. + + +
    + + ({ + value: role.id, + label: role.name, + }))} + value={roleIds} + onValueChange={setRoleIds} + isLoading={roles.isLoading} + placeholder="Access only" + searchPlaceholder="Search roles..." + emptyText="No roles found." + /> +
    + + {group.mapped ? ( + + ) : ( + + )} +
    + + +
    +
    +
    +
    + ); +} + +/** The groups the provider has pushed, and the roles each one grants. */ +export function GroupMappingCard({ + provider, + canWrite, +}: { + provider: IdentityProvider; + canWrite: boolean; +}) { + const query = useIdentityProviderGroups(provider.id); + const [editing, setEditing] = useState(null); + const groups = query.data ?? []; + return ( + + + Group Mapping + + {groups.length} {groups.length === 1 ? "group" : "groups"}{" "} + pushed by the identity provider. Assign groups to the app in + the identity provider to see them here. + + + + {query.isLoading ? ( + + ) : groups.length === 0 ? ( +

    + No groups yet. They appear after the identity provider's + next provisioning cycle. +

    + ) : ( + + + + Identity Group + + + + {groups.map((group) => ( + + + + {group.display_name} + + + + + + {group.mapped ? ( + + + ) : ( + + + )} + + + {group.mapped && + group.roles.length === 0 ? ( + + Access Only + + ) : ( +
      + {group.roles.map((role) => ( +
    • + + {role.name} + +
    • + ))} +
    + )} +
    + + + {group.member_count} + + + {canWrite && ( + + + + )} +
    + ))} +
    +
    + )} +
    + {editing && ( + !open && setEditing(null)} + /> + )} +
    + ); +} diff --git a/client/src/components/identity-providers/IdentityProviderBadges.tsx b/client/src/components/identity-providers/IdentityProviderBadges.tsx new file mode 100644 index 0000000000..c111ca0dde --- /dev/null +++ b/client/src/components/identity-providers/IdentityProviderBadges.tsx @@ -0,0 +1,53 @@ +import { CheckCircle2, CircleSlash, Clock, XCircle } from "lucide-react"; + +import { Badge } from "@/components/ui/badge"; +import { formatDateShort } from "@/lib/utils"; +import { + credentialStatus, + type IdentityProvider, +} from "@/services/identity-providers"; + +/** Whether the connection accepts provisioning requests. */ +export function ProvisioningStatusBadge({ enabled }: { enabled: boolean }) { + return enabled ? ( + + + ) : ( + + + ); +} + +/** The token's expiry, flagged once it is close or past. */ +export function CredentialExpiryBadge({ + token, +}: { + token: IdentityProvider["token"]; +}) { + const status = credentialStatus(token); + if (!token || status === "none") { + return No Token; + } + const date = formatDateShort(token.expires_at); + if (status === "expired") { + return ( + + + ); + } + if (status === "expiring") { + return ( + + + ); + } + return {date}; +} diff --git a/client/src/components/identity-providers/NewIdentityProviderDialog.tsx b/client/src/components/identity-providers/NewIdentityProviderDialog.tsx new file mode 100644 index 0000000000..8f139e4523 --- /dev/null +++ b/client/src/components/identity-providers/NewIdentityProviderDialog.tsx @@ -0,0 +1,245 @@ +import { useEffect, useRef, useState } from "react"; +import { AlertCircle, Building2, Loader2 } from "lucide-react"; + +import { Alert, AlertDescription } from "@/components/ui/alert"; +import { Button } from "@/components/ui/button"; +import { Combobox, type ComboboxOption } from "@/components/ui/combobox"; +import { + Dialog, + DialogContent, + DialogDescription, + DialogFooter, + DialogHeader, + DialogTitle, +} from "@/components/ui/dialog"; +import { Input } from "@/components/ui/input"; +import { Label } from "@/components/ui/label"; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from "@/components/ui/select"; +import { Switch } from "@/components/ui/switch"; +import { useOrganizations } from "@/hooks/useOrganizations"; +import { getErrorMessage } from "@/lib/api-error"; +import { orgTarget } from "@/lib/authorization"; +import type { components } from "@/lib/v1"; +import { useAuthorization } from "@/services/authorization"; +import { + TOKEN_LIFETIMES, + VENDORS, + useCreateIdentityProvider, + vendorLabel, + type IdentityProviderVendor, + type TokenLifetimeDays, +} from "@/services/identity-providers"; + +export type IdentityProviderCreated = + components["schemas"]["IdentityProviderCreated"]; + +const WRITE = "identityproviders.readwrite"; + +function NewIdentityProviderDialogContent({ + onOpenChange, + onCreated, +}: { + onOpenChange: (open: boolean) => void; + onCreated: (created: IdentityProviderCreated) => void; +}) { + const authorization = useAuthorization(); + const organizationsQuery = useOrganizations(); + const create = useCreateIdentityProvider(); + const [name, setName] = useState(""); + const [organizationId, setOrganizationId] = useState(""); + const [vendor, setVendor] = useState("entra"); + const [lifetime, setLifetime] = useState(365); + const [external, setExternal] = useState(false); + const [error, setError] = useState(null); + const errorRef = useRef(null); + useEffect(() => { + if (error) errorRef.current?.focus(); + }, [error]); + + const choices: ComboboxOption[] = (organizationsQuery.data ?? []) + .filter((org) => authorization.canAt(WRITE, orgTarget(org.id))) + .sort((a, b) => a.name.localeCompare(b.name)) + .map((org) => ({ value: org.id, label: org.name, icon: Building2 })); + const busy = create.isPending; + const ready = name.trim().length > 0 && organizationId !== ""; + + const handleSubmit = async (event: React.FormEvent) => { + event.preventDefault(); + if (!ready || busy) return; + setError(null); + try { + const created = await create.mutateAsync({ + body: { + name: name.trim(), + organization_id: organizationId, + vendor, + source_type: "scim", + token_lifetime_days: lifetime, + provision_as_external: external, + }, + }); + onOpenChange(false); + onCreated(created); + } catch (cause) { + setError( + getErrorMessage(cause, "The connection could not be created."), + ); + } + }; + + return ( + busy && event.preventDefault()} + onInteractOutside={(event) => busy && event.preventDefault()} + className="sm:max-w-lg" + > + + New Identity Provider + + Connect an organization's directory over SCIM. Bifrost + creates the connection and its first token; you paste both + into the identity provider. + + +
    + {error && ( + + + {error} + + )} +
    + + setName(event.target.value)} + placeholder="Contoso Entra ID" + maxLength={255} + required + /> +
    +
    + + +

    + Users are provisioned into this organization, on its + verified email domains only. +

    +
    +
    +
    + + +
    +
    + + +
    +
    +
    + +
    + +

    + External users see only what a role or an explicit + grant gives them, not what's shared with every + signed-in user. Use this for portal or guest + accounts. +

    +
    +
    + + + + +
    +
    + ); +} + +export function NewIdentityProviderDialog({ + open, + onOpenChange, + onCreated, +}: { + open: boolean; + onOpenChange: (open: boolean) => void; + onCreated: (created: IdentityProviderCreated) => void; +}) { + return ( + + {open && ( + + )} + + ); +} diff --git a/client/src/components/identity-providers/ProvisioningCard.tsx b/client/src/components/identity-providers/ProvisioningCard.tsx new file mode 100644 index 0000000000..d2481477a2 --- /dev/null +++ b/client/src/components/identity-providers/ProvisioningCard.tsx @@ -0,0 +1,237 @@ +import { useState } from "react"; +import { Loader2, RotateCw } from "lucide-react"; +import { toast } from "sonner"; + +import { HmacSecretReveal } from "@/components/forms/HmacSecretReveal"; +import { CopyField } from "@/components/identity-providers/CopyField"; +import { + CredentialExpiryBadge, + ProvisioningStatusBadge, +} from "@/components/identity-providers/IdentityProviderBadges"; +import { Button } from "@/components/ui/button"; +import { + Card, + CardContent, + CardDescription, + CardHeader, + CardTitle, +} from "@/components/ui/card"; +import { + Dialog, + DialogContent, + DialogDescription, + DialogFooter, + DialogHeader, + DialogTitle, +} from "@/components/ui/dialog"; +import { Label } from "@/components/ui/label"; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from "@/components/ui/select"; +import { getErrorMessage } from "@/lib/api-error"; +import { formatDate } from "@/lib/utils"; +import { + TOKEN_LIFETIMES, + useRotateIdentityProviderToken, + type IdentityProvider, + type IdentityProviderTokenIssued, + type TokenLifetimeDays, +} from "@/services/identity-providers"; + +function Row({ + label, + children, +}: { + label: string; + children: React.ReactNode; +}) { + return ( +
    +
    {label}
    +
    {children}
    +
    + ); +} + +function RotateTokenDialog({ + provider, + open, + onOpenChange, + onIssued, +}: { + provider: IdentityProvider; + open: boolean; + onOpenChange: (open: boolean) => void; + onIssued: (issued: IdentityProviderTokenIssued) => void; +}) { + const rotate = useRotateIdentityProviderToken(); + const [lifetime, setLifetime] = useState(365); + const submit = async () => { + try { + const issued = await rotate.mutateAsync({ + params: { path: { identity_provider_id: provider.id } }, + body: { lifetime_days: lifetime }, + }); + onOpenChange(false); + onIssued(issued); + } catch (cause) { + toast.error( + getErrorMessage(cause, "The token could not be regenerated."), + ); + } + }; + return ( + + + + Regenerate Token + + The current token keeps working for one hour, so you + have time to paste the new one into the identity + provider and test the connection. + + +
    + + +
    + + + + +
    +
    + ); +} + +/** + * What to paste into the identity provider (Tenant URL and token) and how + * the connection is doing. A newly issued token is shown here once. + */ +export function ProvisioningCard({ + provider, + canWrite, + issuedToken, + onDismissToken, + onIssued, +}: { + provider: IdentityProvider; + canWrite: boolean; + issuedToken: IdentityProviderTokenIssued | null; + onDismissToken: () => void; + onIssued: (issued: IdentityProviderTokenIssued) => void; +}) { + const [rotateOpen, setRotateOpen] = useState(false); + return ( + + +
    + + System for Cross-domain Identity Management (SCIM) + + + Paste the Tenant URL and token into the identity + provider's provisioning settings. + +
    + {canWrite && ( + + )} +
    + + {issuedToken && ( + + )} +
    + + + + + + + + {provider.token ? ( + + ••••{provider.token.hint} + + ) : ( + "None" + )} + + + + + + {provider.last_request_at + ? formatDate(provider.last_request_at) + : "Never"} + + + + {provider.user_count} + + +
    +
    + +
    + ); +} diff --git a/client/src/components/users/UsersViewTabs.tsx b/client/src/components/users/UsersViewTabs.tsx index 557e8fe723..f68e61f989 100644 --- a/client/src/components/users/UsersViewTabs.tsx +++ b/client/src/components/users/UsersViewTabs.tsx @@ -1,13 +1,24 @@ import { WorkspaceTabs } from "@/components/layout/WorkspaceTabs"; +import { useAuthorization } from "@/services/authorization"; +import { IDENTITY_PROVIDERS_PATH } from "@/services/identity-providers"; -/** Users | Identities, the title of both lists. */ +/** Users | Identities | Identity Providers, the title of each list. */ export function UsersViewTabs() { + const authorization = useAuthorization(); return ( ); diff --git a/client/src/pages/IdentityProviderDetail.test.tsx b/client/src/pages/IdentityProviderDetail.test.tsx new file mode 100644 index 0000000000..b7d5c78d22 --- /dev/null +++ b/client/src/pages/IdentityProviderDetail.test.tsx @@ -0,0 +1,244 @@ +import { beforeEach, describe, expect, it, vi } from "vitest"; +import { Route, Routes } from "react-router-dom"; + +import { renderWithProviders, screen } from "@/test-utils"; +import { + canAnywhere, + canAt, + meetsRequirement, + type AuthorizationSummary, + type AuthorizationTarget, + type PermissionRequirement, +} from "@/lib/authorization"; +import type { + IdentityProvider, + IdentityProviderGroup, + OrganizationDomain, +} from "@/services/identity-providers"; + +const authz = vi.hoisted(() => ({ + summary: undefined as AuthorizationSummary | undefined, +})); +vi.mock("@/services/authorization", () => ({ + useAuthorization: () => ({ + authorization: authz.summary, + canAt: (permission: string, target: AuthorizationTarget) => + canAt(authz.summary, permission, target), + canAnywhere: (permission: string) => + canAnywhere(authz.summary, permission), + meets: (requirement: PermissionRequirement) => + meetsRequirement(authz.summary, requirement), + }), +})); + +const data = vi.hoisted(() => ({ + provider: undefined as IdentityProvider | undefined, + groups: [] as IdentityProviderGroup[], + domains: [] as OrganizationDomain[], +})); +const mutation = vi.hoisted(() => () => ({ + mutateAsync: vi.fn(), + isPending: false, +})); +vi.mock("@/services/identity-providers", async (importOriginal) => ({ + ...(await importOriginal()), + useIdentityProvider: () => ({ data: data.provider, isLoading: false }), + useIdentityProviderGroups: () => ({ data: data.groups, isLoading: false }), + useOrganizationDomains: () => ({ data: data.domains, isLoading: false }), + useDeleteIdentityProvider: mutation, + useUpdateIdentityProvider: mutation, + useRotateIdentityProviderToken: mutation, + useMapIdentityProviderGroup: mutation, + useUnmapIdentityProviderGroup: mutation, + useAddOrganizationDomain: mutation, + useVerifyOrganizationDomain: mutation, + useDeleteOrganizationDomain: mutation, +})); + +import { IdentityProviderDetail } from "./IdentityProviderDetail"; + +const DAY = 24 * 60 * 60 * 1000; + +function summary(admin: boolean): AuthorizationSummary { + return { + is_platform_admin: admin, + home_organization_id: "org-provider", + provider_organization_id: "org-provider", + base_role: { id: "role", name: admin ? "Platform Admin" : "User" }, + grants: admin + ? [] + : [ + { + permission: "identityproviders.read", + boundary: { + kind: "organization", + organization_id: "org-1", + }, + }, + ], + }; +} + +function provider(overrides: Partial = {}): IdentityProvider { + return { + id: "idp-1", + organization_id: "org-1", + organization_name: "Contoso", + name: "MTG-NinjaSSO", + source_type: "scim", + vendor: "entra", + is_enabled: true, + provision_as_external: false, + scim_base_url: "https://bifrost.example/api/scim/v2/idp-1", + token: { + hint: "abcd", + expires_at: new Date(Date.now() + 200 * DAY).toISOString(), + created_at: new Date().toISOString(), + last_used_at: null, + }, + verified_domains: ["contoso.com"], + user_count: 30, + group_count: 2, + mapped_group_count: 2, + last_request_at: null, + last_sync_at: null, + created_at: new Date().toISOString(), + updated_at: new Date().toISOString(), + ...overrides, + }; +} + +function renderPage(state?: unknown) { + return renderWithProviders( + + } + /> + , + { + initialEntries: [ + { pathname: "/users/identity-providers/idp-1", state }, + ] as never, + }, + ); +} + +describe("IdentityProviderDetail", () => { + beforeEach(() => { + authz.summary = summary(true); + data.provider = provider(); + data.groups = []; + data.domains = []; + }); + + it("shows the Tenant URL to paste into the identity provider", () => { + renderPage(); + + expect( + screen.getByRole("heading", { name: "MTG-NinjaSSO" }), + ).toBeInTheDocument(); + expect( + screen.getByDisplayValue(data.provider!.scim_base_url), + ).toBeInTheDocument(); + expect(screen.getByText("••••abcd")).toBeInTheDocument(); + }); + + it("warns when the token has expired", () => { + data.provider = provider({ + token: { + hint: "abcd", + expires_at: new Date(Date.now() - DAY).toISOString(), + created_at: new Date().toISOString(), + last_used_at: null, + }, + }); + renderPage(); + + expect( + screen.getByText("SCIM authentication token has expired"), + ).toBeInTheDocument(); + }); + + it("reveals a newly issued token once and hides it when dismissed", async () => { + const { user } = renderPage({ + issuedToken: { + token: "bfidp_one_time", + info: data.provider!.token, + previous_tokens_expire_at: null, + }, + }); + + expect(screen.getByText("bfidp_one_time")).toBeInTheDocument(); + await user.click( + screen.getByRole("button", { name: "Dismiss secret" }), + ); + expect(screen.queryByText("bfidp_one_time")).toBeNull(); + }); + + it("lists groups with their mapping status and roles", () => { + data.groups = [ + { + id: "g1", + display_name: "NinjaUsers", + external_id: null, + member_count: 12, + mapped: true, + roles: [{ id: "r1", name: "Support Tech" }], + }, + { + id: "g2", + display_name: "Everyone", + external_id: null, + member_count: 30, + mapped: true, + roles: [], + }, + { + id: "g3", + display_name: "Contractors", + external_id: null, + member_count: 3, + mapped: false, + roles: [], + }, + ]; + renderPage(); + + expect(screen.getByText("Support Tech")).toBeInTheDocument(); + expect(screen.getByText("Access Only")).toBeInTheDocument(); + expect(screen.getByText("Not Mapped")).toBeInTheDocument(); + }); + + it("gives the TXT record to publish for a pending domain", () => { + data.domains = [ + { + id: "d1", + organization_id: "org-1", + domain: "contoso.com", + verified_at: null, + last_checked_at: null, + txt_record_name: "contoso.com", + txt_record_value: "bifrost-verification=abc123", + created_at: new Date().toISOString(), + }, + ]; + renderPage(); + + expect(screen.getByText("Pending")).toBeInTheDocument(); + expect( + screen.getByDisplayValue("bifrost-verification=abc123"), + ).toBeInTheDocument(); + }); + + it("hides changes from a caller who can only read connections", () => { + authz.summary = summary(false); + renderPage(); + + expect(screen.queryByRole("button", { name: /Edit/ })).toBeNull(); + expect(screen.queryByRole("button", { name: /Delete/ })).toBeNull(); + expect( + screen.queryByRole("button", { name: /Regenerate Token/ }), + ).toBeNull(); + }); +}); diff --git a/client/src/pages/IdentityProviderDetail.tsx b/client/src/pages/IdentityProviderDetail.tsx new file mode 100644 index 0000000000..16036afb91 --- /dev/null +++ b/client/src/pages/IdentityProviderDetail.tsx @@ -0,0 +1,261 @@ +import { useState } from "react"; +import { Link, useLocation, useNavigate, useParams } from "react-router-dom"; +import { AlertCircle, ArrowLeft, Pencil, Trash2 } from "lucide-react"; +import { toast } from "sonner"; + +import { DomainClaimsCard } from "@/components/identity-providers/DomainClaimsCard"; +import { EditIdentityProviderDialog } from "@/components/identity-providers/EditIdentityProviderDialog"; +import { GroupMappingCard } from "@/components/identity-providers/GroupMappingCard"; +import { ProvisioningCard } from "@/components/identity-providers/ProvisioningCard"; +import { PageWorkspace } from "@/components/layout/PageWorkspace"; +import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert"; +import { + AlertDialog, + AlertDialogAction, + AlertDialogCancel, + AlertDialogContent, + AlertDialogDescription, + AlertDialogFooter, + AlertDialogHeader, + AlertDialogTitle, +} from "@/components/ui/alert-dialog"; +import { Button } from "@/components/ui/button"; +import { Skeleton } from "@/components/ui/skeleton"; +import { getErrorMessage } from "@/lib/api-error"; +import { orgTarget } from "@/lib/authorization"; +import { formatDateShort } from "@/lib/utils"; +import { useAuthorization } from "@/services/authorization"; +import { + IDENTITY_PROVIDERS_PATH, + credentialStatus, + useDeleteIdentityProvider, + useIdentityProvider, + vendorLabel, + type IdentityProvider, + type IdentityProviderTokenIssued, +} from "@/services/identity-providers"; + +/** What needs attention before provisioning can work, most urgent first. */ +function Attention({ provider }: { provider: IdentityProvider }) { + const status = credentialStatus(provider.token); + if (status === "expired" || status === "none") { + return ( + + + + {status === "none" + ? "This connection has no token" + : "SCIM authentication token has expired"} + + + Regenerate the token, copy it, then update and test the + connection in the identity provider. + + + ); + } + if (status === "expiring" && provider.token) { + return ( + + + + SCIM token expires{" "} + {formatDateShort(provider.token.expires_at)} + + + Regenerate it and update the identity provider before then. + + + ); + } + if (provider.verified_domains.length === 0) { + return ( + + + No verified email domains + + The identity provider can't create users until a domain is + verified below. + + + ); + } + return null; +} + +export function IdentityProviderDetail() { + const { identityProviderId = "" } = useParams(); + const navigate = useNavigate(); + const location = useLocation(); + const authorization = useAuthorization(); + const query = useIdentityProvider(identityProviderId); + const remove = useDeleteIdentityProvider(); + const [editOpen, setEditOpen] = useState(false); + const [deleteOpen, setDeleteOpen] = useState(false); + const [issuedToken, setIssuedToken] = + useState( + ( + location.state as { + issuedToken?: IdentityProviderTokenIssued; + } | null + )?.issuedToken ?? null, + ); + const dismissToken = () => { + setIssuedToken(null); + // Don't show the token again after a reload or back navigation. + navigate(location.pathname, { replace: true, state: null }); + }; + + const provider = query.data; + if (query.isLoading) { + return ( +
    + + +
    + ); + } + if (!provider) { + return ( + + + {query.error + ? getErrorMessage( + query.error, + "The identity provider could not be loaded.", + ) + : "Identity provider not found."} + + + + ); + } + + const canWrite = authorization.canAt( + "identityproviders.readwrite", + orgTarget(provider.organization_id), + ); + const onDelete = async () => { + try { + await remove.mutateAsync({ + params: { path: { identity_provider_id: provider.id } }, + }); + toast.success(`${provider.name} deleted`); + navigate(IDENTITY_PROVIDERS_PATH); + } catch (cause) { + toast.error( + getErrorMessage(cause, "The connection could not be deleted."), + ); + } + }; + + return ( + +
    + + + Identity Providers + +
    +
    +

    + {provider.name} +

    +

    + {vendorLabel(provider.vendor)} ·{" "} + {provider.organization_name} +

    +
    + {canWrite && ( +
    + + +
    + )} +
    +
    + + + + + + + + + + + + Delete {provider.name}? + + + Its {provider.user_count} users keep their accounts + and roles but become Native, and the identity + provider can no longer provision them. Remove the + app from the identity provider too. + + + + + Cancel + + { + event.preventDefault(); + void onDelete(); + }} + > + Delete Connection + + + + +
    + ); +} diff --git a/client/src/pages/IdentityProviders.test.tsx b/client/src/pages/IdentityProviders.test.tsx new file mode 100644 index 0000000000..3b1cd4d4f3 --- /dev/null +++ b/client/src/pages/IdentityProviders.test.tsx @@ -0,0 +1,230 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { Route, Routes, useLocation } from "react-router-dom"; + +import { renderWithProviders, screen, waitFor, within } from "@/test-utils"; +import { + canAnywhere, + canAt, + meetsRequirement, + type AuthorizationSummary, + type AuthorizationTarget, + type PermissionRequirement, +} from "@/lib/authorization"; +import type { IdentityProvider } from "@/services/identity-providers"; + +const mockUseMediaQuery = vi.fn(() => false); +vi.mock("@/hooks/useMediaQuery", () => ({ + useMediaQuery: () => mockUseMediaQuery(), +})); +afterEach(() => mockUseMediaQuery.mockReturnValue(false)); + +const authz = vi.hoisted(() => ({ + summary: undefined as AuthorizationSummary | undefined, +})); +vi.mock("@/services/authorization", () => ({ + useAuthorization: () => ({ + authorization: authz.summary, + canAt: (permission: string, target: AuthorizationTarget) => + canAt(authz.summary, permission, target), + canAnywhere: (permission: string) => + canAnywhere(authz.summary, permission), + meets: (requirement: PermissionRequirement) => + meetsRequirement(authz.summary, requirement), + }), +})); + +const mockUseIdentityProviders = vi.fn(); +vi.mock("@/services/identity-providers", async (importOriginal) => ({ + ...(await importOriginal()), + useIdentityProviders: () => mockUseIdentityProviders(), +})); + +const dialog = vi.hoisted(() => ({ + onCreated: undefined as undefined | ((created: unknown) => void), +})); +vi.mock("@/components/identity-providers/NewIdentityProviderDialog", () => ({ + NewIdentityProviderDialog: (props: { + open: boolean; + onCreated: (created: unknown) => void; + }) => { + dialog.onCreated = props.onCreated; + return props.open ?

    New provider dialog

    : null; + }, +})); + +import { IdentityProviders } from "./IdentityProviders"; + +function summary(admin: boolean): AuthorizationSummary { + return { + is_platform_admin: admin, + home_organization_id: "org-provider", + provider_organization_id: "org-provider", + base_role: { id: "role", name: admin ? "Platform Admin" : "User" }, + grants: admin + ? [] + : [ + { + permission: "identityproviders.read", + boundary: { + kind: "organization", + organization_id: "org-1", + }, + }, + ], + }; +} + +const DAY = 24 * 60 * 60 * 1000; + +function provider(overrides: Partial = {}): IdentityProvider { + return { + id: "idp-1", + organization_id: "org-1", + organization_name: "Contoso", + name: "Contoso Entra ID", + source_type: "scim", + vendor: "entra", + is_enabled: true, + provision_as_external: false, + scim_base_url: "https://bifrost.example/api/scim/v2/idp-1", + token: { + hint: "abcd", + expires_at: new Date(Date.now() + 200 * DAY).toISOString(), + created_at: new Date().toISOString(), + last_used_at: null, + }, + verified_domains: ["contoso.com"], + user_count: 30, + group_count: 2, + mapped_group_count: 2, + last_request_at: null, + last_sync_at: null, + created_at: new Date().toISOString(), + updated_at: new Date().toISOString(), + ...overrides, + }; +} + +function query(data: IdentityProvider[]) { + return { + data, + isLoading: false, + isError: false, + isFetching: false, + refetch: vi.fn(), + }; +} + +function LocationProbe() { + const location = useLocation(); + return ( +

    + {location.pathname}|{JSON.stringify(location.state)} +

    + ); +} + +function renderPage() { + return renderWithProviders( + + } + /> + } /> + , + { initialEntries: ["/users/identity-providers"] }, + ); +} + +describe("IdentityProviders", () => { + beforeEach(() => { + authz.summary = summary(true); + mockUseIdentityProviders.mockReturnValue(query([])); + }); + + it("lists each connection with its domains, status and mapped groups", () => { + mockUseIdentityProviders.mockReturnValue(query([provider()])); + renderPage(); + + const row = screen + .getByRole("link", { name: "Contoso Entra ID" }) + .closest("tr"); + expect(row).not.toBeNull(); + const cells = within(row as HTMLElement); + expect(cells.getByText("Contoso")).toBeInTheDocument(); + expect(cells.getByText("Microsoft Entra ID")).toBeInTheDocument(); + expect(cells.getByText("@contoso.com")).toBeInTheDocument(); + expect(cells.getByText("Enabled")).toBeInTheDocument(); + expect(cells.getByText("2/2")).toBeInTheDocument(); + }); + + it("flags an expired token in the list", () => { + mockUseIdentityProviders.mockReturnValue( + query([ + provider({ + token: { + hint: "abcd", + expires_at: new Date(Date.now() - DAY).toISOString(), + created_at: new Date().toISOString(), + last_used_at: null, + }, + }), + ]), + ); + renderPage(); + + expect(screen.getByText(/^Expired/)).toBeInTheDocument(); + }); + + it("filters by domain", async () => { + mockUseIdentityProviders.mockReturnValue( + query([ + provider(), + provider({ + id: "idp-2", + name: "Fabrikam Okta", + vendor: "okta", + verified_domains: ["fabrikam.com"], + }), + ]), + ); + const { user } = renderPage(); + + await user.type(screen.getByRole("textbox"), "fabrikam"); + + expect( + screen.getByRole("link", { name: "Fabrikam Okta" }), + ).toBeInTheDocument(); + await waitFor(() => + expect( + screen.queryByRole("link", { name: "Contoso Entra ID" }), + ).toBeNull(), + ); + }); + + it("offers Add Provider only to callers who can manage connections", () => { + authz.summary = summary(false); + renderPage(); + + expect( + screen.queryByRole("button", { name: /Add Provider/ }), + ).toBeNull(); + expect(screen.getByText("No Identity Providers")).toBeInTheDocument(); + }); + + it("opens the new connection with its first token to reveal once", async () => { + const { user } = renderPage(); + await user.click(screen.getByRole("button", { name: /Add Provider/ })); + expect(screen.getByText("New provider dialog")).toBeInTheDocument(); + + dialog.onCreated?.({ + identity_provider: { id: "idp-9" }, + token: { token: "bfidp_secret" }, + }); + + expect(await screen.findByTestId("location")).toHaveTextContent( + '/users/identity-providers/idp-9|{"issuedToken":{"token":"bfidp_secret"}}', + ); + }); +}); diff --git a/client/src/pages/IdentityProviders.tsx b/client/src/pages/IdentityProviders.tsx new file mode 100644 index 0000000000..3d203910fc --- /dev/null +++ b/client/src/pages/IdentityProviders.tsx @@ -0,0 +1,337 @@ +import { useMemo, useState } from "react"; +import { Link, useNavigate } from "react-router-dom"; +import { KeyRound, Plus, RefreshCw } from "lucide-react"; + +import { + CredentialExpiryBadge, + ProvisioningStatusBadge, +} from "@/components/identity-providers/IdentityProviderBadges"; +import { + NewIdentityProviderDialog, + type IdentityProviderCreated, +} from "@/components/identity-providers/NewIdentityProviderDialog"; +import { ListLoadError } from "@/components/layout/ListLoadError"; +import { ListPageHeader } from "@/components/layout/ListPageHeader"; +import { ListToolbar } from "@/components/layout/ListToolbar"; +import { + PageScrollArea, + PageWorkspace, +} from "@/components/layout/PageWorkspace"; +import { SearchBox } from "@/components/search/SearchBox"; +import { Button } from "@/components/ui/button"; +import { + DataTable, + DataTableBody, + DataTableCell, + DataTableHead, + DataTableHeader, + DataTableRow, +} from "@/components/ui/data-table"; +import { EmptyState } from "@/components/ui/empty-state"; +import { Skeleton } from "@/components/ui/skeleton"; +import { UsersViewTabs } from "@/components/users/UsersViewTabs"; +import { useMediaQuery } from "@/hooks/useMediaQuery"; +import { cn } from "@/lib/utils"; +import { useAuthorization } from "@/services/authorization"; +import { + IDENTITY_PROVIDERS_PATH, + mappedGroupsLabel, + useIdentityProviders, + vendorLabel, + type IdentityProvider, +} from "@/services/identity-providers"; + +function matches(provider: IdentityProvider, search: string): boolean { + const term = search.trim().toLowerCase(); + return ( + !term || + [ + provider.name, + provider.organization_name, + vendorLabel(provider.vendor), + ...provider.verified_domains, + ].some((value) => value.toLowerCase().includes(term)) + ); +} + +function Domains({ domains }: { domains: string[] }) { + if (domains.length === 0) { + return None Verified; + } + return ( + + {domains.map((domain) => `@${domain}`).join(", ")} + + ); +} + +/** + * Each organization's identity provider connections: the directories that + * provision its users and map their groups to roles. + */ +export function IdentityProviders() { + const navigate = useNavigate(); + const isNarrow = useMediaQuery("(max-width: 1023px)"); + const authorization = useAuthorization(); + const [search, setSearch] = useState(""); + const [isCreateOpen, setIsCreateOpen] = useState(false); + const query = useIdentityProviders(); + const providers = useMemo( + () => (query.data ?? []).filter((p) => matches(p, search)), + [query.data, search], + ); + const detailPath = (provider: Pick) => + `${IDENTITY_PROVIDERS_PATH}/${provider.id}`; + const onCreated = (created: IdentityProviderCreated) => + navigate(detailPath(created.identity_provider), { + state: { issuedToken: created.token }, + }); + + return ( + + } + description="Directories that provision each organization's users over SCIM and map their groups to roles." + actions={ + <> + + {authorization.canAnywhere( + "identityproviders.readwrite", + ) && ( + + )} + + } + /> + + + + + + {query.isError && ( + void query.refetch()} + /> + )} + + {query.isLoading ? ( +
    + {[...Array(3)].map((_, i) => ( + + ))} +
    + ) : query.isError && !query.data ? null : providers.length === + 0 ? ( + + ) : isNarrow ? ( +
      + {providers.map((provider) => ( +
    • +
      + + {provider.name} + + +
      +
      +
      +
      + Organization +
      +
      + {provider.organization_name} +
      +
      +
      +
      + Provider +
      +
      + {vendorLabel(provider.vendor)} +
      +
      +
      +
      + Email Domains +
      +
      + +
      +
      +
      +
      + Token Expiry +
      +
      + +
      +
      +
      +
      + Users · Groups Mapped +
      +
      + {provider.user_count} ·{" "} + {mappedGroupsLabel(provider)} +
      +
      +
      +
    • + ))} +
    + ) : ( + + + + + Name + + + Organization + + + Provider + + Email Domains + + Provisioning + + + Token Expiry + + + Users + + + Groups Mapped + + + + + {providers.map((provider) => ( + + navigate(detailPath(provider)) + } + > + + + {provider.name} + + + + {provider.organization_name} + + + {vendorLabel(provider.vendor)} + + + + + + + + + + + + + {provider.user_count} + + + + + {mappedGroupsLabel(provider)} + + + + ))} + + + )} +
    + + +
    + ); +} diff --git a/client/src/pages/Users.tsx b/client/src/pages/Users.tsx index 3276f9dd55..dcef878b4e 100644 --- a/client/src/pages/Users.tsx +++ b/client/src/pages/Users.tsx @@ -76,6 +76,7 @@ import { } from "@/components/users/BulkUserDialogs"; import { orgTarget } from "@/lib/authorization"; import { useAuthorization } from "@/services/authorization"; +import { identitySourceLabel } from "@/services/identity-providers"; import { ListPagination } from "@/components/pagination/ListPagination"; import type { components, components as v1 } from "@/lib/v1"; type User = components["schemas"]["UserPublic"]; @@ -537,6 +538,14 @@ export function Users() { )} +
    +
    + Identity Source +
    +
    + {identitySourceLabel(user)} +
    +
    Created @@ -655,6 +664,9 @@ export function Users() { /> + + Identity Source + )} + + {identitySourceLabel(user)} + {user.created_at ? new Date( @@ -830,7 +845,7 @@ export function Users() { ({ + useQueryMock: vi.fn(), + useMutationMock: vi.fn(), + invalidateQueries: vi.fn(), +})); + +vi.mock("@/lib/api-client", () => ({ + $api: { + useQuery: (...args: unknown[]) => useQueryMock(...args), + useMutation: (...args: unknown[]) => useMutationMock(...args), + }, +})); + +vi.mock("@tanstack/react-query", () => ({ + useQueryClient: () => ({ invalidateQueries }), +})); + +import { + IDENTITY_PROVIDERS_QUERY_KEY, + credentialStatus, + identitySourceLabel, + mappedGroupsLabel, + useDeleteIdentityProvider, + useIdentityProvider, + useIdentityProviders, + useMapIdentityProviderGroup, + useOrganizationDomains, + useRotateIdentityProviderToken, + useVerifyOrganizationDomain, + vendorLabel, +} from "./identity-providers"; + +const DAY = 24 * 60 * 60 * 1000; + +/** The keys a mutation hook's onSuccess invalidates. */ +function invalidatedBy(hook: () => unknown): unknown[] { + hook(); + const options = useMutationMock.mock.calls[0][2] as { + onSuccess: () => void; + }; + options.onSuccess(); + return invalidateQueries.mock.calls.map(([filter]) => filter.queryKey); +} + +describe("identity providers service", () => { + beforeEach(() => { + useQueryMock.mockReset(); + useMutationMock.mockReset(); + invalidateQueries.mockReset(); + }); + + it("reads connections and one connection from their endpoints", () => { + useIdentityProviders(); + useIdentityProvider("idp-1"); + + expect(useQueryMock).toHaveBeenNthCalledWith( + 1, + "get", + "/api/identity-providers", + ); + expect(useQueryMock).toHaveBeenNthCalledWith( + 2, + "get", + "/api/identity-providers/{identity_provider_id}", + { params: { path: { identity_provider_id: "idp-1" } } }, + ); + }); + + it("scopes domain claims to one organization", () => { + useOrganizationDomains("org-1", { enabled: false }); + + expect(useQueryMock).toHaveBeenCalledWith( + "get", + "/api/identity-providers/domains", + { params: { query: { organization_id: "org-1" } } }, + { enabled: false }, + ); + }); + + it("refreshes the list and detail after a token rotation", () => { + expect(invalidatedBy(useRotateIdentityProviderToken)).toEqual([ + [...IDENTITY_PROVIDERS_QUERY_KEY], + ["get", "/api/identity-providers/{identity_provider_id}"], + ]); + }); + + it("refreshes users after a mapping change, since it changes their roles", () => { + expect(invalidatedBy(useMapIdentityProviderGroup)).toContainEqual([ + "get", + "/api/users", + ]); + }); + + it("refreshes users after deleting a connection, since they become Native", () => { + expect(invalidatedBy(useDeleteIdentityProvider)).toContainEqual([ + "get", + "/api/users", + ]); + }); + + it("refreshes connections after verifying a domain, which they list", () => { + expect(invalidatedBy(useVerifyOrganizationDomain)).toContainEqual([ + ...IDENTITY_PROVIDERS_QUERY_KEY, + ]); + }); + + it("names each vendor", () => { + expect(vendorLabel("entra")).toBe("Microsoft Entra ID"); + expect(vendorLabel("other")).toBe("Other SCIM 2.0"); + }); + + it("flags tokens that are missing, expiring within 14 days, or expired", () => { + const now = new Date("2026-11-16T12:00:00Z"); + const at = (offset: number) => ({ + expires_at: new Date(now.getTime() + offset).toISOString(), + }); + + expect(credentialStatus(null, now)).toBe("none"); + expect(credentialStatus(at(30 * DAY), now)).toBe("active"); + expect(credentialStatus(at(14 * DAY), now)).toBe("expiring"); + expect(credentialStatus(at(-1), now)).toBe("expired"); + }); + + it("names a user's identity source, or Native when none manages them", () => { + expect(identitySourceLabel({ identity_source: null })).toBe("Native"); + expect( + identitySourceLabel({ + identity_source: { + identity_provider_id: "idp-1", + name: "Contoso Entra ID", + vendor: "entra", + }, + }), + ).toBe("Contoso Entra ID"); + }); + + it("shows mapped groups out of all groups", () => { + expect( + mappedGroupsLabel({ mapped_group_count: 2, group_count: 3 }), + ).toBe("2/3"); + }); +}); diff --git a/client/src/services/identity-providers.ts b/client/src/services/identity-providers.ts new file mode 100644 index 0000000000..f1858ef4e4 --- /dev/null +++ b/client/src/services/identity-providers.ts @@ -0,0 +1,221 @@ +import { useQueryClient } from "@tanstack/react-query"; + +import { $api } from "@/lib/api-client"; +import type { components } from "@/lib/v1"; + +export type IdentityProvider = + components["schemas"]["IdentityProviderResponse"]; +export type IdentityProviderVendor = IdentityProvider["vendor"]; +export type IdentityProviderToken = + components["schemas"]["IdentityProviderTokenInfo"]; +export type IdentityProviderTokenIssued = + components["schemas"]["IdentityProviderTokenIssued"]; +export type IdentityProviderGroup = + components["schemas"]["IdentityProviderGroupResponse"]; +export type OrganizationDomain = + components["schemas"]["OrganizationDomainResponse"]; +export type TokenLifetimeDays = + components["schemas"]["IdentityProviderTokenRotate"]["lifetime_days"]; + +/** Where the Identity Providers list lives, under the Users views. */ +export const IDENTITY_PROVIDERS_PATH = "/users/identity-providers"; + +/** Prefix of every identity-provider query key. */ +export const IDENTITY_PROVIDERS_QUERY_KEY = [ + "get", + "/api/identity-providers", +] as const; +const DETAIL_KEY = ["get", "/api/identity-providers/{identity_provider_id}"]; +const GROUPS_KEY = [ + "get", + "/api/identity-providers/{identity_provider_id}/groups", +]; +const DOMAINS_KEY = ["get", "/api/identity-providers/domains"]; +const USERS_KEY = ["get", "/api/users"]; + +/** Tokens close to expiry are flagged this many days ahead. */ +export const EXPIRY_WARNING_DAYS = 14; +export const TOKEN_LIFETIMES: TokenLifetimeDays[] = [90, 180, 365, 730]; + +const VENDOR_LABELS: Record = { + entra: "Microsoft Entra ID", + okta: "Okta", + jumpcloud: "JumpCloud", + google: "Google Workspace", + other: "Other SCIM 2.0", +}; + +export const VENDORS = Object.keys(VENDOR_LABELS) as IdentityProviderVendor[]; + +export function vendorLabel(vendor: IdentityProviderVendor): string { + return VENDOR_LABELS[vendor]; +} + +export type CredentialStatus = "none" | "active" | "expiring" | "expired"; + +/** Whether the connection's token works, and whether it needs replacing soon. */ +export function credentialStatus( + token: Pick | null | undefined, + now: Date = new Date(), +): CredentialStatus { + if (!token) return "none"; + const remaining = new Date(token.expires_at).getTime() - now.getTime(); + if (remaining <= 0) return "expired"; + if (remaining <= EXPIRY_WARNING_DAYS * 24 * 60 * 60 * 1000) { + return "expiring"; + } + return "active"; +} + +/** The connection that manages a user, or "Native" for one Bifrost manages. */ +export function identitySourceLabel( + user: Pick, +): string { + return user.identity_source?.name ?? "Native"; +} + +/** "2/3": mapped groups out of the groups the provider has pushed. */ +export function mappedGroupsLabel( + provider: Pick, +): string { + return `${provider.mapped_group_count}/${provider.group_count}`; +} + +export function useIdentityProviders() { + return $api.useQuery("get", "/api/identity-providers"); +} + +export function useIdentityProvider(identityProviderId: string) { + return $api.useQuery( + "get", + "/api/identity-providers/{identity_provider_id}", + { params: { path: { identity_provider_id: identityProviderId } } }, + ); +} + +export function useIdentityProviderGroups(identityProviderId: string) { + return $api.useQuery( + "get", + "/api/identity-providers/{identity_provider_id}/groups", + { params: { path: { identity_provider_id: identityProviderId } } }, + ); +} + +export function useOrganizationDomains( + organizationId: string, + options: { enabled?: boolean } = {}, +) { + return $api.useQuery( + "get", + "/api/identity-providers/domains", + { params: { query: { organization_id: organizationId } } }, + options, + ); +} + +function useInvalidate(...keys: readonly (readonly string[])[]) { + const queryClient = useQueryClient(); + return () => { + for (const queryKey of keys) { + queryClient.invalidateQueries({ queryKey: [...queryKey] }); + } + }; +} + +/** Creates a connection; the response carries its first token, shown once. */ +export function useCreateIdentityProvider() { + const onSuccess = useInvalidate(IDENTITY_PROVIDERS_QUERY_KEY); + return $api.useMutation("post", "/api/identity-providers", { onSuccess }); +} + +export function useUpdateIdentityProvider() { + const onSuccess = useInvalidate(IDENTITY_PROVIDERS_QUERY_KEY, DETAIL_KEY); + return $api.useMutation( + "patch", + "/api/identity-providers/{identity_provider_id}", + { onSuccess }, + ); +} + +/** Deleting a connection leaves its users as Native accounts. */ +export function useDeleteIdentityProvider() { + const onSuccess = useInvalidate(IDENTITY_PROVIDERS_QUERY_KEY, USERS_KEY); + return $api.useMutation( + "delete", + "/api/identity-providers/{identity_provider_id}", + { onSuccess }, + ); +} + +/** Issues a new token; earlier tokens keep working for a short overlap. */ +export function useRotateIdentityProviderToken() { + const onSuccess = useInvalidate(IDENTITY_PROVIDERS_QUERY_KEY, DETAIL_KEY); + return $api.useMutation( + "post", + "/api/identity-providers/{identity_provider_id}/token", + { onSuccess }, + ); +} + +/** Mapping changes re-evaluate every linked user's roles and access. */ +export function useMapIdentityProviderGroup() { + const onSuccess = useInvalidate( + GROUPS_KEY, + DETAIL_KEY, + IDENTITY_PROVIDERS_QUERY_KEY, + USERS_KEY, + ); + return $api.useMutation( + "put", + "/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping", + { onSuccess }, + ); +} + +export function useUnmapIdentityProviderGroup() { + const onSuccess = useInvalidate( + GROUPS_KEY, + DETAIL_KEY, + IDENTITY_PROVIDERS_QUERY_KEY, + USERS_KEY, + ); + return $api.useMutation( + "delete", + "/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping", + { onSuccess }, + ); +} + +export function useAddOrganizationDomain() { + const onSuccess = useInvalidate(DOMAINS_KEY); + return $api.useMutation("post", "/api/identity-providers/domains", { + onSuccess, + }); +} + +/** Checks DNS now; a verified claim also shows on every connection. */ +export function useVerifyOrganizationDomain() { + const onSuccess = useInvalidate( + DOMAINS_KEY, + IDENTITY_PROVIDERS_QUERY_KEY, + DETAIL_KEY, + ); + return $api.useMutation( + "post", + "/api/identity-providers/domains/{claim_id}/verify", + { onSuccess }, + ); +} + +export function useDeleteOrganizationDomain() { + const onSuccess = useInvalidate( + DOMAINS_KEY, + IDENTITY_PROVIDERS_QUERY_KEY, + DETAIL_KEY, + ); + return $api.useMutation( + "delete", + "/api/identity-providers/domains/{claim_id}", + { onSuccess }, + ); +} From bc05d458ce1222165369727fe78b5c8deaee90a3 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Mon, 16 Nov 2026 14:35:09 -0500 Subject: [PATCH 06/26] Warn before identity provider tokens expire An expired SCIM token silently stops provisioning. An hourly scheduled task now checks each enabled connection's current token (superseded tokens in their rotation overlap are ignored): - Within 14 days of expiry, or past it, it keeps a platform-admin notification up with a Regenerate Token action. Notifications live an hour in Redis, so each run re-raises one that lapsed. - It emits identity_provider.token_expiry once per stage (expiring, then expired), recorded on the token, so a workflow can reach the organization's own admins. - Rotating the token dismisses the outstanding warnings. Co-Authored-By: Claude Opus 5.5 --- .../20261116_idp_token_expiry_stage.py | 34 ++++ .../identity_provider_token_expiry.py | 29 +++ api/src/models/orm/identity_providers.py | 7 + api/src/routers/identity_providers.py | 8 +- api/src/scheduler/registry.py | 7 + api/src/services/events/builtins.py | 23 +++ api/src/services/events/registry.py | 19 ++ api/src/services/identity_providers.py | 7 +- .../services/identity_provisioning/expiry.py | 179 ++++++++++++++++++ api/tests/unit/scheduler/test_control.py | 3 + .../test_identity_provider_token_expiry.py | 152 +++++++++++++++ .../2026-11-16-scim-identity-providers.md | 15 +- 12 files changed, 477 insertions(+), 6 deletions(-) create mode 100644 api/alembic/versions/20261116_idp_token_expiry_stage.py create mode 100644 api/src/jobs/schedulers/identity_provider_token_expiry.py create mode 100644 api/src/services/identity_provisioning/expiry.py create mode 100644 api/tests/unit/services/test_identity_provider_token_expiry.py diff --git a/api/alembic/versions/20261116_idp_token_expiry_stage.py b/api/alembic/versions/20261116_idp_token_expiry_stage.py new file mode 100644 index 0000000000..483ff868a0 --- /dev/null +++ b/api/alembic/versions/20261116_idp_token_expiry_stage.py @@ -0,0 +1,34 @@ +"""Record which token-expiry warning stage was announced. + +Revision ID: 20261116_idp_token_expiry_stage +Revises: 20261116_identity_providers +""" + +import sqlalchemy as sa +from alembic import op + +revision = "20261116_idp_token_expiry_stage" +down_revision = "20261116_identity_providers" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + op.add_column( + "identity_provider_tokens", + sa.Column("expiry_warning_stage", sa.String(16), nullable=True), + ) + op.create_check_constraint( + "ck_identity_provider_tokens_expiry_warning_stage", + "identity_provider_tokens", + "expiry_warning_stage IN ('expiring', 'expired')", + ) + + +def downgrade() -> None: + op.drop_constraint( + "ck_identity_provider_tokens_expiry_warning_stage", + "identity_provider_tokens", + type_="check", + ) + op.drop_column("identity_provider_tokens", "expiry_warning_stage") diff --git a/api/src/jobs/schedulers/identity_provider_token_expiry.py b/api/src/jobs/schedulers/identity_provider_token_expiry.py new file mode 100644 index 0000000000..3f47f058d1 --- /dev/null +++ b/api/src/jobs/schedulers/identity_provider_token_expiry.py @@ -0,0 +1,29 @@ +"""Hourly check for identity provider tokens close to or past expiry.""" + +import logging +from datetime import datetime, timezone + +from src.core.database import get_session_factory +from src.services.identity_provisioning.expiry import ( + collect_expiry_notices, + deliver_expiry_notices, +) + +logger = logging.getLogger(__name__) + + +async def check_identity_provider_token_expiry() -> dict: + """Warn about tokens that expire within 14 days or have expired.""" + session_factory = get_session_factory() + async with session_factory() as db: + notices = await collect_expiry_notices(db, datetime.now(timezone.utc)) + await db.commit() + await deliver_expiry_notices(notices) + summary = { + "expiring": sum(1 for n in notices if n.stage == "expiring"), + "expired": sum(1 for n in notices if n.stage == "expired"), + "announced": sum(1 for n in notices if n.announce), + } + if notices: + logger.info("Identity provider token expiry check: %s", summary) + return summary diff --git a/api/src/models/orm/identity_providers.py b/api/src/models/orm/identity_providers.py index 01c325c2ab..dec09ea3f5 100644 --- a/api/src/models/orm/identity_providers.py +++ b/api/src/models/orm/identity_providers.py @@ -122,6 +122,9 @@ class IdentityProviderToken(Base): expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None) last_used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None) + # The expiry stage ("expiring" | "expired") already announced with a + # platform event, so the hourly check fires each stage once. + expiry_warning_stage: Mapped[str | None] = mapped_column(String(16), default=None) created_by: Mapped[str] = mapped_column(String(255), nullable=False) created_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), default=_now, server_default=text("NOW()") @@ -129,6 +132,10 @@ class IdentityProviderToken(Base): __table_args__ = ( Index("ix_identity_provider_tokens_identity_provider_id", "identity_provider_id"), + CheckConstraint( + "expiry_warning_stage IN ('expiring', 'expired')", + name="ck_identity_provider_tokens_expiry_warning_stage", + ), ) diff --git a/api/src/routers/identity_providers.py b/api/src/routers/identity_providers.py index 36fddf4b43..df2ce2c340 100644 --- a/api/src/routers/identity_providers.py +++ b/api/src/routers/identity_providers.py @@ -23,6 +23,7 @@ ) from src.services import identity_providers as service from src.services.authorization.enforce import load_caller +from src.services.identity_provisioning.expiry import clear_expiry_notifications from src.services.operation_catalog import operation_route router = APIRouter(prefix="/api/identity-providers", tags=["Identity Providers"]) @@ -163,7 +164,12 @@ async def rotate_identity_provider_token( user: CurrentActiveUser, ) -> IdentityProviderTokenIssued: caller = await load_caller(db, user) - return await service.rotate_token(db, caller, identity_provider_id, request.lifetime_days) + issued, name = await service.rotate_token( + db, caller, identity_provider_id, request.lifetime_days + ) + await db.commit() + await clear_expiry_notifications(name) + return issued # --------------------------------------------------------------------------- diff --git a/api/src/scheduler/registry.py b/api/src/scheduler/registry.py index 7c091d138d..abb83dbd93 100644 --- a/api/src/scheduler/registry.py +++ b/api/src/scheduler/registry.py @@ -62,6 +62,13 @@ class ScheduledTaskOutcome: callback="src.jobs.schedulers.metrics_refresh:refresh_metrics_snapshot", trigger_args={"minutes": 60}, run_at_startup=True, ), + # Hourly because admin notifications are Redis-backed with a one-hour + # life: each run re-raises a warning that lapsed while it still applies. + ScheduledTaskDefinition( + "identity_provider_token_expiry", "Warn About Expiring Identity Provider Tokens", "Hourly", + callback="src.jobs.schedulers.identity_provider_token_expiry:check_identity_provider_token_expiry", + trigger_args={"minutes": 60}, run_at_startup=True, + ), ScheduledTaskDefinition( "knowledge_storage_refresh", "Refresh Knowledge Storage Metrics", "Daily at 02:00 UTC", callback="src.jobs.schedulers.knowledge_storage_refresh:refresh_knowledge_storage_daily", diff --git a/api/src/services/events/builtins.py b/api/src/services/events/builtins.py index 8591d98a0b..7d20a88f88 100644 --- a/api/src/services/events/builtins.py +++ b/api/src/services/events/builtins.py @@ -480,6 +480,29 @@ def identity_provider_user_body( } +async def emit_identity_provider_token_expiry( + *, + organization_id: str | UUID, + organization_name: str | None, + identity_provider: dict[str, str | None], + token_hint: str, + expires_at: datetime, + stage: str, +) -> None: + """A connection's token entered the ``expiring`` or ``expired`` stage.""" + body = { + **_base_body(organization_id=organization_id, organization_name=organization_name), + "identity_provider": identity_provider, + "token": {"hint": token_hint, "expires_at": _iso(expires_at), "stage": stage}, + } + await _emit( + "identity_provider.token_expiry", + body, + organization_id=organization_id, + triggered_by=f"identity_provider:{identity_provider['id']}", + ) + + async def emit_identity_provider_event( topic: str, body: dict[str, Any], diff --git a/api/src/services/events/registry.py b/api/src/services/events/registry.py index f112874506..597aa602f4 100644 --- a/api/src/services/events/registry.py +++ b/api/src/services/events/registry.py @@ -137,6 +137,25 @@ def _body(*, actor: dict, **fields: object) -> dict: roles_removed=[], ), }, + { + "topic": "identity_provider.token_expiry", + "description": ( + "Fired once when an identity provider connection's token comes within " + "14 days of expiry, and once more when it expires. Provisioning stops " + "until the token is regenerated and updated in the identity provider." + ), + "category": "Users", + "emitted_by": "Identity provisioning", + "example_body": _body( + actor=_SYSTEM_ACTOR, + identity_provider=_IDENTITY_PROVIDER, + token={ + "hint": "x7Qa", + "expires_at": "2026-06-11T12:34:56Z", + "stage": "expiring", + }, + ), + }, { "topic": "workflow.failed", "description": "Fired when a workflow execution attempt fails.", diff --git a/api/src/services/identity_providers.py b/api/src/services/identity_providers.py index a916b1934b..7aefc3a2bf 100644 --- a/api/src/services/identity_providers.py +++ b/api/src/services/identity_providers.py @@ -359,7 +359,9 @@ async def delete_connection( async def rotate_token( session: AsyncSession, caller: Caller, identity_provider_id: UUID, lifetime_days: int -) -> IdentityProviderTokenIssued: +) -> tuple[IdentityProviderTokenIssued, str]: + """Issue a new token; returns it with the connection's name, whose expiry + warnings the caller clears after committing.""" connection = await _connection( session, caller, identity_provider_id, operation="identityproviders.token.rotate" ) @@ -380,9 +382,10 @@ async def rotate_token( ) info = _token_info(token) assert info is not None - return IdentityProviderTokenIssued( + issued = IdentityProviderTokenIssued( token=raw, info=info, previous_tokens_expire_at=previous_expiry ) + return issued, connection.name # --------------------------------------------------------------------------- diff --git a/api/src/services/identity_provisioning/expiry.py b/api/src/services/identity_provisioning/expiry.py new file mode 100644 index 0000000000..1b7fa19111 --- /dev/null +++ b/api/src/services/identity_provisioning/expiry.py @@ -0,0 +1,179 @@ +"""Token-expiry warnings for identity provider connections. + +An expired token silently stops provisioning: the identity provider's +requests fail and nobody in Bifrost sees them. An hourly scheduled check +looks at each enabled connection's current token and, while it is within +``EXPIRY_WARNING`` of expiry or past it: + +- keeps a platform-admin notification up (notifications are Redis-backed + and live an hour, so each run re-raises one that lapsed), and +- emits ``identity_provider.token_expiry`` once per stage, recorded on the + token, so a workflow can tell the organization's own admins. + +A rotated token starts with no stage; superseded tokens are ignored. +""" + +from __future__ import annotations + +import logging +from dataclasses import dataclass +from datetime import datetime, timedelta +from typing import Literal +from uuid import UUID + +from sqlalchemy import select +from sqlalchemy.ext.asyncio import AsyncSession + +from src.models.orm.identity_providers import IdentityProvider, IdentityProviderToken +from src.models.orm.organizations import Organization + +logger = logging.getLogger(__name__) + +EXPIRY_WARNING = timedelta(days=14) + +Stage = Literal["expiring", "expired"] + + +def token_stage(expires_at: datetime, now: datetime) -> Stage | None: + if expires_at <= now: + return "expired" + if expires_at - now <= EXPIRY_WARNING: + return "expiring" + return None + + +@dataclass(frozen=True) +class ExpiryNotice: + identity_provider_id: UUID + identity_provider_name: str + vendor: str + organization_id: UUID + organization_name: str + token_hint: str + expires_at: datetime + stage: Stage + # True the first time this token reaches this stage. + announce: bool + + +async def collect_expiry_notices(session: AsyncSession, now: datetime) -> list[ExpiryNotice]: + """Each enabled connection whose current token needs attention. Marks + newly reached stages on the tokens; the caller commits, then delivers.""" + rows = ( + await session.execute( + select(IdentityProvider, IdentityProviderToken, Organization.name) + .join( + IdentityProviderToken, + IdentityProviderToken.identity_provider_id == IdentityProvider.id, + ) + .join(Organization, Organization.id == IdentityProvider.organization_id) + .where( + IdentityProvider.is_enabled.is_(True), + IdentityProviderToken.revoked_at.is_(None), + ) + .order_by(IdentityProvider.id, IdentityProviderToken.created_at.desc()) + .distinct(IdentityProvider.id) + ) + ).all() + notices: list[ExpiryNotice] = [] + for connection, token, organization_name in rows: + stage = token_stage(token.expires_at, now) + if stage is None: + continue + announce = token.expiry_warning_stage != stage + if announce: + token.expiry_warning_stage = stage + notices.append( + ExpiryNotice( + identity_provider_id=connection.id, + identity_provider_name=connection.name, + vendor=connection.vendor, + organization_id=connection.organization_id, + organization_name=organization_name, + token_hint=token.token_hint, + expires_at=token.expires_at, + stage=stage, + announce=announce, + ) + ) + await session.flush() + return notices + + +def _titles(identity_provider_name: str) -> dict[Stage, str]: + return { + "expiring": f"SCIM Token Expiring: {identity_provider_name}", + "expired": f"SCIM Token Expired: {identity_provider_name}", + } + + +def notification_title(notice: ExpiryNotice) -> str: + return _titles(notice.identity_provider_name)[notice.stage] + + +async def clear_expiry_notifications(identity_provider_name: str) -> None: + """Dismiss a connection's expiry warnings once a new token is issued.""" + from src.models.contracts.notifications import NotificationCategory + from src.services.notification_service import get_notification_service + + service = get_notification_service() + for title in _titles(identity_provider_name).values(): + existing = await service.find_admin_notification_by_title( + title=title, category=NotificationCategory.SYSTEM + ) + if existing: + await service.dismiss_notification(existing.id, user_id="system") + + +async def deliver_expiry_notices(notices: list[ExpiryNotice]) -> None: + """Send events for new stages and keep admin notifications up.""" + from src.models.contracts.notifications import ( + NotificationCategory, + NotificationCreate, + NotificationStatus, + ) + from src.services.events.builtins import emit_identity_provider_token_expiry + from src.services.notification_service import get_notification_service + + service = get_notification_service() + for notice in notices: + if notice.announce: + await emit_identity_provider_token_expiry( + organization_id=notice.organization_id, + organization_name=notice.organization_name, + identity_provider={ + "id": str(notice.identity_provider_id), + "name": notice.identity_provider_name, + "vendor": notice.vendor, + }, + token_hint=notice.token_hint, + expires_at=notice.expires_at, + stage=notice.stage, + ) + title = notification_title(notice) + if await service.find_admin_notification_by_title( + title=title, category=NotificationCategory.SYSTEM + ): + continue + when = notice.expires_at.strftime("%b %d, %Y") + description = ( + f"{notice.organization_name}: the token " + f"{'expired' if notice.stage == 'expired' else 'expires'} {when}. " + "Regenerate it and update the identity provider." + ) + await service.create_notification( + user_id="system", + request=NotificationCreate( + category=NotificationCategory.SYSTEM, + title=title, + description=description, + metadata={ + "action": "open_identity_provider", + "action_label": "Regenerate Token", + "action_url": f"/users/identity-providers/{notice.identity_provider_id}", + "identity_provider_id": str(notice.identity_provider_id), + }, + ), + for_admins=True, + initial_status=NotificationStatus.AWAITING_ACTION, + ) diff --git a/api/tests/unit/scheduler/test_control.py b/api/tests/unit/scheduler/test_control.py index 5dbcb7f896..8abaf000b7 100644 --- a/api/tests/unit/scheduler/test_control.py +++ b/api/tests/unit/scheduler/test_control.py @@ -88,6 +88,9 @@ def _control(lease: FakeLease, **kwargs) -> ControlPlane: "scheduler_diagnostics_cleanup": ("interval", {"hours": 1}, False), # Added with reusable executors (#769 stage 3). "execution_worker_cleanup": ("interval", {"hours": 1}, False), + # Added with identity provider connections: re-raises token-expiry + # warnings, which live an hour as notifications. + "identity_provider_token_expiry": ("interval", {"minutes": 60}, True), } diff --git a/api/tests/unit/services/test_identity_provider_token_expiry.py b/api/tests/unit/services/test_identity_provider_token_expiry.py new file mode 100644 index 0000000000..a7cc2906a5 --- /dev/null +++ b/api/tests/unit/services/test_identity_provider_token_expiry.py @@ -0,0 +1,152 @@ +"""Token-expiry warnings: which tokens need attention, and each stage is +announced once.""" + +from datetime import datetime, timedelta, timezone +from unittest.mock import AsyncMock, patch +from uuid import uuid4 + +import pytest +import pytest_asyncio + +from src.models.orm.identity_providers import IdentityProvider, IdentityProviderToken +from src.models.orm.organizations import Organization +from src.services.identity_provisioning.expiry import ( + ExpiryNotice, + collect_expiry_notices, + deliver_expiry_notices, + token_stage, +) + +NOW = datetime(2026, 11, 16, 12, 0, tzinfo=timezone.utc) + + +def test_stages(): + assert token_stage(NOW + timedelta(days=30), NOW) is None + assert token_stage(NOW + timedelta(days=14), NOW) == "expiring" + assert token_stage(NOW, NOW) == "expired" + + +@pytest_asyncio.fixture +async def connection(db_session): + organization = Organization(name=f"Contoso {uuid4().hex[:8]}", created_by="test") + db_session.add(organization) + await db_session.flush() + connection = IdentityProvider( + organization_id=organization.id, + name="Contoso Entra", + source_type="scim", + vendor="entra", + created_by="test", + ) + db_session.add(connection) + await db_session.flush() + return connection + + +async def _token(session, connection, expires_in: timedelta, *, created_at: datetime = NOW): + token = IdentityProviderToken( + identity_provider_id=connection.id, + token_hash=uuid4().hex, + token_hint="abcd", + expires_at=NOW + expires_in, + created_by="test", + created_at=created_at, + ) + session.add(token) + await session.flush() + return token + + +async def _notices_for(session, connection): + return [ + n + for n in await collect_expiry_notices(session, NOW) + if n.identity_provider_id == connection.id + ] + + +@pytest.mark.asyncio +async def test_each_stage_is_announced_once(db_session, connection): + token = await _token(db_session, connection, timedelta(days=5)) + + first = await _notices_for(db_session, connection) + second = await _notices_for(db_session, connection) + + assert [(n.stage, n.announce) for n in first] == [("expiring", True)] + assert [(n.stage, n.announce) for n in second] == [("expiring", False)] + assert token.expiry_warning_stage == "expiring" + + token.expires_at = NOW - timedelta(minutes=1) + await db_session.flush() + assert [(n.stage, n.announce) for n in await _notices_for(db_session, connection)] == [ + ("expired", True) + ] + + +@pytest.mark.asyncio +async def test_only_the_current_token_counts(db_session, connection): + # A rotation leaves the old token alive for an hour; that's not a warning. + await _token(db_session, connection, timedelta(minutes=30), created_at=NOW - timedelta(days=1)) + await _token(db_session, connection, timedelta(days=365)) + + assert await _notices_for(db_session, connection) == [] + + +@pytest.mark.asyncio +async def test_disabled_connections_are_skipped(db_session, connection): + await _token(db_session, connection, timedelta(days=-1)) + connection.is_enabled = False + await db_session.flush() + + assert await _notices_for(db_session, connection) == [] + + +def _notice(*, announce: bool) -> ExpiryNotice: + return ExpiryNotice( + identity_provider_id=uuid4(), + identity_provider_name="Contoso Entra", + vendor="entra", + organization_id=uuid4(), + organization_name="Contoso", + token_hint="abcd", + expires_at=NOW + timedelta(days=5), + stage="expiring", + announce=announce, + ) + + +@pytest.mark.asyncio +async def test_delivery_announces_new_stages_and_keeps_one_notification(): + service = AsyncMock() + service.find_admin_notification_by_title.side_effect = [None, object()] + emit = AsyncMock() + with ( + patch("src.services.notification_service.get_notification_service", return_value=service), + patch("src.services.events.builtins.emit_identity_provider_token_expiry", emit), + ): + await deliver_expiry_notices([_notice(announce=True), _notice(announce=False)]) + + assert emit.await_count == 1 + assert emit.await_args.kwargs["stage"] == "expiring" + # The second notice's notification already exists, so only one is raised. + assert service.create_notification.await_count == 1 + metadata = service.create_notification.await_args.kwargs["request"].metadata + assert metadata["action_url"].startswith("/users/identity-providers/") + + +@pytest.mark.asyncio +async def test_rotation_dismisses_outstanding_warnings(): + from src.services.identity_provisioning.expiry import clear_expiry_notifications + + expired = AsyncMock() + expired.id = "n-expired" + service = AsyncMock() + service.find_admin_notification_by_title.side_effect = ( + lambda title, category: expired if title.startswith("SCIM Token Expired") else None + ) + with patch( + "src.services.notification_service.get_notification_service", return_value=service + ): + await clear_expiry_notifications("Contoso Entra") + + service.dismiss_notification.assert_awaited_once_with("n-expired", user_id="system") diff --git a/docs/plans/2026-11-16-scim-identity-providers.md b/docs/plans/2026-11-16-scim-identity-providers.md index 77101310fa..b956078322 100644 --- a/docs/plans/2026-11-16-scim-identity-providers.md +++ b/docs/plans/2026-11-16-scim-identity-providers.md @@ -232,9 +232,18 @@ Topics `user.provisioned`, `user.updated`, `user.deprovisioned` and 2. **UI:** list, detail and edit dialog, the domains card, the token reveal, the mapping editor, and the Identity Source column; vitest plus one Playwright happy path. -3. **Credential expiry warnings:** a daily scheduled task that re-raises admin - notifications (they are Redis-backed with a TTL) for tokens within 14 days - of expiry. +3. **Credential expiry warnings (done):** an hourly scheduled task checks each + enabled connection's current token. Superseded tokens are ignored, so the + one-hour rotation overlap never raises a warning. + - While a token is within 14 days of expiry or past it, the task keeps a + platform-admin notification up with a Regenerate Token action. + Notifications are Redis-backed and live an hour, which is why the check + runs hourly. + - It emits `identity_provider.token_expiry` once per stage (`expiring`, + then `expired`), recorded on the token. A workflow can use it to reach the + organization's own admins, who don't see platform notifications. + - Rotating the token dismisses the outstanding warnings. + - The list and detail pages flag the same states. 4. **Google Directory sync:** connections with `source_type="google_directory"`. The customer supplies a service account with domain-wide delegation and read-only Admin SDK scopes (`admin.directory.group.readonly`, plus From d3a65a522485c4fa943ebd62cdd96488d4a1ed28 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Mon, 16 Nov 2026 14:49:38 -0500 Subject: [PATCH 07/26] Release domain claims when an organization is disabled Deleting an organization only disables it, so its verified domain claims stayed and kept every other organization from verifying those domains. Disabling now releases the claims (reactivating means verifying again, since control of a domain may have changed), and a disabled organization's connections stop: SCIM requests are refused and the expiry check skips them, resuming if the organization is reactivated. Co-Authored-By: Claude Opus 5.5 --- api/shared/sdk_organizations.py | 19 ++++- .../services/identity_provisioning/expiry.py | 1 + .../services/identity_provisioning/tokens.py | 5 ++ .../test_identity_provider_org_lifecycle.py | 77 +++++++++++++++++++ 4 files changed, 100 insertions(+), 2 deletions(-) create mode 100644 api/tests/unit/services/test_identity_provider_org_lifecycle.py diff --git a/api/shared/sdk_organizations.py b/api/shared/sdk_organizations.py index c4f35d4761..0463859adb 100644 --- a/api/shared/sdk_organizations.py +++ b/api/shared/sdk_organizations.py @@ -36,7 +36,7 @@ from typing import TYPE_CHECKING from uuid import UUID -from sqlalchemy import select +from sqlalchemy import delete, select from sqlalchemy.ext.asyncio import AsyncSession from src.core.log_safety import log_safe @@ -352,6 +352,21 @@ async def delete_organization( org.is_active = False org.updated_at = datetime.now(timezone.utc) + # Release the organization's email-domain claims: a disabled organization + # must not keep another one from verifying a domain. Reactivating means + # verifying again, since control of the domain may have changed. + from src.models.orm.identity_providers import OrganizationDomain + + released_domains = list( + ( + await session.execute( + delete(OrganizationDomain) + .where(OrganizationDomain.organization_id == org_id) + .returning(OrganizationDomain.domain) + ) + ).scalars() + ) + await session.flush() try: @@ -367,6 +382,6 @@ async def delete_organization( "organization.delete", resource_type="organization", resource_id=org.id, - details={"name": deleted_name}, + details={"name": deleted_name, "released_domains": released_domains}, ) return deleted_name diff --git a/api/src/services/identity_provisioning/expiry.py b/api/src/services/identity_provisioning/expiry.py index 1b7fa19111..a796957b3c 100644 --- a/api/src/services/identity_provisioning/expiry.py +++ b/api/src/services/identity_provisioning/expiry.py @@ -69,6 +69,7 @@ async def collect_expiry_notices(session: AsyncSession, now: datetime) -> list[E .join(Organization, Organization.id == IdentityProvider.organization_id) .where( IdentityProvider.is_enabled.is_(True), + Organization.is_active.is_(True), IdentityProviderToken.revoked_at.is_(None), ) .order_by(IdentityProvider.id, IdentityProviderToken.created_at.desc()) diff --git a/api/src/services/identity_provisioning/tokens.py b/api/src/services/identity_provisioning/tokens.py index aa053cdaf8..10d8c3d8db 100644 --- a/api/src/services/identity_provisioning/tokens.py +++ b/api/src/services/identity_provisioning/tokens.py @@ -18,6 +18,7 @@ from sqlalchemy.ext.asyncio import AsyncSession from src.models.orm.identity_providers import IdentityProvider, IdentityProviderToken +from src.models.orm.organizations import Organization TOKEN_PREFIX = "bfidp_" ROTATION_OVERLAP = timedelta(hours=1) @@ -101,12 +102,16 @@ async def authenticate( IdentityProvider, IdentityProvider.id == IdentityProviderToken.identity_provider_id, ) + # A disabled organization's connections stop provisioning; they + # resume if it is reactivated. + .join(Organization, Organization.id == IdentityProvider.organization_id) .where( IdentityProviderToken.token_hash == digest, IdentityProviderToken.identity_provider_id == identity_provider_id, IdentityProviderToken.revoked_at.is_(None), IdentityProviderToken.expires_at > now, IdentityProvider.is_enabled.is_(True), + Organization.is_active.is_(True), ) ) ).one_or_none() diff --git a/api/tests/unit/services/test_identity_provider_org_lifecycle.py b/api/tests/unit/services/test_identity_provider_org_lifecycle.py new file mode 100644 index 0000000000..5b64b529f5 --- /dev/null +++ b/api/tests/unit/services/test_identity_provider_org_lifecycle.py @@ -0,0 +1,77 @@ +"""A disabled organization stops provisioning and gives up its domains.""" + +from datetime import datetime, timezone +from uuid import uuid4 + +import pytest +from sqlalchemy import select + +from shared.sdk_organizations import delete_organization +from src.models.orm.identity_providers import IdentityProvider, OrganizationDomain +from src.models.orm.organizations import Organization +from src.services.identity_provisioning import tokens + + +async def _organization(session) -> Organization: + organization = Organization(name=f"Contoso {uuid4().hex[:8]}", created_by="test") + session.add(organization) + await session.flush() + return organization + + +@pytest.mark.asyncio +async def test_a_disabled_organizations_connection_is_refused(db_session): + organization = await _organization(db_session) + connection = IdentityProvider( + organization_id=organization.id, + name="Contoso Entra", + source_type="scim", + vendor="entra", + created_by="test", + ) + db_session.add(connection) + await db_session.flush() + raw, _ = await tokens.issue_token(db_session, connection, lifetime_days=90, created_by="test") + assert await tokens.authenticate(db_session, connection.id, raw) is not None + + organization.is_active = False + await db_session.flush() + + assert await tokens.authenticate(db_session, connection.id, raw) is None + + +@pytest.mark.asyncio +async def test_disabling_an_organization_releases_its_domains(db_session): + domain = f"contoso-{uuid4().hex[:8]}.example" + first = await _organization(db_session) + second = await _organization(db_session) + db_session.add( + OrganizationDomain( + organization_id=first.id, + domain=domain, + verification_token="t", + verified_at=datetime.now(timezone.utc), + created_by="test", + ) + ) + await db_session.flush() + + await delete_organization(db_session, org_id=first.id) + + assert ( + await db_session.scalar( + select(OrganizationDomain).where(OrganizationDomain.organization_id == first.id) + ) + is None + ) + # The domain is free for another organization to verify. + db_session.add( + OrganizationDomain( + organization_id=second.id, + domain=domain, + verification_token="t", + verified_at=datetime.now(timezone.utc), + created_by="test", + ) + ) + await db_session.flush() From c9a26d3ba99cae6c332184d4c16efa5d30031006 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 11:44:20 -0500 Subject: [PATCH 08/26] Batch provisioning grants so a sync burst can't churn every token Bifrost has one global authority epoch: any committed change to a user's activity or name, or to a role assignment, retires every access token on the instance. Applied request by request, a 2,000-user SCIM initial sync did that about 2,000 times in 40 minutes. Provisioning now revokes in the request and defers grants. Activation, new roles and renames are recorded on the link (access_pending_since, display_name) and applied by the access pass, a scheduled task every two minutes that applies up to 500 waiting links, oldest first, in one transaction: at most one epoch change per run. New users are inserted in their final state, since an insert doesn't change the epoch. Admin mapping changes still apply at once. Concurrency, from two GPT-6.1 Sol reviews: - A per-connection advisory policy lock: shared for provisioning writes, exclusive for mapping changes and group deletion, always before user locks. The pass takes every involved connection's lock up front. - Users are locked in id order before links, memberships and assignments are read; SCIM PUT/PATCH/DELETE lock before reading what they change. - Deleting the last mapped group reconciles every linked user. - Cache invalidation runs after commit, once per role. - A failing link rolls back in its own savepoint and is re-queued; a policy-refused grant is audited. user.groups_changed now reports membership only; the new user.access_changed reports applied activation and role changes. Co-Authored-By: Claude Opus 5.5 --- .../versions/20261117_idp_access_pending.py | 43 +++ .../identity_provider_access_pass.py | 22 ++ api/src/models/orm/identity_providers.py | 13 + api/src/routers/identity_providers.py | 4 +- api/src/routers/scim.py | 44 ++- api/src/scheduler/registry.py | 8 + api/src/services/events/registry.py | 27 +- api/src/services/identity_providers.py | 6 +- .../identity_provisioning/access_pass.py | 136 +++++++ .../services/identity_provisioning/core.py | 325 +++++++++++++---- api/tests/unit/scheduler/test_control.py | 3 + .../test_identity_provider_access_pass.py | 340 ++++++++++++++++++ .../test_identity_provisioning_core.py | 17 +- .../2026-11-16-scim-identity-providers.md | 62 +++- 14 files changed, 964 insertions(+), 86 deletions(-) create mode 100644 api/alembic/versions/20261117_idp_access_pending.py create mode 100644 api/src/jobs/schedulers/identity_provider_access_pass.py create mode 100644 api/src/services/identity_provisioning/access_pass.py create mode 100644 api/tests/unit/services/test_identity_provider_access_pass.py diff --git a/api/alembic/versions/20261117_idp_access_pending.py b/api/alembic/versions/20261117_idp_access_pending.py new file mode 100644 index 0000000000..d88ff5a1f3 --- /dev/null +++ b/api/alembic/versions/20261117_idp_access_pending.py @@ -0,0 +1,43 @@ +"""Batch identity provider grants into a periodic access pass. + +Revision ID: 20261117_idp_access_pending +Revises: 20261116_idp_token_expiry_stage + +Every role or activation change bumps the global authority epoch, which +retires every access token on the instance. Provisioning applies revocations +in the request but records grants here for a pass that applies them all in +one transaction. +""" + +import sqlalchemy as sa +from alembic import op + +revision = "20261117_idp_access_pending" +down_revision = "20261116_idp_token_expiry_stage" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + op.add_column( + "identity_provider_users", + sa.Column("display_name", sa.String(255), nullable=True), + ) + op.add_column( + "identity_provider_users", + sa.Column("access_pending_since", sa.DateTime(timezone=True), nullable=True), + ) + op.create_index( + "ix_identity_provider_users_access_pending", + "identity_provider_users", + ["access_pending_since"], + postgresql_where=sa.text("access_pending_since IS NOT NULL"), + ) + + +def downgrade() -> None: + op.drop_index( + "ix_identity_provider_users_access_pending", table_name="identity_provider_users" + ) + op.drop_column("identity_provider_users", "access_pending_since") + op.drop_column("identity_provider_users", "display_name") diff --git a/api/src/jobs/schedulers/identity_provider_access_pass.py b/api/src/jobs/schedulers/identity_provider_access_pass.py new file mode 100644 index 0000000000..4d1f43c7cb --- /dev/null +++ b/api/src/jobs/schedulers/identity_provider_access_pass.py @@ -0,0 +1,22 @@ +"""Apply identity provider grants in one transaction every two minutes.""" + +import logging + +from src.core.database import get_session_factory +from src.services.identity_provisioning.access_pass import run_access_pass + +logger = logging.getLogger(__name__) + + +async def run_identity_provider_access_pass() -> dict: + """Apply the activations, role grants and renames provisioning deferred.""" + session_factory = get_session_factory() + async with session_factory() as db: + result = await run_access_pass(db) + await db.commit() + for provisioner in result.provisioners: + await provisioner.after_commit() + summary = result.summary() + if result.applied or result.failed: + logger.info("Identity provider access pass: %s", summary) + return summary diff --git a/api/src/models/orm/identity_providers.py b/api/src/models/orm/identity_providers.py index dec09ea3f5..dbf78ed2dc 100644 --- a/api/src/models/orm/identity_providers.py +++ b/api/src/models/orm/identity_providers.py @@ -164,6 +164,14 @@ class IdentityProviderUser(Base): ) # True when this connection created the Bifrost user (vs adopting one). created_user: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False) + # The name the provider sent; the access pass copies it to the user. + display_name: Mapped[str | None] = mapped_column(String(255), default=None) + # When a change that grants something (activation, a role, a new name) + # began waiting for the batched access pass; None when nothing waits. + # Revocations never wait. See ``identity_provisioning.access_pass``. + access_pending_since: Mapped[datetime | None] = mapped_column( + DateTime(timezone=True), default=None + ) created_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), default=_now, server_default=text("NOW()") ) @@ -178,6 +186,11 @@ class IdentityProviderUser(Base): UniqueConstraint( "identity_provider_id", "user_name", name="uq_identity_provider_users_user_name" ), + Index( + "ix_identity_provider_users_access_pending", + "access_pending_since", + postgresql_where=text("access_pending_since IS NOT NULL"), + ), ) diff --git a/api/src/routers/identity_providers.py b/api/src/routers/identity_providers.py index df2ce2c340..9c03db461c 100644 --- a/api/src/routers/identity_providers.py +++ b/api/src/routers/identity_providers.py @@ -206,7 +206,7 @@ async def map_identity_provider_group( db, caller, identity_provider_id, group_id, request.role_ids ) await db.commit() - await provisioner.emit_pending_events() + await provisioner.after_commit() return response @@ -221,6 +221,6 @@ async def unmap_identity_provider_group( caller = await load_caller(db, user) provisioner = await service.delete_group_mapping(db, caller, identity_provider_id, group_id) await db.commit() - await provisioner.emit_pending_events() + await provisioner.after_commit() return Response(status_code=204) diff --git a/api/src/routers/scim.py b/api/src/routers/scim.py index caed8a8f79..feccf08628 100644 --- a/api/src/routers/scim.py +++ b/api/src/routers/scim.py @@ -104,7 +104,7 @@ def _paging(start_index: int, count: int | None) -> tuple[int, int]: async def _commit_and_emit(db: DbSession, provisioner: Provisioner) -> None: await db.commit() - await provisioner.emit_pending_events() + await provisioner.after_commit() def _parse_id(resource_id: str) -> UUID: @@ -164,6 +164,15 @@ async def _user_link(provisioner: Provisioner, user_id: str) -> IdentityProvider return link +async def _locked_user_link(provisioner: Provisioner, user_id: str) -> IdentityProviderUser: + """The link, locked for a write: a change computed from it can't + overwrite a concurrent one.""" + link = await provisioner.lock_user_link(_parse_id(user_id)) + if link is None: + raise ScimError(404, "User not found") + return link + + @router.get("/Users") async def list_users( connection: Connection, @@ -240,9 +249,9 @@ async def get_user( async def replace_user( user_id: str, request: Request, connection: Connection, db: DbSession ) -> Response: - provisioner = Provisioner(db, connection) - link = await _user_link(provisioner, user_id) resource, unknown = scim.parse_user(await _json_body(request)) + provisioner = Provisioner(db, connection) + link = await _locked_user_link(provisioner, user_id) try: link = await provisioner.replace_user(link, scim.user_record(resource, unknown)) except ProvisioningError as exc: @@ -257,9 +266,10 @@ async def replace_user( async def patch_user( user_id: str, request: Request, connection: Connection, db: DbSession ) -> Response: + patch = await _json_body(request) provisioner = Provisioner(db, connection) - link = await _user_link(provisioner, user_id) - resource, unknown = scim.apply_user_patch(link, await _json_body(request)) + link = await _locked_user_link(provisioner, user_id) + resource, unknown = scim.apply_user_patch(link, patch) try: link = await provisioner.replace_user(link, scim.user_record(resource, unknown)) except ProvisioningError as exc: @@ -273,7 +283,7 @@ async def patch_user( @router.delete("/Users/{user_id}") async def delete_user(user_id: str, connection: Connection, db: DbSession) -> Response: provisioner = Provisioner(db, connection) - link = await _user_link(provisioner, user_id) + link = await _locked_user_link(provisioner, user_id) try: await provisioner.delete_user(link) except ProvisioningError as exc: @@ -294,6 +304,17 @@ async def _group(provisioner: Provisioner, group_id: str) -> IdentityProviderGro return group +async def _locked_group( + provisioner: Provisioner, group_id: str, *, exclusive_policy: bool = False +) -> IdentityProviderGroup: + """The group, locked for a write: a membership change computed from it + can't overwrite a concurrent one.""" + group = await provisioner.lock_group(_parse_id(group_id), exclusive_policy=exclusive_policy) + if group is None: + raise ScimError(404, "Group not found") + return group + + def _wants_members(attributes: str | None, excluded_attributes: str | None) -> bool: excluded = {a.strip().lower() for a in (excluded_attributes or "").split(",")} included = {a.strip().split(".")[0].lower() for a in (attributes or "").split(",") if a.strip()} @@ -421,9 +442,9 @@ async def get_group( async def replace_group( group_id: str, request: Request, connection: Connection, db: DbSession ) -> Response: - provisioner = Provisioner(db, connection) - group = await _group(provisioner, group_id) resource = scim.parse_group(await _json_body(request)) + provisioner = Provisioner(db, connection) + group = await _locked_group(provisioner, group_id) try: await provisioner.replace_group(group, scim.group_record(resource)) await provisioner.set_group_members(group, scim.group_member_ids(resource)) @@ -439,10 +460,11 @@ async def replace_group( async def patch_group( group_id: str, request: Request, connection: Connection, db: DbSession ) -> Response: + patch = await _json_body(request) provisioner = Provisioner(db, connection) - group = await _group(provisioner, group_id) + group = await _locked_group(provisioner, group_id) current_members = await provisioner.group_member_ids(group.id) - resource = scim.apply_group_patch(group, current_members, await _json_body(request)) + resource = scim.apply_group_patch(group, current_members, patch) try: await provisioner.replace_group(group, scim.group_record(resource)) await provisioner.set_group_members(group, scim.group_member_ids(resource)) @@ -456,7 +478,7 @@ async def patch_group( @router.delete("/Groups/{group_id}") async def delete_group(group_id: str, connection: Connection, db: DbSession) -> Response: provisioner = Provisioner(db, connection) - group = await _group(provisioner, group_id) + group = await _locked_group(provisioner, group_id, exclusive_policy=True) try: await provisioner.delete_group(group) except ProvisioningError as exc: diff --git a/api/src/scheduler/registry.py b/api/src/scheduler/registry.py index abb83dbd93..3a1f7d45a0 100644 --- a/api/src/scheduler/registry.py +++ b/api/src/scheduler/registry.py @@ -69,6 +69,14 @@ class ScheduledTaskOutcome: callback="src.jobs.schedulers.identity_provider_token_expiry:check_identity_provider_token_expiry", trigger_args={"minutes": 60}, run_at_startup=True, ), + # Every role or activation change retires every access token on the + # instance; provisioning grants wait for this pass so a sync burst + # changes the authority epoch at most once a run. + ScheduledTaskDefinition( + "identity_provider_access_pass", "Apply Identity Provider Access Changes", "Every 2 minutes", + callback="src.jobs.schedulers.identity_provider_access_pass:run_identity_provider_access_pass", + trigger_args={"minutes": 2}, run_at_startup=True, + ), ScheduledTaskDefinition( "knowledge_storage_refresh", "Refresh Knowledge Storage Metrics", "Daily at 02:00 UTC", callback="src.jobs.schedulers.knowledge_storage_refresh:refresh_knowledge_storage_daily", diff --git a/api/src/services/events/registry.py b/api/src/services/events/registry.py index 597aa602f4..95b3d6bd18 100644 --- a/api/src/services/events/registry.py +++ b/api/src/services/events/registry.py @@ -121,7 +121,10 @@ def _body(*, actor: dict, **fields: object) -> dict: }, { "topic": "user.groups_changed", - "description": "Fired when an identity provider adds a user to a group or removes them.", + "description": ( + "Fired when an identity provider adds a user to a group or removes them. " + "The access that follows is reported by user.access_changed." + ), "category": "Users", "emitted_by": "Identity provisioning", "example_body": _body( @@ -133,8 +136,26 @@ def _body(*, actor: dict, **fields: object) -> dict: "display_name": "Support Techs", "change": "added", }, - roles_added=["550e8400-e29b-41d4-a716-446655440053"], - roles_removed=[], + ), + }, + { + "topic": "user.access_changed", + "description": ( + "Fired when identity provisioning activates a user or grants or removes " + "mapped roles. Removals apply at once; activations and grants are applied " + "in batches within about two minutes of the identity provider's change." + ), + "category": "Users", + "emitted_by": "Identity provisioning", + "example_body": _body( + actor=_IDENTITY_PROVIDER_ACTOR, + identity_provider=_IDENTITY_PROVIDER, + user=_PROVISIONED_USER, + access={ + "activated": True, + "roles_added": ["550e8400-e29b-41d4-a716-446655440053"], + "roles_removed": [], + }, ), }, { diff --git a/api/src/services/identity_providers.py b/api/src/services/identity_providers.py index 7aefc3a2bf..17bf9a119f 100644 --- a/api/src/services/identity_providers.py +++ b/api/src/services/identity_providers.py @@ -503,6 +503,8 @@ async def set_group_mapping( group = await _group(session, connection, group_id) wanted = set(role_ids) await _authorize_mapped_roles(session, caller, connection.organization_id, wanted) + provisioner = Provisioner(session, connection) + await provisioner.hold_policy(exclusive=True) mapping = await session.get(IdentityProviderGroupMapping, group.id) if mapping is None: session.add( @@ -531,7 +533,6 @@ async def set_group_mapping( "role_ids": sorted(str(r) for r in wanted), }, ) - provisioner = Provisioner(session, connection) await provisioner.reconcile_all() (response,) = [g for g in await list_groups(session, caller, connection.id) if g.id == group.id] return response, provisioner @@ -544,6 +545,8 @@ async def delete_group_mapping( session, caller, identity_provider_id, operation="identityproviders.groups.unmap" ) group = await _group(session, connection, group_id) + provisioner = Provisioner(session, connection) + await provisioner.hold_policy(exclusive=True) mapping = await session.get(IdentityProviderGroupMapping, group.id) if mapping is None: raise HTTPException(404, "This group is not mapped") @@ -556,7 +559,6 @@ async def delete_group_mapping( resource_id=connection.id, details={"group": group.display_name}, ) - provisioner = Provisioner(session, connection) await provisioner.reconcile_all() return provisioner diff --git a/api/src/services/identity_provisioning/access_pass.py b/api/src/services/identity_provisioning/access_pass.py new file mode 100644 index 0000000000..acdb3f8445 --- /dev/null +++ b/api/src/services/identity_provisioning/access_pass.py @@ -0,0 +1,136 @@ +"""The access pass: applies provisioning grants in batches. + +Bifrost has one global authority epoch. Any committed change to a fact an +access token carries (a user's activity or name, a role assignment) replaces +it, and every access token on the instance must then refresh. A SCIM +initial sync sends one request per user or membership change, so applying +each grant in its own request would retire every token on the platform about +once a second for the length of the sync. + +So provisioning applies revocations at once, in the request, and records +grants on the link (``IdentityProviderUser.access_pending_since``). This pass +runs every ``INTERVAL`` and applies the waiting grants across all connections +in one transaction: at most one epoch change per run, so grants add at most +30 an hour. Revocations stay in the request and count separately. + +The pass is short leader-only scheduler work (``BATCH_LIMIT`` links, no +outside calls before the commit) and idempotent, so a failover rerun is +harmless. +""" + +from __future__ import annotations + +import logging +from dataclasses import dataclass, field +from datetime import datetime, timedelta, timezone +from typing import Any +from uuid import UUID + +from sqlalchemy import Select, select, update +from sqlalchemy.ext.asyncio import AsyncSession + +from src.models.orm.identity_providers import IdentityProvider, IdentityProviderUser +from src.models.orm.organizations import Organization +from src.models.orm.users import User +from src.services.identity_provisioning.core import Provisioner + +logger = logging.getLogger(__name__) + +# How often the pass runs: the longest a grant waits. +INTERVAL = timedelta(minutes=2) +# Links applied per run; the rest wait for the next run, oldest first. +BATCH_LIMIT = 500 + + +@dataclass +class AccessPassResult: + applied: int = 0 + failed: int = 0 + # Provisioners to run ``after_commit`` on once the caller commits. + provisioners: list[Provisioner] = field(default_factory=list) + + def summary(self) -> dict[str, int]: + return {"applied": self.applied, "failed": self.failed} + + +def _waiting(*columns: Any) -> Select[Any]: + """Links waiting for a grant on an enabled connection in an active + organization (a disabled one's grants wait until it's enabled again).""" + return ( + select(*columns) + .select_from(IdentityProviderUser) + .join(IdentityProvider, IdentityProvider.id == IdentityProviderUser.identity_provider_id) + .join(Organization, Organization.id == IdentityProvider.organization_id) + .where( + IdentityProviderUser.access_pending_since.is_not(None), + IdentityProvider.is_enabled.is_(True), + Organization.is_active.is_(True), + ) + ) + + +async def run_access_pass(session: AsyncSession) -> AccessPassResult: + """Apply waiting grants; the caller commits, then runs each + provisioner's ``after_commit``. + + The oldest waiting links go first, whatever their connection. The pass + takes every involved connection's policy lock (shared, in id order) + before touching any user or role, so it never holds one connection's + rows while waiting on another's mapping change. Users a request holds + are skipped until the next run. A link that fails rolls back alone and + goes to the back of the queue. + """ + candidates = ( + await session.execute( + _waiting(IdentityProviderUser.id, IdentityProviderUser.identity_provider_id) + .order_by(IdentityProviderUser.access_pending_since, IdentityProviderUser.id) + .limit(BATCH_LIMIT) + ) + ).all() + result = AccessPassResult() + if not candidates: + return result + provisioners: dict[UUID, Provisioner] = {} + for connection_id in sorted({connection_id for _, connection_id in candidates}): + connection = await session.get(IdentityProvider, connection_id) + assert connection is not None + provisioner = Provisioner(session, connection) + await provisioner.hold_policy() + provisioners[connection_id] = provisioner + # Recheck under the locks: a request may have applied or cleared a link, + # or a connection may have been disabled, since the candidates were read. + locked = set( + await session.scalars( + _waiting(IdentityProviderUser.id) + .join(User, User.id == IdentityProviderUser.user_id) + .where(IdentityProviderUser.id.in_([link_id for link_id, _ in candidates])) + .with_for_update(of=User, skip_locked=True) + ) + ) + for link_id, connection_id in candidates: + if link_id not in locked: + continue + provisioner = provisioners[connection_id] + queued = len(provisioner.pending_events) + stale_roles, stale_users = set(provisioner.stale_roles), set(provisioner.stale_users) + try: + async with session.begin_nested(): + await provisioner.apply_pending([link_id]) + except Exception: + del provisioner.pending_events[queued:] + provisioner.stale_roles, provisioner.stale_users = stale_roles, stale_users + result.failed += 1 + logger.exception( + "Identity provider %s could not apply access for link %s", + connection_id, + link_id, + ) + await session.execute( + update(IdentityProviderUser) + .where(IdentityProviderUser.id == link_id) + .values(access_pending_since=datetime.now(timezone.utc)) + ) + continue + result.applied += 1 + result.provisioners = list(provisioners.values()) + return result diff --git a/api/src/services/identity_provisioning/core.py b/api/src/services/identity_provisioning/core.py index 106836c614..3ce2b6bd3f 100644 --- a/api/src/services/identity_provisioning/core.py +++ b/api/src/services/identity_provisioning/core.py @@ -17,10 +17,18 @@ - Deleting a user disables and unlinks them, except a user this connection created who has never signed in: that account has no history to keep and is removed permanently. - -Audit rows are written in the transaction; platform events are collected in -``pending_events`` and sent with :meth:`Provisioner.emit_pending_events` -after the caller commits. +- Revoke now, grant later: deactivation and role removal apply in the + request; activation, new roles and renames wait for the access pass + (``access_pass``), which applies a whole burst in one transaction. Each of + those writes changes the global authority epoch. + +Locking: every command takes the connection's policy lock (shared; a mapping +change takes it exclusively) before any user row lock, and locks users in id +order before reading the links and memberships it decides from. + +Audit rows are written in the transaction; platform events and cache +invalidations are collected and run by :meth:`Provisioner.after_commit` once +the caller commits. """ from __future__ import annotations @@ -33,7 +41,7 @@ from typing import Any from uuid import UUID -from sqlalchemy import delete, or_, select +from sqlalchemy import delete, or_, select, text from sqlalchemy.ext.asyncio import AsyncSession from shared.builtin_roles import ( @@ -115,6 +123,10 @@ def __init__(self, session: AsyncSession, connection: IdentityProvider): self.session = session self.connection = connection self.pending_events: list[tuple[str, dict[str, Any]]] = [] + # Cache entries to drop once the transaction commits. + self.stale_roles: set[UUID] = set() + self.stale_users: set[UUID] = set() + self._policy_lock: str | None = None self._actor = ActorContext( user_id=None, organization_id=connection.organization_id, @@ -124,6 +136,28 @@ def __init__(self, session: AsyncSession, connection: IdentityProvider): surface="service", ) + async def hold_policy(self, *, exclusive: bool = False) -> None: + """Take the connection's policy lock for this transaction. + + Every provisioning command decides access from the connection's group + mappings, and a mapping change recomputes every linked user. Commands + share the lock and a mapping change takes it exclusively, so a user + created while the first mapping is added can't be missed by either. + Take it before any user row lock; every path keeps that order. Decide + the mode up front: two transactions upgrading shared to exclusive + would deadlock, so an upgrade is refused. + """ + if self._policy_lock == "exclusive" or (self._policy_lock == "shared" and not exclusive): + return + if self._policy_lock == "shared": + raise RuntimeError("The policy lock is held shared; take it exclusively from the start") + function = "pg_advisory_xact_lock" if exclusive else "pg_advisory_xact_lock_shared" + await self.session.execute( + text(f"SELECT {function}(hashtextextended(:key, 0))"), + {"key": f"identity_provider_policy:{self.connection.id}"}, + ) + self._policy_lock = "exclusive" if exclusive else "shared" + # ------------------------------------------------------------------ # Lookups # ------------------------------------------------------------------ @@ -136,6 +170,43 @@ async def get_user_link(self, link_id: UUID) -> IdentityProviderUser | None: ) ) + async def lock_user_link(self, link_id: UUID) -> IdentityProviderUser | None: + """The link, read after taking the policy lock and its user's row + lock, so a change computed from it (a PATCH) can't overwrite a + concurrent one.""" + await self.hold_policy() + user_id = await self.session.scalar( + select(IdentityProviderUser.user_id).where( + IdentityProviderUser.id == link_id, + IdentityProviderUser.identity_provider_id == self.connection.id, + ) + ) + if user_id is None: + return None + await self._lock_user(user_id) + return await self.session.scalar( + select(IdentityProviderUser) + .where(IdentityProviderUser.id == link_id) + .execution_options(populate_existing=True) + ) + + async def lock_group( + self, group_id: UUID, *, exclusive_policy: bool = False + ) -> IdentityProviderGroup | None: + """The group, read after taking the policy lock and its row lock, so + a membership change computed from it can't overwrite a concurrent + one. Deleting a group changes policy: pass ``exclusive_policy``.""" + await self.hold_policy(exclusive=exclusive_policy) + return await self.session.scalar( + select(IdentityProviderGroup) + .where( + IdentityProviderGroup.id == group_id, + IdentityProviderGroup.identity_provider_id == self.connection.id, + ) + .with_for_update() + .execution_options(populate_existing=True) + ) + async def get_group(self, group_id: UUID) -> IdentityProviderGroup | None: return await self.session.scalar( select(IdentityProviderGroup).where( @@ -157,30 +228,42 @@ async def group_member_ids(self, group_id: UUID) -> set[UUID]: # ------------------------------------------------------------------ async def create_user(self, record: UserRecord) -> IdentityProviderUser: + await self.hold_policy() await self._refuse_duplicate_link(record, exclude_link_id=None) email = normalize_email(record.email) await self._require_claimed_domain(email) - user = await self.session.scalar(select(User).where(User.email == email)) + # Lock a user being adopted before deciding it may be. + user = await self.session.scalar( + select(User) + .where(User.email == email) + .with_for_update() + .execution_options(populate_existing=True) + ) created_user = user is None if user is None: - user = await self._new_user(email, record) + # Insert in the final state: a new user carries no token, so the + # insert leaves the authority epoch alone where a later flip of + # is_active would not. A new user has no groups yet, so once the + # connection maps any group, the user waits for one. + active = record.active and not await self._mapped_group_roles() + user = await self._new_user(email, record, active=active) else: await self._require_adoptable(user) - user.name = record.display_name or user.name link = IdentityProviderUser( identity_provider_id=self.connection.id, user_id=user.id, external_id=record.external_id, user_name=record.user_name, + display_name=record.display_name, active=record.active, attributes=record.attributes, created_user=created_user, ) self.session.add(link) await self.session.flush() - await self._reconcile_access([link.id]) + await self._reconcile_access([link.id], defer_grants=True) await self._audit( "identity_provider.user_provisioned", user, @@ -192,9 +275,9 @@ async def create_user(self, record: UserRecord) -> IdentityProviderUser: async def replace_user( self, link: IdentityProviderUser, record: UserRecord ) -> IdentityProviderUser: + await self.hold_policy() await self._refuse_duplicate_link(record, exclude_link_id=link.id) - user = await self.session.get(User, link.user_id, with_for_update=True) - assert user is not None + user = await self._lock_user(link.user_id) email = normalize_email(record.email) if email != user.email: await self._require_claimed_domain(email) @@ -206,20 +289,20 @@ async def replace_user( ProvisioningErrorKind.CONFLICT, "Another user already has this email" ) user.email = email - user.name = record.display_name or user.name link.external_id = record.external_id link.user_name = record.user_name + link.display_name = record.display_name link.active = record.active link.attributes = record.attributes await self.session.flush() - await self._reconcile_access([link.id]) + await self._reconcile_access([link.id], defer_grants=True) await self._audit("identity_provider.user_updated", user, {"email": user.email}) self._queue_user_event("user.updated", user, link) return link async def delete_user(self, link: IdentityProviderUser) -> None: - user = await self.session.get(User, link.user_id, with_for_update=True) - assert user is not None + await self.hold_policy() + user = await self._lock_user(link.user_id) if link.created_user and user.last_login is None: from shared.run_cancellation import cancel_unstarted_runs_for_deleted_user @@ -240,8 +323,7 @@ async def delete_user(self, link: IdentityProviderUser) -> None: removed = await self._remove_granted_roles(user.id, None) await self.session.delete(link) await self.session.flush() - for role_id in removed: - await self._invalidate_role(role_id) + self._mark_stale(user.id, removed) await self._audit("identity_provider.user_unlinked", user, {"email": user.email}) self.pending_events.append( ( @@ -285,6 +367,14 @@ async def replace_group( async def set_group_members( self, group: IdentityProviderGroup, member_link_ids: Iterable[UUID] ) -> None: + await self.hold_policy() + # Two membership changes to one group serialize on the group row, so + # neither computes its difference from a stale member list. + await self.session.execute( + select(IdentityProviderGroup.id) + .where(IdentityProviderGroup.id == group.id) + .with_for_update() + ) wanted = await self._known_links(set(member_link_ids)) current = await self.group_member_ids(group.id) added, removed = wanted - current, current - wanted @@ -301,42 +391,105 @@ async def set_group_members( ) await self.session.flush() if added or removed: - await self._reconcile_access(added | removed, group_changes={ - link_id: ("added" if link_id in added else "removed") for link_id in added | removed - }, group=group) + await self._reconcile_access( + added | removed, + defer_grants=True, + group_changes={ + link_id: ("added" if link_id in added else "removed") + for link_id in added | removed + }, + group=group, + ) async def delete_group(self, group: IdentityProviderGroup) -> None: - members = await self.group_member_ids(group.id) + # Deleting a mapped group deletes its mapping: a policy change. + await self.hold_policy(exclusive=True) + affected = await self.group_member_ids(group.id) + mapped = await self._mapped_group_roles() + if mapped.keys() == {group.id}: + # Deleting the last mapped group lifts the mapped-group + # requirement from every linked user, members or not. + affected = set( + await self.session.scalars( + select(IdentityProviderUser.id).where( + IdentityProviderUser.identity_provider_id == self.connection.id + ) + ) + ) await self.session.delete(group) await self.session.flush() - if members: - await self._reconcile_access(members) + await self._reconcile_access(affected, defer_grants=True) # ------------------------------------------------------------------ # Access reconciliation # ------------------------------------------------------------------ async def reconcile_all(self) -> None: - """Recompute access for every linked user (after a mapping change).""" + """Recompute and apply access for every linked user, grants included: + an administrator changed a mapping and expects it to take effect. + The caller took the policy lock exclusively before the change.""" + assert self._policy_lock == "exclusive", "take hold_policy(exclusive=True) first" link_ids = await self.session.scalars( select(IdentityProviderUser.id).where( IdentityProviderUser.identity_provider_id == self.connection.id ) ) - await self._reconcile_access(set(link_ids)) + await self._reconcile_access(set(link_ids), defer_grants=False) + + async def apply_pending(self, link_ids: Iterable[UUID]) -> None: + """Apply the grants these links wait for (the access pass).""" + await self._reconcile_access(link_ids, defer_grants=False) async def _reconcile_access( self, link_ids: Iterable[UUID], *, + defer_grants: bool, group_changes: dict[UUID, str] | None = None, group: IdentityProviderGroup | None = None, ) -> None: + """Bring each linked user's activity, mapped roles and name in line + with the directory. + + Revocations (deactivating, removing roles this connection granted) + always apply now. With ``defer_grants``, grants (activating, adding a + role, renaming) are recorded on the link for the access pass instead + (``access_pass``). Each of those writes changes the global authority + epoch, which retires every access token on the instance; the pass + applies a whole provisioning burst in one transaction. + """ link_ids = set(link_ids) if not link_ids: return + await self.hold_policy() mapped_roles = await self._mapped_group_roles() has_mappings = bool(mapped_roles) + # Lock users in id order so a change touching many users can't + # deadlock with another, then read everything the decision rests on + # (links, memberships, assignments) fresh: a writer that held a lock + # has committed by now, and READ COMMITTED shows its work. + user_ids = set( + await self.session.scalars( + select(IdentityProviderUser.user_id).where(IdentityProviderUser.id.in_(link_ids)) + ) + ) + users = { + user.id: user + for user in await self.session.scalars( + select(User) + .where(User.id.in_(user_ids)) + .order_by(User.id) + .with_for_update() + .execution_options(populate_existing=True) + ) + } + links = ( + await self.session.scalars( + select(IdentityProviderUser) + .where(IdentityProviderUser.id.in_(link_ids)) + .execution_options(populate_existing=True) + ) + ).all() memberships: dict[UUID, set[UUID]] = {} for link_id, group_id in ( await self.session.execute( @@ -348,32 +501,39 @@ async def _reconcile_access( ).all(): memberships.setdefault(link_id, set()).add(group_id) - links = ( - await self.session.scalars( - select(IdentityProviderUser).where(IdentityProviderUser.id.in_(link_ids)) - ) - ).all() - for link in links: - user = await self.session.get(User, link.user_id, with_for_update=True) - assert user is not None + now = datetime.now(timezone.utc) + for link in sorted(links, key=lambda link: link.user_id): + user = users[link.user_id] mapped = memberships.get(link.id, set()) & mapped_roles.keys() desired_roles: set[UUID] = set() for group_id in mapped: desired_roles |= mapped_roles[group_id] desired_active = link.active and (not has_mappings or bool(mapped)) + desired_name = link.display_name or user.name + change = _AccessChange() - if user.is_active != desired_active: - await self._set_active(user, desired_active) - change.activated = desired_active - change.deactivated = not desired_active + if user.is_active and not desired_active: + await self._set_active(user, False) + change.deactivated = True change.roles_removed = await self._remove_granted_roles(user.id, desired_roles) - change.roles_added = await self._grant_roles(user, desired_roles) - for role_id in change.roles_added | change.roles_removed: - await self._invalidate_role(role_id) - if change.roles_added or change.roles_removed: - from shared.role_cache import invalidate_user + if defer_grants: + waiting = ( + (desired_active and not user.is_active) + or bool(await self._missing_roles(user, desired_roles)) + or user.name != desired_name + ) + link.access_pending_since = (link.access_pending_since or now) if waiting else None + else: + if desired_active and not user.is_active: + await self._set_active(user, True) + change.activated = True + change.roles_added = await self._grant_roles(user, desired_roles) + if user.name != desired_name: + user.name = desired_name + link.access_pending_since = None + await self.session.flush() - await invalidate_user(user.id) + self._mark_stale(user.id, change.roles_added | change.roles_removed) if group is not None and group_changes and link.id in group_changes: self.pending_events.append( ( @@ -385,8 +545,20 @@ async def _reconcile_access( "display_name": group.display_name, "change": group_changes[link.id], }, - "roles_added": sorted(str(r) for r in change.roles_added), - "roles_removed": sorted(str(r) for r in change.roles_removed), + }, + ) + ) + if change.activated or change.roles_added or change.roles_removed: + self.pending_events.append( + ( + "user.access_changed", + { + **self._user_event_body(user, link), + "access": { + "activated": change.activated, + "roles_added": sorted(str(r) for r in change.roles_added), + "roles_removed": sorted(str(r) for r in change.roles_removed), + }, }, ) ) @@ -431,6 +603,13 @@ async def _mapped_group_roles(self) -> dict[UUID, set[UUID]]: roles.add(role_id) return mapped + async def _missing_roles(self, user: User, desired: set[UUID]) -> set[UUID]: + """Desired roles the user doesn't hold yet.""" + held = set( + await self.session.scalars(select(UserRole.role_id).where(UserRole.user_id == user.id)) + ) + return desired - held - {user.base_role_id} + async def _grant_roles(self, user: User, desired: set[UUID]) -> set[UUID]: from shared.account_guards import AccountGuardError from src.services.user_role_assignments import ( @@ -439,11 +618,8 @@ async def _grant_roles(self, user: User, desired: set[UUID]) -> set[UUID]: insert_assignment, ) - held = set( - await self.session.scalars(select(UserRole.role_id).where(UserRole.user_id == user.id)) - ) added: set[UUID] = set() - for role_id in sorted(desired - held - {user.base_role_id}): + for role_id in sorted(await self._missing_roles(user, desired)): try: await insert_assignment( self.session, @@ -454,14 +630,22 @@ async def _grant_roles(self, user: User, desired: set[UUID]) -> set[UUID]: identity_provider_id=self.connection.id, ) except (RoleAssignmentError, AccountGuardError) as exc: - # e.g. a privileged role mapped for an External user: skip the - # grant, keep provisioning the rest. + # A policy refusal, e.g. a privileged role mapped for an + # External user. Retrying can't change it, so skip the grant, + # keep provisioning the rest and record why; the next mapping + # or directory change tries again. + detail = exc.detail logger.warning( "Identity provider %s skipped role %s for user %s: %s", self.connection.id, role_id, user.id, - log_safe(str(exc)), + log_safe(detail), + ) + await self._audit( + "identity_provider.role_grant_refused", + user, + {"email": user.email, "role_id": str(role_id), "reason": detail}, ) continue added.add(role_id) @@ -505,10 +689,22 @@ async def _set_active(self, user: User, active: bool) -> None: user.is_active = True await self.session.flush() - async def _invalidate_role(self, role_id: UUID) -> None: - from src.core.cache import invalidate_role_users + async def _lock_user(self, user_id: UUID) -> User: + user = await self.session.scalar( + select(User) + .where(User.id == user_id) + .with_for_update() + .execution_options(populate_existing=True) + ) + assert user is not None + return user - await invalidate_role_users(None, str(role_id)) + def _mark_stale(self, user_id: UUID, role_ids: set[UUID]) -> None: + """Queue cache invalidation for after the commit. Each role's + invalidation scans Redis, so a burst touching one role scans once.""" + if role_ids: + self.stale_roles |= role_ids + self.stale_users.add(user_id) # ------------------------------------------------------------------ # Guards @@ -601,7 +797,7 @@ async def _known_links(self, link_ids: set[UUID]) -> set[UUID]: # Creation, audit and events # ------------------------------------------------------------------ - async def _new_user(self, email: str, record: UserRecord) -> User: + async def _new_user(self, email: str, record: UserRecord, *, active: bool) -> User: from shared.base_role_defaults import BaseRoleDefaultError, resolve_base_role from shared.identities import is_identity_email from shared.sdk_users import set_user_base_role @@ -619,8 +815,7 @@ async def _new_user(self, email: str, record: UserRecord) -> User: email=email, name=record.display_name, hashed_password="", - # Activation is decided by reconciliation right after linking. - is_active=False, + is_active=active, is_superuser=False, is_external=self.connection.provision_as_external, # The identity provider vouches for the address; the person signs @@ -694,10 +889,20 @@ def _queue_user_event( body["adopted"] = adopted self.pending_events.append((topic, body)) - async def emit_pending_events(self) -> None: - """Send collected platform events. Call only after commit.""" + async def after_commit(self) -> None: + """Drop stale cache entries and send collected platform events. Call + only after the transaction commits, so neither can run ahead of it.""" + from shared.role_cache import invalidate_user + from src.core.cache import invalidate_role_users from src.services.events.builtins import emit_identity_provider_event + roles, self.stale_roles = self.stale_roles, set() + users, self.stale_users = self.stale_users, set() + for role_id in sorted(roles): + await invalidate_role_users(None, str(role_id)) + for user_id in sorted(users): + await invalidate_user(user_id) + events, self.pending_events = self.pending_events, [] for topic, body in events: await emit_identity_provider_event( diff --git a/api/tests/unit/scheduler/test_control.py b/api/tests/unit/scheduler/test_control.py index 8abaf000b7..63b9d5046b 100644 --- a/api/tests/unit/scheduler/test_control.py +++ b/api/tests/unit/scheduler/test_control.py @@ -91,6 +91,9 @@ def _control(lease: FakeLease, **kwargs) -> ControlPlane: # Added with identity provider connections: re-raises token-expiry # warnings, which live an hour as notifications. "identity_provider_token_expiry": ("interval", {"minutes": 60}, True), + # Applies provisioning grants in one transaction per run, so a SCIM sync + # burst changes the global authority epoch at most once a run. + "identity_provider_access_pass": ("interval", {"minutes": 2}, True), } diff --git a/api/tests/unit/services/test_identity_provider_access_pass.py b/api/tests/unit/services/test_identity_provider_access_pass.py new file mode 100644 index 0000000000..0eecd8c7a2 --- /dev/null +++ b/api/tests/unit/services/test_identity_provider_access_pass.py @@ -0,0 +1,340 @@ +"""Provisioning revokes at once and batches grants into the access pass. + +Every grant (activation, role, rename) changes the global authority epoch, +which retires every access token on the instance. These tests count epoch +changes as ``test_authz_epoch_triggers`` does: constraint triggers run +IMMEDIATE, the once-per-transaction flag is cleared before each simulated +request, and the count is this transaction's updates to ``authz_versions``. +""" + +from datetime import datetime, timezone +from uuid import uuid4 + +import pytest +import pytest_asyncio +from sqlalchemy import select, text + +from src.models.orm.identity_providers import ( + IdentityProvider, + IdentityProviderGroupMapping, + IdentityProviderGroupRole, + IdentityProviderUser, + OrganizationDomain, +) +from src.models.orm.organizations import Organization +from src.models.orm.users import Role, User, UserRole +from src.services.identity_provisioning.access_pass import run_access_pass +from src.services.identity_provisioning.core import GroupRecord, Provisioner, UserRecord + +_CLEAR_FLAG = text("SELECT set_config('bifrost.authz_epoch_bumped', '', true)") +_EPOCH_UPDATES = text( + "SELECT COALESCE((SELECT n_tup_upd FROM pg_stat_xact_user_tables " + "WHERE relname = 'authz_versions'), 0)" +) + + +class EpochCounter: + def __init__(self, session) -> None: + self.session = session + + async def start(self) -> None: + await self.session.execute(text("SET CONSTRAINTS ALL IMMEDIATE")) + self.baseline = await self._updates() + + async def request(self) -> None: + """Begin a simulated request: its own transaction may bump once.""" + await self.session.flush() + await self.session.execute(_CLEAR_FLAG) + + async def changes(self) -> int: + await self.session.flush() + return await self._updates() - self.baseline + + async def _updates(self) -> int: + return (await self.session.execute(_EPOCH_UPDATES)).scalar_one() + + +def _record(email: str, *, name: str = "Pat Example") -> UserRecord: + return UserRecord( + user_name=email, + email=email, + external_id=f"ext-{email}", + display_name=name, + active=True, + attributes={"userName": email}, + ) + + +@pytest_asyncio.fixture +async def setup(db_session): + suffix = uuid4().hex[:10] + domain = f"contoso-{suffix}.example" + organization = Organization(name=f"Contoso {suffix}", created_by="test") + db_session.add(organization) + await db_session.flush() + db_session.add( + OrganizationDomain( + organization_id=organization.id, + domain=domain, + verification_token="t", + verified_at=datetime.now(timezone.utc), + created_by="test", + ) + ) + connection = IdentityProvider( + organization_id=organization.id, + name="Contoso Entra", + source_type="scim", + vendor="entra", + created_by="test", + ) + role = Role(name=f"Support Tech {suffix}", created_by="test") + db_session.add_all([connection, role]) + await db_session.flush() + counter = EpochCounter(db_session) + await counter.start() + return { + "session": db_session, + "domain": domain, + "connection": connection, + "role": role, + "epoch": counter, + } + + +async def _mapped_group(setup): + session = setup["session"] + provisioner = Provisioner(session, setup["connection"]) + group = await provisioner.create_group( + GroupRecord(display_name="Techs", external_id=f"g-{uuid4().hex}", attributes={}), [] + ) + session.add(IdentityProviderGroupMapping(group_id=group.id, created_by="test")) + await session.flush() + session.add(IdentityProviderGroupRole(group_id=group.id, role_id=setup["role"].id)) + await session.flush() + return group + + +async def _refreshed(session, link: IdentityProviderUser) -> tuple[User, IdentityProviderUser]: + # Refresh the link first: a rolled-back savepoint expires it. + await session.refresh(link) + user = await session.get(User, link.user_id) + await session.refresh(user) + return user, link + + +async def _has_role(session, user_id, role_id) -> bool: + return ( + await session.scalar( + select(UserRole).where(UserRole.user_id == user_id, UserRole.role_id == role_id) + ) + ) is not None + + +@pytest.mark.asyncio +async def test_a_provisioning_burst_changes_the_epoch_once_in_the_pass(setup): + session, epoch, role = setup["session"], setup["epoch"], setup["role"] + group = await _mapped_group(setup) + links = [] + for index in range(5): + await epoch.request() + links.append( + await Provisioner(session, setup["connection"]).create_user( + _record(f"tech{index}@{setup['domain']}") + ) + ) + await epoch.request() + await Provisioner(session, setup["connection"]).set_group_members( + group, [link.id for link in links] + ) + + assert await epoch.changes() == 0 + for link in links: + user, link = await _refreshed(session, link) + assert not user.is_active and link.access_pending_since is not None + assert not await _has_role(session, user.id, role.id) + + await epoch.request() + result = await run_access_pass(session) + + assert result.failed == 0 and result.applied >= len(links) + assert await epoch.changes() == 1 + for link in links: + user, link = await _refreshed(session, link) + assert user.is_active and link.access_pending_since is None + assert await _has_role(session, user.id, role.id) + + +@pytest.mark.asyncio +async def test_access_changed_reports_the_grants_the_pass_applied(setup): + session, role = setup["session"], setup["role"] + group = await _mapped_group(setup) + provisioner = Provisioner(session, setup["connection"]) + link = await provisioner.create_user(_record(f"tech@{setup['domain']}")) + + await provisioner.set_group_members(group, [link.id]) + + topics = [topic for topic, _ in provisioner.pending_events] + assert "user.groups_changed" in topics and "user.access_changed" not in topics + + result = await run_access_pass(session) + + [body] = [ + body + for applied in result.provisioners + for topic, body in applied.pending_events + if topic == "user.access_changed" and body["user"]["id"] == str(link.user_id) + ] + assert body["access"] == { + "activated": True, + "roles_added": [str(role.id)], + "roles_removed": [], + } + + +@pytest.mark.asyncio +async def test_losing_the_mapped_group_revokes_in_the_request(setup): + session, epoch, role = setup["session"], setup["epoch"], setup["role"] + group = await _mapped_group(setup) + provisioner = Provisioner(session, setup["connection"]) + link = await provisioner.create_user(_record(f"leaver@{setup['domain']}")) + await provisioner.set_group_members(group, [link.id]) + await run_access_pass(session) + + await epoch.request() + await provisioner.set_group_members(group, []) + + user, link = await _refreshed(session, link) + assert not user.is_active + assert not await _has_role(session, user.id, role.id) + assert link.access_pending_since is None + + +@pytest.mark.asyncio +async def test_a_new_user_on_an_unmapped_connection_starts_active_without_a_bump(setup): + session, epoch = setup["session"], setup["epoch"] + + await epoch.request() + link = await Provisioner(session, setup["connection"]).create_user( + _record(f"solo@{setup['domain']}") + ) + + user, link = await _refreshed(session, link) + assert user.is_active and link.access_pending_since is None + assert await epoch.changes() == 0 + + +@pytest.mark.asyncio +async def test_a_rename_waits_for_the_pass(setup): + session = setup["session"] + provisioner = Provisioner(session, setup["connection"]) + email = f"pat@{setup['domain']}" + link = await provisioner.create_user(_record(email)) + + await provisioner.replace_user(link, _record(email, name="Pat Renamed")) + + user, link = await _refreshed(session, link) + assert user.name == "Pat Example" and link.access_pending_since is not None + await run_access_pass(session) + user, link = await _refreshed(session, link) + assert user.name == "Pat Renamed" and link.access_pending_since is None + + +@pytest.mark.asyncio +async def test_deleting_the_last_mapped_group_releases_users_outside_it(setup): + session = setup["session"] + group = await _mapped_group(setup) + provisioner = Provisioner(session, setup["connection"]) + # Created after the mapping, never in the group: inactive. + outsider = await provisioner.create_user(_record(f"outsider@{setup['domain']}")) + assert not (await _refreshed(session, outsider))[0].is_active + + # Deleting a group changes policy: its own exclusive-lock transaction. + await Provisioner(session, setup["connection"]).delete_group(group) + await run_access_pass(session) + + user, link = await _refreshed(session, outsider) + assert user.is_active and link.access_pending_since is None + + +@pytest.mark.asyncio +async def test_a_failing_link_rolls_back_alone_and_waits(setup, monkeypatch): + session, role = setup["session"], setup["role"] + group = await _mapped_group(setup) + provisioner = Provisioner(session, setup["connection"]) + broken = await provisioner.create_user(_record(f"broken@{setup['domain']}")) + fine = await provisioner.create_user(_record(f"fine@{setup['domain']}")) + await provisioner.set_group_members(group, [broken.id, fine.id]) + queued_at = (await _refreshed(session, broken))[1].access_pending_since + broken_user_id = broken.user_id + + audit = Provisioner._audit + + async def failing_audit(self, action, user, details, **kwargs): + # Fails after activating, granting and queueing the event. + if action == "identity_provider.user_activated" and user.id == broken_user_id: + raise RuntimeError("boom") + await audit(self, action, user, details, **kwargs) + + monkeypatch.setattr(Provisioner, "_audit", failing_audit) + result = await run_access_pass(session) + + assert result.failed == 1 + fine_user, fine = await _refreshed(session, fine) + assert fine_user.is_active and fine.access_pending_since is None + broken_user, broken = await _refreshed(session, broken) + assert not broken_user.is_active + assert not await _has_role(session, broken_user_id, role.id) + # Re-queued behind everything already waiting. + assert broken.access_pending_since is not None and broken.access_pending_since > queued_at + # Nothing from the rolled-back link leaks out after the commit. + [applied] = result.provisioners + assert broken_user_id not in applied.stale_users + assert all(body["user"]["id"] != str(broken_user_id) for _, body in applied.pending_events) + + +@pytest.mark.asyncio +async def test_the_oldest_grant_goes_first_across_connections(setup, monkeypatch): + import src.services.identity_provisioning.access_pass as access_pass + + session = setup["session"] + other = IdentityProvider( + organization_id=setup["connection"].organization_id, + name="Contoso Okta", + source_type="scim", + vendor="okta", + created_by="test", + ) + session.add(other) + await session.flush() + older = await Provisioner(session, other).create_user(_record(f"older@{setup['domain']}")) + newer = await Provisioner(session, setup["connection"]).create_user( + _record(f"newer@{setup['domain']}") + ) + for link, renamed in ((older, "Older Renamed"), (newer, "Newer Renamed")): + connection = other if link is older else setup["connection"] + await Provisioner(session, connection).replace_user( + link, _record(link.user_name, name=renamed) + ) + monkeypatch.setattr(access_pass, "BATCH_LIMIT", 1) + + await run_access_pass(session) + + assert (await _refreshed(session, older))[1].access_pending_since is None + assert (await _refreshed(session, newer))[1].access_pending_since is not None + + +@pytest.mark.asyncio +async def test_a_disabled_connections_grants_wait(setup): + session, connection = setup["session"], setup["connection"] + group = await _mapped_group(setup) + provisioner = Provisioner(session, connection) + link = await provisioner.create_user(_record(f"paused@{setup['domain']}")) + await provisioner.set_group_members(group, [link.id]) + connection.is_enabled = False + await session.flush() + + await run_access_pass(session) + + user, link = await _refreshed(session, link) + assert not user.is_active and link.access_pending_since is not None diff --git a/api/tests/unit/services/test_identity_provisioning_core.py b/api/tests/unit/services/test_identity_provisioning_core.py index 7458ed2f9a..22bd535e83 100644 --- a/api/tests/unit/services/test_identity_provisioning_core.py +++ b/api/tests/unit/services/test_identity_provisioning_core.py @@ -87,6 +87,14 @@ async def _map(session, group_id, role_ids=()): await session.flush() +async def _map_and_apply(session, connection, group_id, role_ids): + """What the mapping service does: exclusive policy lock, then apply.""" + mapper = Provisioner(session, connection) + await mapper.hold_policy(exclusive=True) + await _map(session, group_id, role_ids) + await mapper.reconcile_all() + + @pytest.mark.asyncio async def test_creates_an_active_verified_user_in_the_connection_org(setup): provisioner = Provisioner(setup["session"], setup["connection"]) @@ -145,8 +153,7 @@ async def test_mapped_group_grants_tagged_roles_and_removing_it_revokes_access(s group = await provisioner.create_group( GroupRecord(display_name="Techs", external_id="g1", attributes={}), [link.id] ) - await _map(session, group.id, [role.id]) - await provisioner.reconcile_all() + await _map_and_apply(session, setup["connection"], group.id, [role.id]) assignment = await session.scalar( select(UserRole).where(UserRole.user_id == link.user_id, UserRole.role_id == role.id) @@ -176,10 +183,10 @@ async def test_a_manual_grant_of_a_mapped_role_survives_unmapping(setup): group = await provisioner.create_group( GroupRecord(display_name="Techs", external_id="g1", attributes={}), [link.id] ) - await _map(session, group.id, [role.id]) - await provisioner.reconcile_all() + await _map_and_apply(session, setup["connection"], group.id, [role.id]) - await provisioner.delete_group(group) + # Deleting a group changes policy: its own exclusive-lock transaction. + await Provisioner(session, setup["connection"]).delete_group(group) manual = await session.scalar( select(UserRole).where(UserRole.user_id == link.user_id, UserRole.role_id == role.id) diff --git a/docs/plans/2026-11-16-scim-identity-providers.md b/docs/plans/2026-11-16-scim-identity-providers.md index b956078322..579df5934b 100644 --- a/docs/plans/2026-11-16-scim-identity-providers.md +++ b/docs/plans/2026-11-16-scim-identity-providers.md @@ -136,8 +136,57 @@ runs inside the request transaction and: union of mapped-group roles. It inserts missing roles, tagged with the connection, and deletes tagged roles that are no longer wanted. Active = `link.active` and (no mappings, or at least one mapped group). + **Revoke now, grant later** (see below). 5. Emits audit and events after the commit. +### Batched grants (the access pass) + +Every role or activation change replaces the single global authority epoch, +which retires every access token on the instance. Each SCIM request is its +own transaction, so a 2,000-user initial sync applied request by request +would do that about 2,000 times in 40 minutes — far past the authority +runbook's 30-an-hour threshold, and fast enough that a running workflow's +three refresh-and-replay attempts could all lose the race. + +- **Revocations apply in the request:** deactivation (`active=false`, + DELETE, losing the last mapped group, with sessions revoked) and removing + roles the connection granted. +- **Grants wait:** activation, new role assignments and display-name changes + are recorded on the link (`access_pending_since`, with the provider's + `display_name`). The SCIM resource the provider reads back comes from the + link, so it sees no difference. +- **The access pass** (`identity_provider_access_pass`, every 2 minutes) + applies up to 500 waiting links, oldest first, across all connections in + one transaction, skipping users a request holds (`SKIP LOCKED`). Each link + runs in its own savepoint; one that fails rolls back alone and goes to the + back of the queue. Caches are invalidated and events sent after the + commit. Grants cost at most one epoch change per run (30 an hour); + revocations, email changes and deletions still change it per request, so + they count separately against the runbook's threshold. +- **Immediate where an admin acts:** a mapping change in the UI applies + grants in its own single transaction. +- New users are inserted in their final state. An insert doesn't change the + epoch, so on a connection without mappings, creating a user costs nothing. +- Email changes stay in the request: they need the synchronous uniqueness + and domain checks, and they are rare. +- A grant refused by policy (e.g. a privileged role mapped for an External + user) is skipped and audited (`identity_provider.role_grant_refused`); the + next mapping or directory change for that user tries again. +- **Locking.** Every provisioning write takes a per-connection policy lock + (a transaction-scoped advisory lock, shared); a mapping change takes it + exclusively. A user created while the first mapping is added therefore + can't be missed. After that lock, users are locked in id order, and links, + memberships and assignments are read fresh. Membership changes to one + group also lock the group row. Deleting the last mapped group reconciles + every linked user, since the mapped-group requirement lifts for all of + them. +- **Events.** `user.groups_changed` reports the membership change only. + `user.access_changed` reports applied access (`activated`, `roles_added`, + `roles_removed`) when the request revokes and when the pass grants. + +The lasting fix is per-subject epochs (the `authz_versions` table is already +keyed by subject); that belongs to the upstream RFC. + The Google Directory sync (phase 4) calls the same commands with a full snapshot. It adds snapshot-only guards: never deprovision after a failed or partial fetch, and refuse a sync that would remove an unusually large share of @@ -198,9 +247,10 @@ Libraries: ## Events -Topics `user.provisioned`, `user.updated`, `user.deprovisioned` and -`user.groups_changed`, registered in `CURATED_TOPICS` with the standard -`_base_body` envelope. The actor is the connection. +Topics `user.provisioned`, `user.updated`, `user.deprovisioned`, +`user.groups_changed` and `user.access_changed`, registered in +`CURATED_TOPICS` with the standard `_base_body` envelope. The actor is the +connection. ## UI @@ -339,6 +389,12 @@ To keep these options open: ## Open questions (for the RFC) +- Per-subject authority epochs. Provisioning is the workload most likely to + cross the runbook's 30-bumps-an-hour threshold; the access pass caps it in + v1, at the cost of grants waiting up to two minutes. A per-user version + (or skipping the bump for users who hold no credentials) would let grants + apply in the request again. + - Deleting a connection: the proposal is to unlink its users (they become Native) and convert the roles it granted to manual. The alternative is to require disabling or removing the users first. From 826e99a33bf7db89bcc01d7128b1d59be1b3cd27 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 11:50:24 -0500 Subject: [PATCH 09/26] Lock members' users before changing group membership set_group_members deleted membership rows before locking the members' users, while delete_user locks the user first and then cascades into those rows: two concurrent requests could deadlock. Lock the affected users first, in id order, then re-read the member list. Found by GPT-6.1 Sol's verification review. Co-Authored-By: Claude Opus 5.5 --- .../services/identity_provisioning/core.py | 28 ++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/api/src/services/identity_provisioning/core.py b/api/src/services/identity_provisioning/core.py index 3ce2b6bd3f..03c8fde082 100644 --- a/api/src/services/identity_provisioning/core.py +++ b/api/src/services/identity_provisioning/core.py @@ -375,7 +375,16 @@ async def set_group_members( .where(IdentityProviderGroup.id == group.id) .with_for_update() ) - wanted = await self._known_links(set(member_link_ids)) + requested = set(member_link_ids) + current = await self.group_member_ids(group.id) + wanted = await self._known_links(requested) + # Lock the affected users before touching membership rows, the order + # delete_user follows (user, then the link and its memberships), so + # the two can't deadlock. Then read again: a deletion that held one + # of those locks has committed by now, and a member it removed is + # refused as unknown rather than written. + await self._lock_users_of(wanted ^ current) + wanted = await self._known_links(requested) current = await self.group_member_ids(group.id) added, removed = wanted - current, current - wanted if removed: @@ -699,6 +708,23 @@ async def _lock_user(self, user_id: UUID) -> User: assert user is not None return user + async def _lock_users_of(self, link_ids: set[UUID]) -> None: + """Lock the users behind these links, in id order.""" + if not link_ids: + return + await self.session.execute( + select(User.id) + .where( + User.id.in_( + select(IdentityProviderUser.user_id).where( + IdentityProviderUser.id.in_(link_ids) + ) + ) + ) + .order_by(User.id) + .with_for_update() + ) + def _mark_stale(self, user_id: UUID, role_ids: set[UUID]) -> None: """Queue cache invalidation for after the commit. Each role's invalidation scans Redis, so a burst touching one role scans once.""" From 2cd899b3f69a2b6632e58ec452d63c99fa9c51e1 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 12:17:11 -0500 Subject: [PATCH 10/26] Answer deadlock-aborted requests with an explicit error PostgreSQL breaks the rare remaining deadlocks (the access pass and a mapping change on another connection granting one role to External users, or two mapping changes) by rolling one transaction back. That surfaced as a generic 500 or an exception log. Now each surface classifies the abort by SQLSTATE (40P01, 40001, via a new is_transaction_conflict shared with the database retry helper), logs a warning and answers explicitly: - SCIM: 503 with Retry-After: 5, which identity providers retry. - Management API: 409 "nothing was saved; try again". - Access pass: a link rolled back counts as a conflict and waits for the next run; a whole-run abort is logged and left to the next run. A test drives a real deadlock between two sessions to prove the classifier matches the error asyncpg raises. Co-Authored-By: Claude Opus 5.5 --- api/src/core/database_retry.py | 11 ++ api/src/core/transaction_conflicts.py | 48 +++++++ .../identity_provider_access_pass.py | 20 ++- api/src/routers/identity_providers.py | 17 ++- api/src/routers/scim.py | 18 ++- .../identity_provisioning/access_pass.py | 31 +++-- .../services/identity_provisioning/scim.py | 11 +- .../unit/core/test_transaction_conflicts.py | 120 ++++++++++++++++++ .../test_identity_provider_access_pass.py | 22 ++++ .../2026-11-16-scim-identity-providers.md | 9 ++ 10 files changed, 291 insertions(+), 16 deletions(-) create mode 100644 api/src/core/transaction_conflicts.py create mode 100644 api/tests/unit/core/test_transaction_conflicts.py diff --git a/api/src/core/database_retry.py b/api/src/core/database_retry.py index 97f690aeca..1d6dcf9dd4 100644 --- a/api/src/core/database_retry.py +++ b/api/src/core/database_retry.py @@ -84,6 +84,17 @@ def deadline(self) -> float: return self._deadline +def is_transaction_conflict(exc: BaseException) -> bool: + """Whether PostgreSQL aborted the transaction to let a conflicting one + finish (deadlock_detected or serialization_failure). Nothing the + transaction wrote was kept, and running it again can succeed.""" + if isinstance(exc, DBAPIError): + if exc.orig is None: + return False + exc = exc.orig + return getattr(exc, "sqlstate", None) in _CONFLICT_SQLSTATES + + def is_retryable_database_error(exc: BaseException) -> bool: """Whether running the same unit of work again can succeed. diff --git a/api/src/core/transaction_conflicts.py b/api/src/core/transaction_conflicts.py new file mode 100644 index 0000000000..d5fbae8f91 --- /dev/null +++ b/api/src/core/transaction_conflicts.py @@ -0,0 +1,48 @@ +"""Answer a request PostgreSQL aborted to break a deadlock with a clear error. + +When two transactions wait on each other, PostgreSQL detects the cycle +(after ``deadlock_timeout``, one second by default) and aborts one of them +with ``deadlock_detected``. Nothing that transaction wrote is kept, so the +right answer is "nothing was saved; try again" rather than a generic 500. +""" + +from __future__ import annotations + +import logging +from collections.abc import Callable +from typing import Any + +from fastapi import Request, Response +from fastapi.routing import APIRoute + +from src.core.database_retry import is_transaction_conflict + +logger = logging.getLogger(__name__) + + +def conflict_aware_route(on_conflict: Callable[[], Exception]) -> type[APIRoute]: + """An ``APIRoute`` class that turns a transaction conflict into the + exception ``on_conflict`` builds (handled by the app's exception + handlers), and logs it, so the rate of these is visible.""" + + class ConflictAwareRoute(APIRoute): + def get_route_handler(self) -> Callable[[Request], Any]: + handler = super().get_route_handler() + + async def route(request: Request) -> Response: + try: + return await handler(request) + except Exception as exc: + if not is_transaction_conflict(exc): + raise + logger.warning( + "Transaction conflict on %s %s; nothing was saved: %s", + request.method, + request.url.path, + exc, + ) + raise on_conflict() from None + + return route + + return ConflictAwareRoute diff --git a/api/src/jobs/schedulers/identity_provider_access_pass.py b/api/src/jobs/schedulers/identity_provider_access_pass.py index 4d1f43c7cb..de88b543f7 100644 --- a/api/src/jobs/schedulers/identity_provider_access_pass.py +++ b/api/src/jobs/schedulers/identity_provider_access_pass.py @@ -3,6 +3,7 @@ import logging from src.core.database import get_session_factory +from src.core.database_retry import is_transaction_conflict from src.services.identity_provisioning.access_pass import run_access_pass logger = logging.getLogger(__name__) @@ -11,12 +12,23 @@ async def run_identity_provider_access_pass() -> dict: """Apply the activations, role grants and renames provisioning deferred.""" session_factory = get_session_factory() - async with session_factory() as db: - result = await run_access_pass(db) - await db.commit() + try: + async with session_factory() as db: + result = await run_access_pass(db) + await db.commit() + except Exception as exc: + if not is_transaction_conflict(exc): + raise + # A deadlock outside a link's savepoint (taking the run's locks, or + # at commit) rolls back the whole run; every link still waits. + logger.warning( + "Identity provider access pass was rolled back to break a deadlock; " + "the next run applies its links" + ) + return {"applied": 0, "failed": 0, "conflicts": 1} for provisioner in result.provisioners: await provisioner.after_commit() summary = result.summary() - if result.applied or result.failed: + if result.applied or result.failed or result.conflicts: logger.info("Identity provider access pass: %s", summary) return summary diff --git a/api/src/routers/identity_providers.py b/api/src/routers/identity_providers.py index 9c03db461c..6865a19a70 100644 --- a/api/src/routers/identity_providers.py +++ b/api/src/routers/identity_providers.py @@ -5,10 +5,11 @@ from uuid import UUID -from fastapi import APIRouter, Query, Response +from fastapi import APIRouter, HTTPException, Query, Response from src.core.auth import CurrentActiveUser from src.core.db_deps import DbSession +from src.core.transaction_conflicts import conflict_aware_route from src.models.contracts.identity_providers import ( IdentityProviderCreate, IdentityProviderCreated, @@ -26,7 +27,19 @@ from src.services.identity_provisioning.expiry import clear_expiry_notifications from src.services.operation_catalog import operation_route -router = APIRouter(prefix="/api/identity-providers", tags=["Identity Providers"]) +def _conflict() -> HTTPException: + return HTTPException( + 409, + "A concurrent change to the same users or roles was in progress, so this one was " + "cancelled and nothing was saved. Try again.", + ) + + +router = APIRouter( + prefix="/api/identity-providers", + tags=["Identity Providers"], + route_class=conflict_aware_route(_conflict), +) # --------------------------------------------------------------------------- diff --git a/api/src/routers/scim.py b/api/src/routers/scim.py index feccf08628..a76a6f62b7 100644 --- a/api/src/routers/scim.py +++ b/api/src/routers/scim.py @@ -19,6 +19,7 @@ from src.core.db_deps import DbSession from src.core.rate_limit import RateLimiter +from src.core.transaction_conflicts import conflict_aware_route from src.models.orm.identity_providers import ( IdentityProvider, IdentityProviderGroup, @@ -29,10 +30,21 @@ from src.services.identity_provisioning.core import ProvisioningError, Provisioner from src.services.identity_provisioning.scim import ScimError +def _conflict() -> ScimError: + # Identity providers retry a 503; the request changed nothing. + return ScimError( + 503, + "A concurrent change to the same users was in progress, so this request was " + "cancelled and nothing was saved. Retry it.", + retry_after=5, + ) + + router = APIRouter( prefix="/api/scim/v2/{identity_provider_id}", tags=["SCIM"], include_in_schema=False, + route_class=conflict_aware_route(_conflict), ) # Generous: a large Entra or Okta cycle sends bursts of user and membership @@ -49,7 +61,11 @@ def scim_response(body: Any, status_code: int = 200, *, location: str | None = N def scim_error_response(exc: ScimError) -> JSONResponse: - headers = {"WWW-Authenticate": 'Bearer realm="SCIM"'} if exc.status == 401 else None + headers: dict[str, str] = {} + if exc.status == 401: + headers["WWW-Authenticate"] = 'Bearer realm="SCIM"' + if exc.retry_after is not None: + headers["Retry-After"] = str(exc.retry_after) return JSONResponse( exc.body(), status_code=exc.status, media_type=scim.SCIM_MEDIA_TYPE, headers=headers ) diff --git a/api/src/services/identity_provisioning/access_pass.py b/api/src/services/identity_provisioning/access_pass.py index acdb3f8445..82329107c2 100644 --- a/api/src/services/identity_provisioning/access_pass.py +++ b/api/src/services/identity_provisioning/access_pass.py @@ -29,6 +29,7 @@ from sqlalchemy import Select, select, update from sqlalchemy.ext.asyncio import AsyncSession +from src.core.database_retry import is_transaction_conflict from src.models.orm.identity_providers import IdentityProvider, IdentityProviderUser from src.models.orm.organizations import Organization from src.models.orm.users import User @@ -46,11 +47,14 @@ class AccessPassResult: applied: int = 0 failed: int = 0 + # Links PostgreSQL rolled back to break a deadlock; they wait for the + # next run. + conflicts: int = 0 # Provisioners to run ``after_commit`` on once the caller commits. provisioners: list[Provisioner] = field(default_factory=list) def summary(self) -> dict[str, int]: - return {"applied": self.applied, "failed": self.failed} + return {"applied": self.applied, "failed": self.failed, "conflicts": self.conflicts} def _waiting(*columns: Any) -> Select[Any]: @@ -116,15 +120,26 @@ async def run_access_pass(session: AsyncSession) -> AccessPassResult: try: async with session.begin_nested(): await provisioner.apply_pending([link_id]) - except Exception: + except Exception as exc: del provisioner.pending_events[queued:] provisioner.stale_roles, provisioner.stale_users = stale_roles, stale_users - result.failed += 1 - logger.exception( - "Identity provider %s could not apply access for link %s", - connection_id, - link_id, - ) + if is_transaction_conflict(exc): + # PostgreSQL broke a deadlock (e.g. with a mapping change + # granting the same role) by rolling this link back. + result.conflicts += 1 + logger.warning( + "Identity provider %s access for link %s was rolled back to break a " + "deadlock; it waits for the next run", + connection_id, + link_id, + ) + else: + result.failed += 1 + logger.exception( + "Identity provider %s could not apply access for link %s", + connection_id, + link_id, + ) await session.execute( update(IdentityProviderUser) .where(IdentityProviderUser.id == link_id) diff --git a/api/src/services/identity_provisioning/scim.py b/api/src/services/identity_provisioning/scim.py index 0ee403ac30..53638cdbd7 100644 --- a/api/src/services/identity_provisioning/scim.py +++ b/api/src/services/identity_provisioning/scim.py @@ -66,11 +66,20 @@ class ScimError(Exception): """An RFC 7644 §3.12 error response.""" - def __init__(self, status: int, detail: str, scim_type: str | None = None): + def __init__( + self, + status: int, + detail: str, + scim_type: str | None = None, + *, + retry_after: int | None = None, + ): super().__init__(detail) self.status = status self.detail = detail self.scim_type = scim_type + # Seconds, sent as Retry-After. + self.retry_after = retry_after def body(self) -> dict[str, Any]: body: dict[str, Any] = { diff --git a/api/tests/unit/core/test_transaction_conflicts.py b/api/tests/unit/core/test_transaction_conflicts.py new file mode 100644 index 0000000000..29b99de9dd --- /dev/null +++ b/api/tests/unit/core/test_transaction_conflicts.py @@ -0,0 +1,120 @@ +"""A transaction PostgreSQL aborts to break a deadlock answers with a clear, +explicit error instead of a generic 500.""" + +from __future__ import annotations + +import asyncio +import secrets + +import asyncpg +import httpx +import pytest +from fastapi import APIRouter, FastAPI, HTTPException +from sqlalchemy import text +from sqlalchemy.exc import DBAPIError, IntegrityError + +from src.core.database_retry import is_transaction_conflict +from src.core.transaction_conflicts import conflict_aware_route + +DEADLOCK = DBAPIError("UPDATE", {}, asyncpg.exceptions.DeadlockDetectedError("deadlock detected")) + + +@pytest.mark.parametrize( + ("exc", "conflict"), + [ + (DEADLOCK, True), + (DBAPIError("UPDATE", {}, asyncpg.exceptions.SerializationError("could not serialize")), True), + (asyncpg.exceptions.DeadlockDetectedError("deadlock detected"), True), + (IntegrityError("INSERT", {}, asyncpg.exceptions.UniqueViolationError("dup")), False), + (DBAPIError("SELECT 1", {}, asyncpg.exceptions.AdminShutdownError("shutdown")), False), + (ValueError("bad"), False), + ], +) +def test_only_an_aborted_conflict_is_a_transaction_conflict(exc, conflict): + assert is_transaction_conflict(exc) is conflict + + +@pytest.mark.asyncio +async def test_a_real_deadlock_is_recognized(async_session_factory): + """Two transactions take two locks in opposite order; PostgreSQL aborts + one, and the error the driver raises is classified as a conflict.""" + first, second = secrets.randbits(62), secrets.randbits(62) + holding = [asyncio.Event(), asyncio.Event()] + + async def transaction(index: int, keys: tuple[int, int]) -> None: + async with async_session_factory() as session: + async with session.begin(): + await session.execute(text("SELECT pg_advisory_xact_lock(:key)"), {"key": keys[0]}) + holding[index].set() + await holding[1 - index].wait() + await session.execute(text("SELECT pg_advisory_xact_lock(:key)"), {"key": keys[1]}) + + async with asyncio.timeout(30): + results = await asyncio.gather( + transaction(0, (first, second)), + transaction(1, (second, first)), + return_exceptions=True, + ) + + errors = [result for result in results if isinstance(result, BaseException)] + assert len(errors) == 1 + assert is_transaction_conflict(errors[0]) + + +def _app(raised: Exception) -> FastAPI: + router = APIRouter( + route_class=conflict_aware_route(lambda: HTTPException(409, "Nothing was saved; try again.")) + ) + + @router.post("/change") + async def change() -> None: + raise raised + + app = FastAPI() + app.include_router(router) + return app + + +async def _post(app: FastAPI) -> httpx.Response: + transport = httpx.ASGITransport(app=app, raise_app_exceptions=False) + async with httpx.AsyncClient(transport=transport, base_url="http://test") as client: + return await client.post("/change") + + +@pytest.mark.asyncio +async def test_a_conflict_becomes_the_routes_explicit_error(): + response = await _post(_app(DEADLOCK)) + + assert response.status_code == 409 + assert response.json() == {"detail": "Nothing was saved; try again."} + + +@pytest.mark.asyncio +async def test_other_errors_pass_through_unchanged(): + response = await _post(_app(IntegrityError("INSERT", {}, Exception("dup")))) + + assert response.status_code == 500 + + +@pytest.mark.asyncio +async def test_a_scim_conflict_asks_the_identity_provider_to_retry(): + from src.routers.scim import _conflict, scim_error_response + from src.services.identity_provisioning.scim import ScimError + + router = APIRouter(route_class=conflict_aware_route(_conflict)) + + @router.post("/change") + async def change() -> None: + raise DEADLOCK + + app = FastAPI() + app.include_router(router) + app.add_exception_handler(ScimError, lambda _request, exc: scim_error_response(exc)) # type: ignore[arg-type] + + response = await _post(app) + + assert response.status_code == 503 + assert response.headers["retry-after"] == "5" + body = response.json() + assert body["schemas"] == ["urn:ietf:params:scim:api:messages:2.0:Error"] + assert body["status"] == "503" diff --git a/api/tests/unit/services/test_identity_provider_access_pass.py b/api/tests/unit/services/test_identity_provider_access_pass.py index 0eecd8c7a2..abace8f6c3 100644 --- a/api/tests/unit/services/test_identity_provider_access_pass.py +++ b/api/tests/unit/services/test_identity_provider_access_pass.py @@ -293,6 +293,28 @@ async def failing_audit(self, action, user, details, **kwargs): assert all(body["user"]["id"] != str(broken_user_id) for _, body in applied.pending_events) +@pytest.mark.asyncio +async def test_a_deadlocked_link_counts_as_a_conflict_and_waits(setup, monkeypatch): + import asyncpg + from sqlalchemy.exc import DBAPIError + + session = setup["session"] + group = await _mapped_group(setup) + provisioner = Provisioner(session, setup["connection"]) + link = await provisioner.create_user(_record(f"raced@{setup['domain']}")) + await provisioner.set_group_members(group, [link.id]) + + async def deadlocked(self, link_ids): + raise DBAPIError("INSERT", {}, asyncpg.exceptions.DeadlockDetectedError("deadlock")) + + monkeypatch.setattr(Provisioner, "apply_pending", deadlocked) + result = await run_access_pass(session) + + assert result.summary() == {"applied": 0, "failed": 0, "conflicts": 1} + user, link = await _refreshed(session, link) + assert not user.is_active and link.access_pending_since is not None + + @pytest.mark.asyncio async def test_the_oldest_grant_goes_first_across_connections(setup, monkeypatch): import src.services.identity_provisioning.access_pass as access_pass diff --git a/docs/plans/2026-11-16-scim-identity-providers.md b/docs/plans/2026-11-16-scim-identity-providers.md index 579df5934b..e142e25630 100644 --- a/docs/plans/2026-11-16-scim-identity-providers.md +++ b/docs/plans/2026-11-16-scim-identity-providers.md @@ -180,6 +180,15 @@ three refresh-and-replay attempts could all lose the race. group also lock the group row. Deleting the last mapped group reconciles every linked user, since the mapped-group requirement lifts for all of them. +- **Deadlocks that remain possible** (a pass and a mapping change on + another connection both granting one role to External users, or two + mapping changes) are broken by PostgreSQL, which rolls one transaction + back. Each surface answers explicitly and logs a warning: SCIM returns + 503 with `Retry-After: 5` (identity providers retry it), the management + API returns 409 "nothing was saved; try again", and the pass counts a + `conflicts` entry and leaves the link waiting for the next run. + Classification is by SQLSTATE (`40P01`, `40001`), shared with the + database retry helper. - **Events.** `user.groups_changed` reports the membership change only. `user.access_changed` reports applied access (`activated`, `roles_added`, `roles_removed`) when the request revokes and when the pass grants. From 32acf060495b1eda2a5c12fc515305d6ec389d08 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 12:31:19 -0500 Subject: [PATCH 11/26] Show when identity provider access is waiting to apply Grants from an identity provider now wait up to two minutes for the access pass, so a user Entra just added can look inactive with no explanation. The user's identity source carries access_pending_since (Users list: an "Access Pending" badge with the reason), and a connection reports pending_access_count (provisioning card: "N awaiting access"). Co-Authored-By: Claude Opus 5.5 --- api/shared/sdk_users.py | 7 +-- .../models/contracts/identity_providers.py | 5 ++ api/src/services/identity_providers.py | 27 +++++++++-- .../test_identity_provider_access_pass.py | 19 ++++++++ .../IdentityProviderBadges.test.tsx | 48 +++++++++++++++++++ .../IdentityProviderBadges.tsx | 29 +++++++++++ .../identity-providers/ProvisioningCard.tsx | 8 ++++ client/src/lib/v1.d.ts | 4 ++ .../src/pages/IdentityProviderDetail.test.tsx | 1 + client/src/pages/IdentityProviders.test.tsx | 1 + client/src/pages/Users.tsx | 6 +-- 11 files changed, 143 insertions(+), 12 deletions(-) create mode 100644 client/src/components/identity-providers/IdentityProviderBadges.test.tsx diff --git a/api/shared/sdk_users.py b/api/shared/sdk_users.py index 5bb03e163a..41f3b95d49 100644 --- a/api/shared/sdk_users.py +++ b/api/shared/sdk_users.py @@ -235,7 +235,6 @@ async def _with_protection(session: AsyncSession, users: list) -> list["UserPubl from src.models import UserPublic from src.services.authorization.enforce import privileged_user_ids - from src.models.contracts.identity_providers import IdentitySource from src.services.identity_providers import identity_sources user_ids = [u.id for u in users] @@ -245,11 +244,7 @@ async def _with_protection(session: AsyncSession, users: list) -> list["UserPubl for u in users: public = UserPublic.model_validate(u) public.is_protected = u.id in privileged - if u.id in sources: - provider_id, name, vendor = sources[u.id] - public.identity_source = IdentitySource( - identity_provider_id=provider_id, name=name, vendor=vendor # type: ignore[arg-type] - ) + public.identity_source = sources.get(u.id) out.append(public) return out diff --git a/api/src/models/contracts/identity_providers.py b/api/src/models/contracts/identity_providers.py index 98f3bd9378..0840c888c5 100644 --- a/api/src/models/contracts/identity_providers.py +++ b/api/src/models/contracts/identity_providers.py @@ -38,6 +38,9 @@ class IdentitySource(BaseModel): identity_provider_id: UUID name: str vendor: IdentityProviderVendor + # Set while an activation, role grant or rename from the identity + # provider waits for the access pass (applied within about two minutes). + access_pending_since: datetime | None = None class IdentityProviderTokenInfo(BaseModel): @@ -79,6 +82,8 @@ class IdentityProviderResponse(BaseModel): user_count: int group_count: int mapped_group_count: int + # Users whose activation, role grant or rename waits for the access pass. + pending_access_count: int last_request_at: datetime | None last_sync_at: datetime | None created_at: datetime diff --git a/api/src/services/identity_providers.py b/api/src/services/identity_providers.py index 17bf9a119f..0b6178156c 100644 --- a/api/src/services/identity_providers.py +++ b/api/src/services/identity_providers.py @@ -24,6 +24,7 @@ IdentityProviderTokenInfo, IdentityProviderTokenIssued, IdentityProviderUpdate, + IdentitySource, OrganizationDomainResponse, MappedRole, ) @@ -215,6 +216,16 @@ async def _responses( .group_by(IdentityProviderGroup.identity_provider_id) )).all() ) + pending_counts = dict( + (await session.execute( + select(IdentityProviderUser.identity_provider_id, func.count()) + .where( + IdentityProviderUser.identity_provider_id.in_(ids), + IdentityProviderUser.access_pending_since.is_not(None), + ) + .group_by(IdentityProviderUser.identity_provider_id) + )).all() + ) current_tokens = {c.id: await tokens.current_token(session, c.id) for c in connections} return [ IdentityProviderResponse( @@ -232,6 +243,7 @@ async def _responses( user_count=user_counts.get(c.id, 0), group_count=group_counts.get(c.id, 0), mapped_group_count=mapped_counts.get(c.id, 0), + pending_access_count=pending_counts.get(c.id, 0), last_request_at=c.last_request_at, last_sync_at=c.last_sync_at, created_at=c.created_at, @@ -565,8 +577,8 @@ async def delete_group_mapping( async def identity_sources( session: AsyncSession, user_ids: list[UUID] -) -> dict[UUID, tuple[UUID, str, str]]: - """``user_id -> (identity_provider_id, name, vendor)`` for linked users.""" +) -> dict[UUID, IdentitySource]: + """``user_id -> IdentitySource`` for linked users.""" if not user_ids: return {} rows = await session.execute( @@ -575,9 +587,18 @@ async def identity_sources( IdentityProvider.id, IdentityProvider.name, IdentityProvider.vendor, + IdentityProviderUser.access_pending_since, ) .join(IdentityProvider, IdentityProvider.id == IdentityProviderUser.identity_provider_id) .where(IdentityProviderUser.user_id.in_(user_ids)) ) - return {user_id: (idp_id, name, vendor) for user_id, idp_id, name, vendor in rows.all()} + return { + user_id: IdentitySource( + identity_provider_id=idp_id, + name=name, + vendor=vendor, + access_pending_since=pending_since, + ) + for user_id, idp_id, name, vendor, pending_since in rows.all() + } diff --git a/api/tests/unit/services/test_identity_provider_access_pass.py b/api/tests/unit/services/test_identity_provider_access_pass.py index abace8f6c3..8f65f30198 100644 --- a/api/tests/unit/services/test_identity_provider_access_pass.py +++ b/api/tests/unit/services/test_identity_provider_access_pass.py @@ -346,6 +346,25 @@ async def test_the_oldest_grant_goes_first_across_connections(setup, monkeypatch assert (await _refreshed(session, newer))[1].access_pending_since is not None +@pytest.mark.asyncio +async def test_a_waiting_grant_shows_on_the_users_identity_source(setup): + from src.services.identity_providers import identity_sources + + session = setup["session"] + group = await _mapped_group(setup) + provisioner = Provisioner(session, setup["connection"]) + link = await provisioner.create_user(_record(f"waiting@{setup['domain']}")) + await provisioner.set_group_members(group, [link.id]) + + source = (await identity_sources(session, [link.user_id]))[link.user_id] + assert source.name == "Contoso Entra" and source.access_pending_since is not None + + await run_access_pass(session) + + source = (await identity_sources(session, [link.user_id]))[link.user_id] + assert source.access_pending_since is None + + @pytest.mark.asyncio async def test_a_disabled_connections_grants_wait(setup): session, connection = setup["session"], setup["connection"] diff --git a/client/src/components/identity-providers/IdentityProviderBadges.test.tsx b/client/src/components/identity-providers/IdentityProviderBadges.test.tsx new file mode 100644 index 0000000000..dc02c6888c --- /dev/null +++ b/client/src/components/identity-providers/IdentityProviderBadges.test.tsx @@ -0,0 +1,48 @@ +import { describe, expect, it } from "vitest"; + +import { renderWithProviders, screen } from "@/test-utils"; + +import { IdentitySourceLabel } from "./IdentityProviderBadges"; + +const source = { + identity_provider_id: "4f0c7a0e-2a3b-4c5d-8e9f-0a1b2c3d4e5f", + name: "Contoso Entra ID", + vendor: "entra" as const, +}; + +describe("IdentitySourceLabel", () => { + it("names a user Bifrost manages Native", () => { + renderWithProviders(); + + expect(screen.getByText("Native")).toBeInTheDocument(); + }); + + it("names the connection that manages a user", () => { + renderWithProviders( + , + ); + + expect(screen.getByText("Contoso Entra ID")).toBeInTheDocument(); + expect(screen.queryByText("Access Pending")).not.toBeInTheDocument(); + }); + + it("says when access from the identity provider is waiting to apply", () => { + renderWithProviders( + , + ); + + expect(screen.getByText("Access Pending")).toHaveAttribute( + "title", + "Changes from Contoso Entra ID apply within about two minutes", + ); + }); +}); diff --git a/client/src/components/identity-providers/IdentityProviderBadges.tsx b/client/src/components/identity-providers/IdentityProviderBadges.tsx index c111ca0dde..00ea060ecf 100644 --- a/client/src/components/identity-providers/IdentityProviderBadges.tsx +++ b/client/src/components/identity-providers/IdentityProviderBadges.tsx @@ -2,11 +2,40 @@ import { CheckCircle2, CircleSlash, Clock, XCircle } from "lucide-react"; import { Badge } from "@/components/ui/badge"; import { formatDateShort } from "@/lib/utils"; +import type { components } from "@/lib/v1"; import { credentialStatus, + identitySourceLabel, type IdentityProvider, } from "@/services/identity-providers"; +/** + * Where a user's identity comes from. While an activation, role grant or + * rename from the identity provider waits for the batched access pass, says + * so: Bifrost applies those within about two minutes, not at once. + */ +export function IdentitySourceLabel({ + user, +}: { + user: Pick; +}) { + const source = user.identity_source; + return ( + + {identitySourceLabel(user)} + {source?.access_pending_since ? ( + + + ) : null} + + ); +} + /** Whether the connection accepts provisioning requests. */ export function ProvisioningStatusBadge({ enabled }: { enabled: boolean }) { return enabled ? ( diff --git a/client/src/components/identity-providers/ProvisioningCard.tsx b/client/src/components/identity-providers/ProvisioningCard.tsx index d2481477a2..9ead24dd11 100644 --- a/client/src/components/identity-providers/ProvisioningCard.tsx +++ b/client/src/components/identity-providers/ProvisioningCard.tsx @@ -223,6 +223,14 @@ export function ProvisioningCard({ {provider.user_count} + {provider.pending_access_count > 0 ? ( + + {provider.pending_access_count} awaiting access + + ) : null} diff --git a/client/src/lib/v1.d.ts b/client/src/lib/v1.d.ts index bbb5b585ea..4abf96216a 100644 --- a/client/src/lib/v1.d.ts +++ b/client/src/lib/v1.d.ts @@ -21636,6 +21636,8 @@ export interface components { group_count: number; /** Mapped Group Count */ mapped_group_count: number; + /** Pending Access Count */ + pending_access_count: number; /** Last Request At */ last_request_at: string | null; /** Last Sync At */ @@ -21761,6 +21763,8 @@ export interface components { * @enum {string} */ vendor: "entra" | "okta" | "jumpcloud" | "google" | "other"; + /** Access Pending Since */ + access_pending_since?: string | null; }; /** IdentityUpdate */ IdentityUpdate: { diff --git a/client/src/pages/IdentityProviderDetail.test.tsx b/client/src/pages/IdentityProviderDetail.test.tsx index b7d5c78d22..81093ead2f 100644 --- a/client/src/pages/IdentityProviderDetail.test.tsx +++ b/client/src/pages/IdentityProviderDetail.test.tsx @@ -100,6 +100,7 @@ function provider(overrides: Partial = {}): IdentityProvider { user_count: 30, group_count: 2, mapped_group_count: 2, + pending_access_count: 0, last_request_at: null, last_sync_at: null, created_at: new Date().toISOString(), diff --git a/client/src/pages/IdentityProviders.test.tsx b/client/src/pages/IdentityProviders.test.tsx index 3b1cd4d4f3..03af1fb373 100644 --- a/client/src/pages/IdentityProviders.test.tsx +++ b/client/src/pages/IdentityProviders.test.tsx @@ -97,6 +97,7 @@ function provider(overrides: Partial = {}): IdentityProvider { user_count: 30, group_count: 2, mapped_group_count: 2, + pending_access_count: 0, last_request_at: null, last_sync_at: null, created_at: new Date().toISOString(), diff --git a/client/src/pages/Users.tsx b/client/src/pages/Users.tsx index dcef878b4e..681c589f8e 100644 --- a/client/src/pages/Users.tsx +++ b/client/src/pages/Users.tsx @@ -76,7 +76,7 @@ import { } from "@/components/users/BulkUserDialogs"; import { orgTarget } from "@/lib/authorization"; import { useAuthorization } from "@/services/authorization"; -import { identitySourceLabel } from "@/services/identity-providers"; +import { IdentitySourceLabel } from "@/components/identity-providers/IdentityProviderBadges"; import { ListPagination } from "@/components/pagination/ListPagination"; import type { components, components as v1 } from "@/lib/v1"; type User = components["schemas"]["UserPublic"]; @@ -543,7 +543,7 @@ export function Users() { Identity Source
    - {identitySourceLabel(user)} +
    @@ -819,7 +819,7 @@ export function Users() { )} - {identitySourceLabel(user)} + {user.created_at From 69f24c74c9be091d5e647836a402ae9bece8ecd4 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 20:58:29 -0500 Subject: [PATCH 12/26] Fit user tables at 1280px and list new hires while they activate The Users, Identities and Identity Providers tabs switch to cards below 1280px so their tables never scroll sideways. Identity Providers shows the provider under the connection name, Users shows the identity source under the user's name, and truncated emails no longer force the email column to its full width. A new hire an identity provider is activating is listed by default with an Activating status until the access pass turns them on, instead of hiding as an inactive user. Co-Authored-By: Claude Opus 5.5 --- api/shared/sdk_users.py | 11 ++++- .../models/contracts/identity_providers.py | 2 + api/src/services/identity_providers.py | 8 ++++ .../identity_provisioning/access_pass.py | 38 ++++++++++++++++- .../test_identity_provider_access_pass.py | 29 +++++++++++++ .../IdentityProviderBadges.test.tsx | 41 ++++++++++++++++--- .../IdentityProviderBadges.tsx | 23 +++++++++++ client/src/components/users/UserEmailCell.tsx | 2 +- client/src/lib/v1.d.ts | 5 +++ client/src/pages/Identities.tsx | 2 +- client/src/pages/IdentityProviders.tsx | 19 ++++----- client/src/pages/Users.tsx | 30 +++++++------- .../src/services/identity-providers.test.ts | 1 + 13 files changed, 173 insertions(+), 38 deletions(-) diff --git a/api/shared/sdk_users.py b/api/shared/sdk_users.py index 41f3b95d49..b4673ee7c7 100644 --- a/api/shared/sdk_users.py +++ b/api/shared/sdk_users.py @@ -267,7 +267,8 @@ async def list_users( Returns ``(items, total)`` — the caller sets the ``X-Total-Count`` header from ``total``. System users are excluded, inactive users are - excluded unless ``include_inactive`` is set, and each row carries its + excluded unless ``include_inactive`` is set (except those an identity + provider is activating), and each row carries its invite status. Defaults match the historical handler (legacy email order, unbounded when ``limit`` is None). @@ -310,7 +311,13 @@ async def list_users( query = query.where(reach_filter) if not include_inactive: - query = query.where(UserORM.is_active.is_(True)) + # A new hire an identity provider is activating is listed while + # the access pass applies it, so an admin sees them at once. + from src.services.identity_provisioning.access_pass import activating_user_ids + + query = query.where( + or_(UserORM.is_active.is_(True), UserORM.id.in_(activating_user_ids())) + ) if type: if type.lower() == "platform": diff --git a/api/src/models/contracts/identity_providers.py b/api/src/models/contracts/identity_providers.py index 0840c888c5..7c26cc395b 100644 --- a/api/src/models/contracts/identity_providers.py +++ b/api/src/models/contracts/identity_providers.py @@ -41,6 +41,8 @@ class IdentitySource(BaseModel): # Set while an activation, role grant or rename from the identity # provider waits for the access pass (applied within about two minutes). access_pending_since: datetime | None = None + # The user is inactive only until that pass activates them. + activating: bool = False class IdentityProviderTokenInfo(BaseModel): diff --git a/api/src/services/identity_providers.py b/api/src/services/identity_providers.py index 0b6178156c..a622a8af73 100644 --- a/api/src/services/identity_providers.py +++ b/api/src/services/identity_providers.py @@ -579,8 +579,15 @@ async def identity_sources( session: AsyncSession, user_ids: list[UUID] ) -> dict[UUID, IdentitySource]: """``user_id -> IdentitySource`` for linked users.""" + from src.services.identity_provisioning.access_pass import activating_user_ids + if not user_ids: return {} + activating = set( + await session.scalars( + activating_user_ids().where(IdentityProviderUser.user_id.in_(user_ids)) + ) + ) rows = await session.execute( select( IdentityProviderUser.user_id, @@ -598,6 +605,7 @@ async def identity_sources( name=name, vendor=vendor, access_pending_since=pending_since, + activating=user_id in activating, ) for user_id, idp_id, name, vendor, pending_since in rows.all() } diff --git a/api/src/services/identity_provisioning/access_pass.py b/api/src/services/identity_provisioning/access_pass.py index 82329107c2..e48bb05517 100644 --- a/api/src/services/identity_provisioning/access_pass.py +++ b/api/src/services/identity_provisioning/access_pass.py @@ -26,11 +26,17 @@ from typing import Any from uuid import UUID -from sqlalchemy import Select, select, update +from sqlalchemy import Select, exists, or_, select, update from sqlalchemy.ext.asyncio import AsyncSession from src.core.database_retry import is_transaction_conflict -from src.models.orm.identity_providers import IdentityProvider, IdentityProviderUser +from src.models.orm.identity_providers import ( + IdentityProvider, + IdentityProviderGroup, + IdentityProviderGroupMapping, + IdentityProviderGroupMember, + IdentityProviderUser, +) from src.models.orm.organizations import Organization from src.models.orm.users import User from src.services.identity_provisioning.core import Provisioner @@ -73,6 +79,34 @@ def _waiting(*columns: Any) -> Select[Any]: ) +def activating_user_ids() -> Select[Any]: + """Inactive users the next pass will activate: their link is active and + waiting, and their connection maps no groups or they're in a mapped one + (the pass's own ``desired_active``).""" + in_mapped_group = exists( + select(IdentityProviderGroupMember.group_id) + .join( + IdentityProviderGroupMapping, + IdentityProviderGroupMapping.group_id == IdentityProviderGroupMember.group_id, + ) + .where(IdentityProviderGroupMember.identity_provider_user_id == IdentityProviderUser.id) + ) + maps_groups = exists( + select(IdentityProviderGroupMapping.group_id) + .join(IdentityProviderGroup, IdentityProviderGroup.id == IdentityProviderGroupMapping.group_id) + .where(IdentityProviderGroup.identity_provider_id == IdentityProviderUser.identity_provider_id) + ) + return ( + _waiting(IdentityProviderUser.user_id) + .join(User, User.id == IdentityProviderUser.user_id) + .where( + IdentityProviderUser.active.is_(True), + User.is_active.is_(False), + or_(~maps_groups, in_mapped_group), + ) + ) + + async def run_access_pass(session: AsyncSession) -> AccessPassResult: """Apply waiting grants; the caller commits, then runs each provisioner's ``after_commit``. diff --git a/api/tests/unit/services/test_identity_provider_access_pass.py b/api/tests/unit/services/test_identity_provider_access_pass.py index 8f65f30198..cfd8673552 100644 --- a/api/tests/unit/services/test_identity_provider_access_pass.py +++ b/api/tests/unit/services/test_identity_provider_access_pass.py @@ -365,6 +365,35 @@ async def test_a_waiting_grant_shows_on_the_users_identity_source(setup): assert source.access_pending_since is None +@pytest.mark.asyncio +async def test_a_new_hire_waiting_for_activation_is_listed_as_activating(setup): + from shared.sdk_users import list_users + from tests.helpers.authorization import admin_caller + + session = setup["session"] + group = await _mapped_group(setup) + provisioner = Provisioner(session, setup["connection"]) + hire = await provisioner.create_user(_record(f"hire@{setup['domain']}")) + outsider = await provisioner.create_user(_record(f"outsider@{setup['domain']}")) + await provisioner.set_group_members(group, [hire.id]) + + async def listed(): + items, _ = await list_users(session, admin_caller(), search=setup["domain"]) + return {u.email.split("@")[0]: u for u in items} + + users = await listed() + assert set(users) == {"hire"} + assert not users["hire"].is_active + source = users["hire"].identity_source + assert source is not None and source.activating + + await run_access_pass(session) + + source = (await listed())["hire"].identity_source + assert source is not None and not source.activating + assert (await _refreshed(session, outsider))[0].is_active is False + + @pytest.mark.asyncio async def test_a_disabled_connections_grants_wait(setup): session, connection = setup["session"], setup["connection"] diff --git a/client/src/components/identity-providers/IdentityProviderBadges.test.tsx b/client/src/components/identity-providers/IdentityProviderBadges.test.tsx index dc02c6888c..fbc66cd51f 100644 --- a/client/src/components/identity-providers/IdentityProviderBadges.test.tsx +++ b/client/src/components/identity-providers/IdentityProviderBadges.test.tsx @@ -2,12 +2,17 @@ import { describe, expect, it } from "vitest"; import { renderWithProviders, screen } from "@/test-utils"; -import { IdentitySourceLabel } from "./IdentityProviderBadges"; +import { + IdentitySourceLabel, + InactiveStatusBadge, +} from "./IdentityProviderBadges"; const source = { identity_provider_id: "4f0c7a0e-2a3b-4c5d-8e9f-0a1b2c3d4e5f", name: "Contoso Entra ID", vendor: "entra" as const, + access_pending_since: null, + activating: false, }; describe("IdentitySourceLabel", () => { @@ -18,11 +23,7 @@ describe("IdentitySourceLabel", () => { }); it("names the connection that manages a user", () => { - renderWithProviders( - , - ); + renderWithProviders(); expect(screen.getByText("Contoso Entra ID")).toBeInTheDocument(); expect(screen.queryByText("Access Pending")).not.toBeInTheDocument(); @@ -46,3 +47,31 @@ describe("IdentitySourceLabel", () => { ); }); }); + +describe("InactiveStatusBadge", () => { + it("calls an inactive user Inactive", () => { + renderWithProviders(); + + expect(screen.getByText("Inactive")).toBeInTheDocument(); + }); + + it("calls a new hire the identity provider is activating Activating", () => { + renderWithProviders( + , + ); + + expect(screen.getByText("Activating")).toHaveAttribute( + "title", + "Contoso Entra ID added this user; Bifrost activates them within about two minutes", + ); + expect(screen.queryByText("Inactive")).not.toBeInTheDocument(); + }); +}); diff --git a/client/src/components/identity-providers/IdentityProviderBadges.tsx b/client/src/components/identity-providers/IdentityProviderBadges.tsx index 00ea060ecf..94d6e08965 100644 --- a/client/src/components/identity-providers/IdentityProviderBadges.tsx +++ b/client/src/components/identity-providers/IdentityProviderBadges.tsx @@ -36,6 +36,29 @@ export function IdentitySourceLabel({ ); } +/** + * An inactive user's status: Activating while the access pass is about to + * turn on a new hire from an identity provider, Inactive otherwise. + */ +export function InactiveStatusBadge({ + user, +}: { + user: Pick; +}) { + const source = user.identity_source; + return source?.activating ? ( + + + ) : ( + Inactive + ); +} + /** Whether the connection accepts provisioning requests. */ export function ProvisioningStatusBadge({ enabled }: { enabled: boolean }) { return enabled ? ( diff --git a/client/src/components/users/UserEmailCell.tsx b/client/src/components/users/UserEmailCell.tsx index 77c5ebe23f..18c6e26034 100644 --- a/client/src/components/users/UserEmailCell.tsx +++ b/client/src/components/users/UserEmailCell.tsx @@ -42,7 +42,7 @@ export function UserEmailCell({ {email} diff --git a/client/src/lib/v1.d.ts b/client/src/lib/v1.d.ts index 4abf96216a..3c8a48a48c 100644 --- a/client/src/lib/v1.d.ts +++ b/client/src/lib/v1.d.ts @@ -21765,6 +21765,11 @@ export interface components { vendor: "entra" | "okta" | "jumpcloud" | "google" | "other"; /** Access Pending Since */ access_pending_since?: string | null; + /** + * Activating + * @default false + */ + activating: boolean; }; /** IdentityUpdate */ IdentityUpdate: { diff --git a/client/src/pages/Identities.tsx b/client/src/pages/Identities.tsx index 2f49b46ff9..aef72c76f4 100644 --- a/client/src/pages/Identities.tsx +++ b/client/src/pages/Identities.tsx @@ -121,7 +121,7 @@ function matches(identity: Identity, search: string): boolean { */ export function Identities() { const navigate = useNavigate(); - const isNarrow = useMediaQuery("(max-width: 1023px)"); + const isNarrow = useMediaQuery("(max-width: 1279px)"); const authorization = useAuthorization(); const [isCreateOpen, setIsCreateOpen] = useState(false); const [search, setSearch] = useState(""); diff --git a/client/src/pages/IdentityProviders.tsx b/client/src/pages/IdentityProviders.tsx index 3d203910fc..adc137cb25 100644 --- a/client/src/pages/IdentityProviders.tsx +++ b/client/src/pages/IdentityProviders.tsx @@ -71,7 +71,9 @@ function Domains({ domains }: { domains: string[] }) { */ export function IdentityProviders() { const navigate = useNavigate(); - const isNarrow = useMediaQuery("(max-width: 1023px)"); + // The table needs about 950px, so it gives way to cards until the + // content area has that room (a 1280px window with the sidebar open). + const isNarrow = useMediaQuery("(max-width: 1279px)"); const authorization = useAuthorization(); const [search, setSearch] = useState(""); const [isCreateOpen, setIsCreateOpen] = useState(false); @@ -251,12 +253,7 @@ export function IdentityProviders() { Name - - Organization - - - Provider - + Organization Email Domains Provisioning @@ -288,13 +285,13 @@ export function IdentityProviders() { > {provider.name} +
    + {vendorLabel(provider.vendor)} +
    - + {provider.organization_name} - - {vendorLabel(provider.vendor)} - (null); const navigate = useNavigate(); - const isNarrow = useMediaQuery("(max-width: 1023px)"); + const isNarrow = useMediaQuery("(max-width: 1279px)"); const [isCreateOpen, setIsCreateOpen] = useState(false); const [searchTerm, setSearchTerm] = useState(""); const [showDisabled, setShowDisabled] = useState(false); @@ -493,9 +496,7 @@ export function Users() {
    {!user.is_active ? ( - - Inactive - + ) : ( - - Identity Source - )}
    + {user.identity_source && ( +
    + +
    + )} {!user.is_active ? ( - - Inactive - + ) : ( )} - - - {user.created_at ? new Date( @@ -845,7 +845,7 @@ export function Users() { { identity_provider_id: "idp-1", name: "Contoso Entra ID", vendor: "entra", + activating: false, }, }), ).toBe("Contoso Entra ID"); From 7accde25bb703578710867ee570793487980fe4c Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 22:06:35 -0500 Subject: [PATCH 13/26] Resolve the event registry merge conflict The merge of origin/main committed conflict markers in api/src/services/events/registry.py. Keep both sides: main's user.created and user.signed_in topics and this branch's identity provisioning topics, so all stay reserved. Co-Authored-By: Claude Opus 5.5 --- api/src/services/events/registry.py | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/api/src/services/events/registry.py b/api/src/services/events/registry.py index c2f4791504..6d74ea2828 100644 --- a/api/src/services/events/registry.py +++ b/api/src/services/events/registry.py @@ -21,7 +21,6 @@ "name": "Admin User", } -<<<<<<< HEAD _IDENTITY_PROVIDER = { "id": "550e8400-e29b-41d4-a716-446655440050", "name": "Contoso Entra ID", @@ -45,7 +44,8 @@ "userName": "alice@contoso.com", "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User": {"department": "Support"}, }, -======= +} + _USER_ACTOR = { "type": "user", "id": "550e8400-e29b-41d4-a716-446655440030", @@ -58,7 +58,6 @@ "email": "alice@example.com", "name": "Alice", "organization_id": "550e8400-e29b-41d4-a716-446655440010", ->>>>>>> origin/main } _BASE_BODY = { @@ -97,7 +96,6 @@ def _body(*, actor: dict, **fields: object) -> dict: ), }, { -<<<<<<< HEAD "topic": "user.provisioned", "description": "Fired when an identity provider creates a user or adopts an existing one.", "category": "Users", @@ -190,7 +188,10 @@ def _body(*, actor: dict, **fields: object) -> dict: "hint": "x7Qa", "expires_at": "2026-06-11T12:34:56Z", "stage": "expiring", -======= + }, + ), + }, + { "topic": "user.created", "description": ( "Fired after a user account is created: by an administrator (admin), on a first SSO " @@ -241,7 +242,6 @@ def _body(*, actor: dict, **fields: object) -> dict: }, "groups": ["5d8f3c1e-2b4a-4c6d-8e9f-1a2b3c4d5e6f"], "groups_overage": False, ->>>>>>> origin/main }, ), }, From 09097e3bbec391467303257bd333dd480fe68f05 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 22:24:21 -0500 Subject: [PATCH 14/26] Close identity provider gaps found in peer review Privileged access is decided in Bifrost, never by a directory: a connection won't adopt, change or delete a privileged user, the access pass leaves one alone, and a role with privileged permissions can't be mapped and is refused at grant time if it gains them later. An IdP-managed user can't be moved to another organization (directly, in bulk or by promotion), so a connection can't act or grant roles outside its own organization. Also: - Emails and SCIM userNames match without regard to case, with a case-insensitive unique index on userName. - A PUT that omits active keeps the user's current state. - SCIM authenticates before rate limiting; bad tokens are limited per client and can't use up the connection's quota. - Token rotation serializes on the connection, so two concurrent rotations can't both keep a full-lifetime token. - Expiry warnings are matched by connection id, not name, and dismissed before the rotation commits. - A competing domain verification answers 409 instead of a 500. - PATCH rejects unsupported verbs before touching unknown extensions, and malformed input (no userName, non-object extensions, deep JSON, huge startIndex, a lost unique race) answers 400/409 instead of 500. - Mapping changes read the group under the exclusive policy lock. - Access Pending isn't shown for a disabled connection or organization. - A newly issued token leaves browser history as soon as it's shown. Co-Authored-By: Claude Opus 5.5 --- .../versions/20261116_identity_providers.py | 10 +- api/shared/sdk_users.py | 23 ++ api/src/core/database_retry.py | 9 + api/src/core/transaction_conflicts.py | 24 ++- api/src/models/orm/identity_providers.py | 8 +- api/src/models/orm/users.py | 4 +- api/src/routers/identity_providers.py | 6 +- api/src/routers/scim.py | 44 +++- api/src/services/identity_providers.py | 55 ++++- .../services/identity_provisioning/core.py | 126 +++++++---- .../services/identity_provisioning/domains.py | 9 +- .../services/identity_provisioning/expiry.py | 30 +-- .../services/identity_provisioning/scim.py | 24 ++- .../services/identity_provisioning/tokens.py | 6 + api/src/services/notification_service.py | 28 +++ .../unit/core/test_transaction_conflicts.py | 21 ++ .../unit/routers/test_scim_authentication.py | 52 +++++ .../test_identity_provider_domains.py | 53 +++++ .../test_identity_provider_token_expiry.py | 72 +++++-- .../test_identity_provisioning_core.py | 200 ++++++++++++++++++ .../test_identity_provisioning_scim.py | 44 ++++ .../src/pages/IdentityProviderDetail.test.tsx | 30 ++- client/src/pages/IdentityProviderDetail.tsx | 18 +- .../2026-11-16-scim-identity-providers.md | 2 + 24 files changed, 770 insertions(+), 128 deletions(-) create mode 100644 api/tests/unit/routers/test_scim_authentication.py create mode 100644 api/tests/unit/services/test_identity_provider_domains.py diff --git a/api/alembic/versions/20261116_identity_providers.py b/api/alembic/versions/20261116_identity_providers.py index 8f9499b0f1..0ef6db227e 100644 --- a/api/alembic/versions/20261116_identity_providers.py +++ b/api/alembic/versions/20261116_identity_providers.py @@ -141,9 +141,13 @@ def upgrade() -> None: sa.UniqueConstraint( "identity_provider_id", "external_id", name="uq_identity_provider_users_external_id" ), - sa.UniqueConstraint( - "identity_provider_id", "user_name", name="uq_identity_provider_users_user_name" - ), + ) + # SCIM matches userName without regard to case, so uniqueness does too. + op.create_index( + "uq_identity_provider_users_user_name_ci", + "identity_provider_users", + ["identity_provider_id", sa.text("lower(user_name)")], + unique=True, ) op.create_table( diff --git a/api/shared/sdk_users.py b/api/shared/sdk_users.py index b4673ee7c7..18f1917e0a 100644 --- a/api/shared/sdk_users.py +++ b/api/shared/sdk_users.py @@ -668,6 +668,17 @@ async def update_user( and db_user.id in await _platform_operator_holders(session, [db_user.id]) ): raise UserServiceError(409, OPERATOR_MOVE_MESSAGE) + # Promotion moves the user to the provider organization. + destination = PROVIDER_ORG_ID if is_superuser else organization_id + if destination is not None and destination != db_user.organization_id: + from src.services.identity_providers import ( + managed_user_move_message, + managing_connections, + ) + + managing = await managing_connections(session, [db_user.id]) + if db_user.id in managing: + raise UserServiceError(409, managed_user_move_message(managing[db_user.id])) if is_superuser is False and db_user.organization_id is None and organization_id is None: from src.services.user_role_assignments import ADMIN_REMOVAL_MESSAGE @@ -897,6 +908,15 @@ async def bulk_update_users( if request.operation == "move_org" else set() ) + if request.operation == "move_org": + from src.services.identity_providers import ( + managed_user_move_message, + managing_connections, + ) + + managing = await managing_connections(session, list(users_by_id)) + else: + managing = {} actor_id = caller.principal.user_id current_roles: dict[UUID, set[UUID]] = {} @@ -990,6 +1010,9 @@ def fail(uid: UUID, reason: str) -> None: if uid in operators and target != PROVIDER_ORG_ID: fail(uid, OPERATOR_MOVE_MESSAGE) continue + if uid in managing and target != u.organization_id: + fail(uid, managed_user_move_message(managing[uid])) + continue u.organization_id = target u.updated_at = datetime.now(timezone.utc) succeeded.append(uid) diff --git a/api/src/core/database_retry.py b/api/src/core/database_retry.py index 1d6dcf9dd4..ad71bbe080 100644 --- a/api/src/core/database_retry.py +++ b/api/src/core/database_retry.py @@ -95,6 +95,15 @@ def is_transaction_conflict(exc: BaseException) -> bool: return getattr(exc, "sqlstate", None) in _CONFLICT_SQLSTATES +def is_unique_violation(exc: BaseException) -> bool: + """Whether a statement broke a unique constraint (unique_violation).""" + if isinstance(exc, DBAPIError): + if exc.orig is None: + return False + exc = exc.orig + return getattr(exc, "sqlstate", None) == "23505" + + def is_retryable_database_error(exc: BaseException) -> bool: """Whether running the same unit of work again can succeed. diff --git a/api/src/core/transaction_conflicts.py b/api/src/core/transaction_conflicts.py index d5fbae8f91..e49cc646f0 100644 --- a/api/src/core/transaction_conflicts.py +++ b/api/src/core/transaction_conflicts.py @@ -4,6 +4,11 @@ (after ``deadlock_timeout``, one second by default) and aborts one of them with ``deadlock_detected``. Nothing that transaction wrote is kept, so the right answer is "nothing was saved; try again" rather than a generic 500. + +A route can also name the error for a unique violation: two requests that +both pass a "does this already exist?" check and then both insert. The +loser's transaction is rolled back, so its answer is the conflict the check +would have given had it run second. """ from __future__ import annotations @@ -15,14 +20,19 @@ from fastapi import Request, Response from fastapi.routing import APIRoute -from src.core.database_retry import is_transaction_conflict +from src.core.database_retry import is_transaction_conflict, is_unique_violation logger = logging.getLogger(__name__) -def conflict_aware_route(on_conflict: Callable[[], Exception]) -> type[APIRoute]: +def conflict_aware_route( + on_conflict: Callable[[], Exception], + *, + on_duplicate: Callable[[], Exception] | None = None, +) -> type[APIRoute]: """An ``APIRoute`` class that turns a transaction conflict into the - exception ``on_conflict`` builds (handled by the app's exception + exception ``on_conflict`` builds, and a unique violation into the one + ``on_duplicate`` builds when given (handled by the app's exception handlers), and logs it, so the rate of these is visible.""" class ConflictAwareRoute(APIRoute): @@ -33,6 +43,14 @@ async def route(request: Request) -> Response: try: return await handler(request) except Exception as exc: + if on_duplicate is not None and is_unique_violation(exc): + logger.warning( + "Unique violation on %s %s; nothing was saved: %s", + request.method, + request.url.path, + exc, + ) + raise on_duplicate() from None if not is_transaction_conflict(exc): raise logger.warning( diff --git a/api/src/models/orm/identity_providers.py b/api/src/models/orm/identity_providers.py index dbf78ed2dc..899c932005 100644 --- a/api/src/models/orm/identity_providers.py +++ b/api/src/models/orm/identity_providers.py @@ -183,8 +183,12 @@ class IdentityProviderUser(Base): UniqueConstraint( "identity_provider_id", "external_id", name="uq_identity_provider_users_external_id" ), - UniqueConstraint( - "identity_provider_id", "user_name", name="uq_identity_provider_users_user_name" + # SCIM matches userName without regard to case, so uniqueness does too. + Index( + "uq_identity_provider_users_user_name_ci", + "identity_provider_id", + text("lower(user_name)"), + unique=True, ), Index( "ix_identity_provider_users_access_pending", diff --git a/api/src/models/orm/users.py b/api/src/models/orm/users.py index f6c3f059e3..aaab34337e 100644 --- a/api/src/models/orm/users.py +++ b/api/src/models/orm/users.py @@ -178,7 +178,9 @@ class UserRole(Base): ) # Set when identity provisioning created this assignment from a group # mapping; provisioning only ever removes assignments it created. NULL is - # a manual assignment (a manual edit or a deleted connection clears it). + # a manual assignment. Deleting the connection clears it and the role + # stays; manually assigning a role the directory already granted leaves + # it set, so the role still goes when the user leaves the group. identity_provider_id: Mapped[UUID | None] = mapped_column( ForeignKey("identity_providers.id", ondelete="SET NULL"), default=None ) diff --git a/api/src/routers/identity_providers.py b/api/src/routers/identity_providers.py index 6865a19a70..c08b8bfe21 100644 --- a/api/src/routers/identity_providers.py +++ b/api/src/routers/identity_providers.py @@ -24,7 +24,6 @@ ) from src.services import identity_providers as service from src.services.authorization.enforce import load_caller -from src.services.identity_provisioning.expiry import clear_expiry_notifications from src.services.operation_catalog import operation_route def _conflict() -> HTTPException: @@ -177,11 +176,8 @@ async def rotate_identity_provider_token( user: CurrentActiveUser, ) -> IdentityProviderTokenIssued: caller = await load_caller(db, user) - issued, name = await service.rotate_token( - db, caller, identity_provider_id, request.lifetime_days - ) + issued = await service.rotate_token(db, caller, identity_provider_id, request.lifetime_days) await db.commit() - await clear_expiry_notifications(name) return issued diff --git a/api/src/routers/scim.py b/api/src/routers/scim.py index a76a6f62b7..6a1bae0a78 100644 --- a/api/src/routers/scim.py +++ b/api/src/routers/scim.py @@ -18,7 +18,7 @@ from sqlalchemy import func, select from src.core.db_deps import DbSession -from src.core.rate_limit import RateLimiter +from src.core.rate_limit import RateLimiter, get_client_ip from src.core.transaction_conflicts import conflict_aware_route from src.models.orm.identity_providers import ( IdentityProvider, @@ -40,16 +40,24 @@ def _conflict() -> ScimError: ) +def _duplicate() -> ScimError: + # Another request created the same resource first. + return ScimError(409, "A resource with these unique attributes already exists", "uniqueness") + + router = APIRouter( prefix="/api/scim/v2/{identity_provider_id}", tags=["SCIM"], include_in_schema=False, - route_class=conflict_aware_route(_conflict), + route_class=conflict_aware_route(_conflict, on_duplicate=_duplicate), ) # Generous: a large Entra or Okta cycle sends bursts of user and membership -# changes. Applied per connection, before any database work. +# changes. Applied per authenticated connection. _RATE_LIMIT = RateLimiter(max_requests=1200, window_seconds=60) +# Requests with a bad token are limited per client, apart from the +# connection's own quota, so they can't crowd out the identity provider. +_AUTH_FAILURE_LIMIT = RateLimiter(max_requests=60, window_seconds=60) _MAX_BODY_BYTES = 1024 * 1024 @@ -80,17 +88,26 @@ async def scim_connection( try: connection_id = UUID(identity_provider_id) except ValueError: - raise ScimError(401, "Invalid or expired token") from None + connection_id = None + connection = ( + await tokens.authenticate(db, connection_id, token.strip()) + if connection_id is not None + else None + ) + if connection is None or connection.source_type != "scim": + await _limit(_AUTH_FAILURE_LIMIT, "scim_auth_failure", get_client_ip(request)) + raise ScimError(401, "Invalid or expired token") + await _limit(_RATE_LIMIT, "scim", str(connection.id)) + return connection + + +async def _limit(limiter: RateLimiter, endpoint: str, identifier: str) -> None: try: - await _RATE_LIMIT.check("scim", str(connection_id)) + await limiter.check(endpoint, identifier) except HTTPException as exc: if exc.status_code == status.HTTP_429_TOO_MANY_REQUESTS: raise ScimError(429, "Too many requests") from None raise - connection = await tokens.authenticate(db, connection_id, token.strip()) - if connection is None or connection.source_type != "scim": - raise ScimError(401, "Invalid or expired token") - return connection Connection = Annotated[IdentityProvider, Depends(scim_connection)] @@ -108,12 +125,17 @@ async def _json_body(request: Request) -> Any: raise ScimError(413, "The request body is too large", "tooLarge") try: return json.loads(raw) - except ValueError: + # RecursionError: nesting deeper than the parser can follow. + except (ValueError, RecursionError): raise ScimError(400, "The request body is not valid JSON", "invalidSyntax") from None +# Past any real directory; keeps the offset inside PostgreSQL's bigint. +_MAX_START_INDEX = 2**31 + + def _paging(start_index: int, count: int | None) -> tuple[int, int]: - start = max(start_index, 1) + start = min(max(start_index, 1), _MAX_START_INDEX) limit = scim.MAX_RESULTS if count is None else max(0, min(count, scim.MAX_RESULTS)) return start, limit diff --git a/api/src/services/identity_providers.py b/api/src/services/identity_providers.py index a622a8af73..d90e2027c6 100644 --- a/api/src/services/identity_providers.py +++ b/api/src/services/identity_providers.py @@ -49,8 +49,10 @@ require_operation, require_permission, ) +from src.services.authorization.privilege import is_privileged_principal from src.services.identity_provisioning import domains, tokens from src.services.identity_provisioning.core import UNMAPPABLE_ROLE_IDS, Provisioner +from src.services.identity_provisioning.expiry import clear_expiry_notifications def scim_base_url(identity_provider_id: UUID) -> str: return f"{get_settings().public_url.rstrip('/')}/api/scim/v2/{identity_provider_id}" @@ -371,9 +373,12 @@ async def delete_connection( async def rotate_token( session: AsyncSession, caller: Caller, identity_provider_id: UUID, lifetime_days: int -) -> tuple[IdentityProviderTokenIssued, str]: - """Issue a new token; returns it with the connection's name, whose expiry - warnings the caller clears after committing.""" +) -> IdentityProviderTokenIssued: + """Issue a new token and dismiss the connection's expiry warnings. + + The warnings are dismissed before the caller commits: if that fails, + nothing is saved and the old token still works. If the commit fails + afterwards, the next expiry check raises the warning again.""" connection = await _connection( session, caller, identity_provider_id, operation="identityproviders.token.rotate" ) @@ -394,10 +399,10 @@ async def rotate_token( ) info = _token_info(token) assert info is not None - issued = IdentityProviderTokenIssued( + await clear_expiry_notifications(connection.id) + return IdentityProviderTokenIssued( token=raw, info=info, previous_tokens_expire_at=previous_expiry ) - return issued, connection.name # --------------------------------------------------------------------------- @@ -488,6 +493,8 @@ async def _authorize_mapped_roles( missing = role_ids - roles.keys() if missing: raise HTTPException(422, f"Role {sorted(missing, key=str)[0]} not found") + if any(is_privileged_principal(role.permissions) for role in roles.values()): + raise HTTPException(422, "Roles with privileged permissions can't be mapped from a group") require_permission(caller, "roleassignments.readwrite", organization_id) reach = permitted_organizations(caller, "roleassignments.readwrite") boundaries = default_boundaries(organization_id) @@ -512,11 +519,13 @@ async def set_group_mapping( connection = await _connection( session, caller, identity_provider_id, operation="identityproviders.groups.map" ) - group = await _group(session, connection, group_id) wanted = set(role_ids) await _authorize_mapped_roles(session, caller, connection.organization_id, wanted) provisioner = Provisioner(session, connection) + # Read the group under the exclusive policy lock, which a group deletion + # also takes, so it can't vanish before the mapping is written. await provisioner.hold_policy(exclusive=True) + group = await _group(session, connection, group_id) mapping = await session.get(IdentityProviderGroupMapping, group.id) if mapping is None: session.add( @@ -556,9 +565,9 @@ async def delete_group_mapping( connection = await _connection( session, caller, identity_provider_id, operation="identityproviders.groups.unmap" ) - group = await _group(session, connection, group_id) provisioner = Provisioner(session, connection) await provisioner.hold_policy(exclusive=True) + group = await _group(session, connection, group_id) mapping = await session.get(IdentityProviderGroupMapping, group.id) if mapping is None: raise HTTPException(404, "This group is not mapped") @@ -575,6 +584,30 @@ async def delete_group_mapping( return provisioner +def managed_user_move_message(connection_name: str) -> str: + return ( + f"This user is managed by the identity provider {connection_name}. " + "Remove them from the directory before moving them to another organization." + ) + + +async def managing_connections(session: AsyncSession, user_ids: list[UUID]) -> dict[UUID, str]: + """``user_id -> connection name`` for users an identity provider manages. + + A linked user stays in their connection's organization: the connection + may only act on its own organization's users, so a move would hand it a + user (and grant its mapped roles) somewhere else. + """ + if not user_ids: + return {} + rows = await session.execute( + select(IdentityProviderUser.user_id, IdentityProvider.name) + .join(IdentityProvider, IdentityProvider.id == IdentityProviderUser.identity_provider_id) + .where(IdentityProviderUser.user_id.in_(user_ids)) + ) + return {user_id: name for user_id, name in rows.all()} + + async def identity_sources( session: AsyncSession, user_ids: list[UUID] ) -> dict[UUID, IdentitySource]: @@ -595,8 +628,10 @@ async def identity_sources( IdentityProvider.name, IdentityProvider.vendor, IdentityProviderUser.access_pending_since, + IdentityProvider.is_enabled & Organization.is_active, ) .join(IdentityProvider, IdentityProvider.id == IdentityProviderUser.identity_provider_id) + .join(Organization, Organization.id == IdentityProvider.organization_id) .where(IdentityProviderUser.user_id.in_(user_ids)) ) return { @@ -604,9 +639,11 @@ async def identity_sources( identity_provider_id=idp_id, name=name, vendor=vendor, - access_pending_since=pending_since, + # Pending only while the pass will apply it: a disabled + # connection's or organization's changes wait for it to return. + access_pending_since=pending_since if applies else None, activating=user_id in activating, ) - for user_id, idp_id, name, vendor, pending_since in rows.all() + for user_id, idp_id, name, vendor, pending_since, applies in rows.all() } diff --git a/api/src/services/identity_provisioning/core.py b/api/src/services/identity_provisioning/core.py index 03c8fde082..2dad3d9065 100644 --- a/api/src/services/identity_provisioning/core.py +++ b/api/src/services/identity_provisioning/core.py @@ -7,7 +7,12 @@ verified claim of its organization, and never touches a user whose home is another organization. - An existing user in the same organization is adopted by email; one in - another organization is a conflict. + another organization is a conflict. Emails and userNames match without + regard to case. +- Privileged access is decided in Bifrost, never by a directory: a + privileged user is not adopted, changed or deleted (a conflict), the + access pass leaves one alone, and a role carrying privileged permissions + is never granted. - The identity provider owns name, email and active for linked users. - Roles come from group mappings. Provisioning only removes the role assignments it created (``UserRole.identity_provider_id``), so a manual @@ -41,7 +46,7 @@ from typing import Any from uuid import UUID -from sqlalchemy import delete, or_, select, text +from sqlalchemy import delete, func, or_, select, text from sqlalchemy.ext.asyncio import AsyncSession from shared.builtin_roles import ( @@ -72,6 +77,11 @@ {*BASE_ROLE_IDS, PLATFORM_ADMIN_ROLE_ID, PLATFORM_OPERATOR_ROLE_ID, DECRYPTION_ROLE_ID} ) +PRIVILEGED_USER_MESSAGE = ( + "This user holds privileged access, which is managed in Bifrost, not by an identity provider" +) +PRIVILEGED_ROLE_MESSAGE = "Roles with privileged permissions aren't granted by identity providers" + class ProvisioningErrorKind(enum.Enum): CONFLICT = "conflict" @@ -95,7 +105,9 @@ class UserRecord: email: str external_id: str | None display_name: str | None - active: bool + # None when the directory didn't say: a new user starts active, an + # existing one keeps what it had. + active: bool | None attributes: dict[str, Any] @@ -236,17 +248,18 @@ async def create_user(self, record: UserRecord) -> IdentityProviderUser: # Lock a user being adopted before deciding it may be. user = await self.session.scalar( select(User) - .where(User.email == email) + .where(func.lower(User.email) == email) .with_for_update() .execution_options(populate_existing=True) ) created_user = user is None + link_active = record.active is not False if user is None: # Insert in the final state: a new user carries no token, so the # insert leaves the authority epoch alone where a later flip of # is_active would not. A new user has no groups yet, so once the # connection maps any group, the user waits for one. - active = record.active and not await self._mapped_group_roles() + active = link_active and not await self._mapped_group_roles() user = await self._new_user(email, record, active=active) else: await self._require_adoptable(user) @@ -257,7 +270,7 @@ async def create_user(self, record: UserRecord) -> IdentityProviderUser: external_id=record.external_id, user_name=record.user_name, display_name=record.display_name, - active=record.active, + active=link_active, attributes=record.attributes, created_user=created_user, ) @@ -278,11 +291,12 @@ async def replace_user( await self.hold_policy() await self._refuse_duplicate_link(record, exclude_link_id=link.id) user = await self._lock_user(link.user_id) + await self._refuse_privileged(user) email = normalize_email(record.email) if email != user.email: await self._require_claimed_domain(email) taken = await self.session.scalar( - select(User.id).where(User.email == email, User.id != user.id) + select(User.id).where(func.lower(User.email) == email, User.id != user.id) ) if taken is not None: raise ProvisioningError( @@ -292,7 +306,8 @@ async def replace_user( link.external_id = record.external_id link.user_name = record.user_name link.display_name = record.display_name - link.active = record.active + if record.active is not None: + link.active = record.active link.attributes = record.attributes await self.session.flush() await self._reconcile_access([link.id], defer_grants=True) @@ -303,6 +318,7 @@ async def replace_user( async def delete_user(self, link: IdentityProviderUser) -> None: await self.hold_policy() user = await self._lock_user(link.user_id) + await self._refuse_privileged(user) if link.created_user and user.last_login is None: from shared.run_cancellation import cancel_unstarted_runs_for_deleted_user @@ -510,9 +526,30 @@ async def _reconcile_access( ).all(): memberships.setdefault(link_id, set()).add(group_id) + privileged = await self._privileged(users.keys()) + now = datetime.now(timezone.utc) for link in sorted(links, key=lambda link: link.user_id): user = users[link.user_id] + if group is not None and group_changes and link.id in group_changes: + self.pending_events.append( + ( + "user.groups_changed", + { + **self._user_event_body(user, link), + "group": { + "id": str(group.id), + "display_name": group.display_name, + "change": group_changes[link.id], + }, + }, + ) + ) + if user.id in privileged: + # Membership is still recorded, but the directory decides + # nothing about a privileged user's access. + link.access_pending_since = None + continue mapped = memberships.get(link.id, set()) & mapped_roles.keys() desired_roles: set[UUID] = set() for group_id in mapped: @@ -543,20 +580,6 @@ async def _reconcile_access( await self.session.flush() self._mark_stale(user.id, change.roles_added | change.roles_removed) - if group is not None and group_changes and link.id in group_changes: - self.pending_events.append( - ( - "user.groups_changed", - { - **self._user_event_body(user, link), - "group": { - "id": str(group.id), - "display_name": group.display_name, - "change": group_changes[link.id], - }, - }, - ) - ) if change.activated or change.roles_added or change.roles_removed: self.pending_events.append( ( @@ -621,14 +644,23 @@ async def _missing_roles(self, user: User, desired: set[UUID]) -> set[UUID]: async def _grant_roles(self, user: User, desired: set[UUID]) -> set[UUID]: from shared.account_guards import AccountGuardError + from src.services.authorization.privilege import is_privileged_principal from src.services.user_role_assignments import ( RoleAssignmentError, default_boundaries, insert_assignment, + load_roles, ) added: set[UUID] = set() - for role_id in sorted(await self._missing_roles(user, desired)): + missing = await self._missing_roles(user, desired) + # A mapped role checked at mapping time can gain privileged + # permissions later, so check again at every grant. + roles = await load_roles(self.session, missing) + for role_id in sorted(missing): + if role_id in roles and is_privileged_principal(roles[role_id].permissions): + await self._refuse_grant(user, role_id, PRIVILEGED_ROLE_MESSAGE) + continue try: await insert_assignment( self.session, @@ -639,27 +671,29 @@ async def _grant_roles(self, user: User, desired: set[UUID]) -> set[UUID]: identity_provider_id=self.connection.id, ) except (RoleAssignmentError, AccountGuardError) as exc: - # A policy refusal, e.g. a privileged role mapped for an - # External user. Retrying can't change it, so skip the grant, - # keep provisioning the rest and record why; the next mapping - # or directory change tries again. - detail = exc.detail - logger.warning( - "Identity provider %s skipped role %s for user %s: %s", - self.connection.id, - role_id, - user.id, - log_safe(detail), - ) - await self._audit( - "identity_provider.role_grant_refused", - user, - {"email": user.email, "role_id": str(role_id), "reason": detail}, - ) + # A policy refusal, e.g. a role an External user can't hold. + await self._refuse_grant(user, role_id, exc.detail) continue added.add(role_id) return added + async def _refuse_grant(self, user: User, role_id: UUID, detail: str) -> None: + """Record a refused grant. Retrying can't change a policy refusal, + so provisioning skips it and carries on; the next mapping or + directory change tries again.""" + logger.warning( + "Identity provider %s skipped role %s for user %s: %s", + self.connection.id, + role_id, + user.id, + log_safe(detail), + ) + await self._audit( + "identity_provider.role_grant_refused", + user, + {"email": user.email, "role_id": str(role_id), "reason": detail}, + ) + async def _remove_granted_roles(self, user_id: UUID, desired: set[UUID] | None) -> set[UUID]: """Delete assignments this connection made that are no longer wanted (all of them when ``desired`` is None).""" @@ -756,6 +790,7 @@ async def _require_adoptable(self, user: User) -> None: raise ProvisioningError( ProvisioningErrorKind.CONFLICT, "This email belongs to a system account" ) + await self._refuse_privileged(user) linked = await self.session.scalar( select(IdentityProviderUser.identity_provider_id).where( IdentityProviderUser.user_id == user.id @@ -767,10 +802,19 @@ async def _require_adoptable(self, user: User) -> None: "This user is already managed by an identity provider", ) + async def _privileged(self, user_ids: Iterable[UUID]) -> frozenset[UUID]: + from src.services.authorization.enforce import privileged_user_ids + + return await privileged_user_ids(self.session, list(user_ids)) + + async def _refuse_privileged(self, user: User) -> None: + if user.is_superuser or user.id in await self._privileged([user.id]): + raise ProvisioningError(ProvisioningErrorKind.CONFLICT, PRIVILEGED_USER_MESSAGE) + async def _refuse_duplicate_link( self, record: UserRecord, *, exclude_link_id: UUID | None ) -> None: - clauses = [IdentityProviderUser.user_name == record.user_name] + clauses = [func.lower(IdentityProviderUser.user_name) == record.user_name.lower()] if record.external_id is not None: clauses.append(IdentityProviderUser.external_id == record.external_id) stmt = select(IdentityProviderUser.id).where( diff --git a/api/src/services/identity_provisioning/domains.py b/api/src/services/identity_provisioning/domains.py index 62402feb55..f3de5660f6 100644 --- a/api/src/services/identity_provisioning/domains.py +++ b/api/src/services/identity_provisioning/domains.py @@ -144,12 +144,17 @@ async def verify_claim(session: AsyncSession, claim: OrganizationDomain) -> Orga await session.flush() return claim if claim.verified_at is None: - claim.verified_at = now + # Flush first: begin_nested flushes pending changes before its + # savepoint, and the claim must only be marked verified inside it. + await session.flush() + # Read before the savepoint: rolling it back expires the claim. + domain = claim.domain try: async with session.begin_nested(): + claim.verified_at = now await session.flush() except IntegrityError: raise DomainClaimError( - 409, f"{claim.domain} is already verified by another organization" + 409, f"{domain} is already verified by another organization" ) from None return claim diff --git a/api/src/services/identity_provisioning/expiry.py b/api/src/services/identity_provisioning/expiry.py index a796957b3c..6c6249b1fe 100644 --- a/api/src/services/identity_provisioning/expiry.py +++ b/api/src/services/identity_provisioning/expiry.py @@ -18,7 +18,7 @@ import logging from dataclasses import dataclass from datetime import datetime, timedelta -from typing import Literal +from typing import Literal, get_args from uuid import UUID from sqlalchemy import select @@ -101,26 +101,25 @@ async def collect_expiry_notices(session: AsyncSession, now: datetime) -> list[E return notices -def _titles(identity_provider_name: str) -> dict[Stage, str]: - return { - "expiring": f"SCIM Token Expiring: {identity_provider_name}", - "expired": f"SCIM Token Expired: {identity_provider_name}", - } +def notification_title(notice: ExpiryNotice) -> str: + verb = "Expired" if notice.stage == "expired" else "Expiring" + return f"SCIM Token {verb}: {notice.identity_provider_name}" -def notification_title(notice: ExpiryNotice) -> str: - return _titles(notice.identity_provider_name)[notice.stage] +def _notification_key(identity_provider_id: UUID, stage: Stage) -> dict[str, str]: + # Matched by connection id: connection names repeat across organizations. + return {"identity_provider_id": str(identity_provider_id), "token_stage": stage} -async def clear_expiry_notifications(identity_provider_name: str) -> None: +async def clear_expiry_notifications(identity_provider_id: UUID) -> None: """Dismiss a connection's expiry warnings once a new token is issued.""" from src.models.contracts.notifications import NotificationCategory from src.services.notification_service import get_notification_service service = get_notification_service() - for title in _titles(identity_provider_name).values(): - existing = await service.find_admin_notification_by_title( - title=title, category=NotificationCategory.SYSTEM + for stage in get_args(Stage): + existing = await service.find_admin_notification_by_metadata( + _notification_key(identity_provider_id, stage), category=NotificationCategory.SYSTEM ) if existing: await service.dismiss_notification(existing.id, user_id="system") @@ -152,8 +151,9 @@ async def deliver_expiry_notices(notices: list[ExpiryNotice]) -> None: stage=notice.stage, ) title = notification_title(notice) - if await service.find_admin_notification_by_title( - title=title, category=NotificationCategory.SYSTEM + key = _notification_key(notice.identity_provider_id, notice.stage) + if await service.find_admin_notification_by_metadata( + key, category=NotificationCategory.SYSTEM ): continue when = notice.expires_at.strftime("%b %d, %Y") @@ -172,7 +172,7 @@ async def deliver_expiry_notices(notices: list[ExpiryNotice]) -> None: "action": "open_identity_provider", "action_label": "Regenerate Token", "action_url": f"/users/identity-providers/{notice.identity_provider_id}", - "identity_provider_id": str(notice.identity_provider_id), + **key, }, ), for_admins=True, diff --git a/api/src/services/identity_provisioning/scim.py b/api/src/services/identity_provisioning/scim.py index 53638cdbd7..6752cff919 100644 --- a/api/src/services/identity_provisioning/scim.py +++ b/api/src/services/identity_provisioning/scim.py @@ -191,6 +191,16 @@ def _patch_op_message(operations: list[dict[str, Any]]) -> dict[str, Any]: return {"schemas": [PATCH_SCHEMA], "Operations": operations} +_PATCH_VERBS = frozenset({"add", "replace", "remove"}) + + +def _extension_object(unknown: dict[str, Any], schema: str) -> dict[str, Any]: + target = unknown.setdefault(schema, {}) + if not isinstance(target, dict): + raise ScimError(400, f"The extension '{schema}' is not an object", "invalidValue") + return target + + def _apply_unknown_extension_ops( operations: list[dict[str, Any]], unknown: dict[str, Any] ) -> list[dict[str, Any]]: @@ -199,6 +209,10 @@ def _apply_unknown_extension_ops( remaining: list[dict[str, Any]] = [] for op in operations: verb = str(op.get("op", "")).lower() + # Checked before any operation is applied: the model validates only + # the operations left for it. + if verb not in _PATCH_VERBS: + raise ScimError(400, f"Unsupported patch operation '{op.get('op')}'", "invalidSyntax") path = op.get("path") value = op.get("value") if isinstance(path, str) and path.startswith("urn:") and not any( @@ -207,7 +221,7 @@ def _apply_unknown_extension_ops( schema, _, attribute = path.rpartition(":") if path in unknown or not schema: schema, attribute = path, "" - target = unknown.setdefault(schema, {}) + target = _extension_object(unknown, schema) if verb == "remove": if attribute: target.pop(attribute, None) @@ -223,7 +237,7 @@ def _apply_unknown_extension_ops( for key, item in value.items(): if key.startswith("urn:") and key not in KNOWN_USER_SCHEMAS: if isinstance(item, dict): - unknown.setdefault(key, {}).update(item) + _extension_object(unknown, key).update(item) else: known_value[key] = item if not known_value: @@ -338,7 +352,7 @@ def _primary_email(resource: Any) -> str: ) if chosen is not None: return str(chosen.value) - if "@" in resource.user_name: + if resource.user_name and "@" in resource.user_name: return resource.user_name raise ScimError(400, "The user needs an email address or an email userName", "invalidValue") @@ -356,12 +370,14 @@ def _display_name(resource: Any) -> str | None: def user_record(resource: Any, unknown: dict[str, Any]) -> UserRecord: + if not resource.user_name: + raise ScimError(400, "userName is required", "invalidValue") return UserRecord( user_name=resource.user_name, email=_primary_email(resource), external_id=resource.external_id, display_name=_display_name(resource), - active=True if resource.active is None else bool(resource.active), + active=None if resource.active is None else bool(resource.active), attributes=_attributes(resource, unknown), ) diff --git a/api/src/services/identity_provisioning/tokens.py b/api/src/services/identity_provisioning/tokens.py index 10d8c3d8db..23d7f358ba 100644 --- a/api/src/services/identity_provisioning/tokens.py +++ b/api/src/services/identity_provisioning/tokens.py @@ -63,6 +63,12 @@ async def issue_token( """Create a new token; earlier live tokens keep working for the overlap.""" if lifetime_days not in ALLOWED_LIFETIME_DAYS: raise ValueError(f"lifetime_days must be one of {ALLOWED_LIFETIME_DAYS}") + # Rotations of one connection serialize here. Otherwise two at once + # each shorten only the tokens they can see, and both new tokens keep + # their full lifetime. + await session.execute( + select(IdentityProvider.id).where(IdentityProvider.id == connection.id).with_for_update() + ) now = datetime.now(timezone.utc) overlap_end = now + ROTATION_OVERLAP await session.execute( diff --git a/api/src/services/notification_service.py b/api/src/services/notification_service.py index daff332617..f4eb8106eb 100644 --- a/api/src/services/notification_service.py +++ b/api/src/services/notification_service.py @@ -345,6 +345,34 @@ async def find_admin_notification_by_title( return None + async def find_admin_notification_by_metadata( + self, + metadata: dict[str, str], + category: NotificationCategory | None = None, + ) -> NotificationPublic | None: + """Find an admin notification whose metadata holds every pair in + ``metadata``, for deduplicating by an id where titles can repeat.""" + redis_client = await self._get_redis() + admin_ids: set[str] = await cast( + Awaitable[set[str]], redis_client.smembers(ADMIN_NOTIFICATIONS_KEY) + ) + + for notification_id in admin_ids: + data = await redis_client.get(f"{NOTIFICATION_KEY_PREFIX}{notification_id}") + if data is None: + continue + try: + notification = NotificationPublic.model_validate_json(data) + except Exception: + continue + held = notification.metadata or {} + if all(held.get(key) == value for key, value in metadata.items()) and ( + category is None or notification.category == category + ): + return notification + + return None + # ========================================================================= # Query Methods # ========================================================================= diff --git a/api/tests/unit/core/test_transaction_conflicts.py b/api/tests/unit/core/test_transaction_conflicts.py index 29b99de9dd..f9f533998f 100644 --- a/api/tests/unit/core/test_transaction_conflicts.py +++ b/api/tests/unit/core/test_transaction_conflicts.py @@ -118,3 +118,24 @@ async def change() -> None: body = response.json() assert body["schemas"] == ["urn:ietf:params:scim:api:messages:2.0:Error"] assert body["status"] == "503" + + +@pytest.mark.asyncio +async def test_a_scim_create_that_loses_a_unique_race_is_a_uniqueness_conflict(): + from src.routers.scim import _conflict, _duplicate, scim_error_response + from src.services.identity_provisioning.scim import ScimError + + router = APIRouter(route_class=conflict_aware_route(_conflict, on_duplicate=_duplicate)) + + @router.post("/change") + async def change() -> None: + raise IntegrityError("INSERT", {}, asyncpg.exceptions.UniqueViolationError("dup")) + + app = FastAPI() + app.include_router(router) + app.add_exception_handler(ScimError, lambda _request, exc: scim_error_response(exc)) # type: ignore[arg-type] + + response = await _post(app) + + assert response.status_code == 409 + assert response.json()["scimType"] == "uniqueness" diff --git a/api/tests/unit/routers/test_scim_authentication.py b/api/tests/unit/routers/test_scim_authentication.py new file mode 100644 index 0000000000..ed05865c00 --- /dev/null +++ b/api/tests/unit/routers/test_scim_authentication.py @@ -0,0 +1,52 @@ +"""SCIM bearer authentication: bad tokens can't use up a connection's quota.""" + +from types import SimpleNamespace +from unittest.mock import AsyncMock, patch +from uuid import uuid4 + +import pytest + +from src.routers import scim as scim_router +from src.services.identity_provisioning.scim import ScimError + + +def _request(token: str) -> SimpleNamespace: + return SimpleNamespace( + headers={"authorization": f"Bearer {token}"}, + client=SimpleNamespace(host="203.0.113.9"), + ) + + +@pytest.mark.asyncio +async def test_a_bad_token_counts_against_the_client_not_the_connection(): + connection_quota, failure_quota = AsyncMock(), AsyncMock() + with ( + patch.object(scim_router._RATE_LIMIT, "check", connection_quota), + patch.object(scim_router._AUTH_FAILURE_LIMIT, "check", failure_quota), + patch.object(scim_router.tokens, "authenticate", AsyncMock(return_value=None)), + patch.object(scim_router, "get_client_ip", return_value="203.0.113.9"), + ): + with pytest.raises(ScimError) as raised: + await scim_router.scim_connection(str(uuid4()), _request("bfidp_junk"), AsyncMock()) + + assert raised.value.status == 401 + connection_quota.assert_not_awaited() + failure_quota.assert_awaited_once_with("scim_auth_failure", "203.0.113.9") + + +@pytest.mark.asyncio +async def test_a_valid_token_counts_against_its_connection(): + connection = SimpleNamespace(id=uuid4(), source_type="scim") + connection_quota, failure_quota = AsyncMock(), AsyncMock() + with ( + patch.object(scim_router._RATE_LIMIT, "check", connection_quota), + patch.object(scim_router._AUTH_FAILURE_LIMIT, "check", failure_quota), + patch.object(scim_router.tokens, "authenticate", AsyncMock(return_value=connection)), + ): + result = await scim_router.scim_connection( + str(connection.id), _request("bfidp_valid"), AsyncMock() + ) + + assert result is connection + connection_quota.assert_awaited_once_with("scim", str(connection.id)) + failure_quota.assert_not_awaited() diff --git a/api/tests/unit/services/test_identity_provider_domains.py b/api/tests/unit/services/test_identity_provider_domains.py new file mode 100644 index 0000000000..457acfec9a --- /dev/null +++ b/api/tests/unit/services/test_identity_provider_domains.py @@ -0,0 +1,53 @@ +"""Domain claim verification against the database.""" + +from datetime import datetime, timezone +from unittest.mock import AsyncMock, patch +from uuid import uuid4 + +import pytest +from sqlalchemy import select + +from src.models.orm.identity_providers import OrganizationDomain +from src.models.orm.organizations import Organization +from src.services.identity_provisioning import domains + + +@pytest.mark.asyncio +async def test_a_domain_another_org_verified_is_a_conflict_and_the_transaction_survives( + db_session, +): + suffix = uuid4().hex[:10] + domain = f"contested-{suffix}.example" + first = Organization(name=f"First {suffix}", created_by="test") + second = Organization(name=f"Second {suffix}", created_by="test") + db_session.add_all([first, second]) + await db_session.flush() + db_session.add( + OrganizationDomain( + organization_id=first.id, + domain=domain, + verification_token="first", + verified_at=datetime.now(timezone.utc), + created_by="test", + ) + ) + claim = OrganizationDomain( + organization_id=second.id, domain=domain, verification_token="second", created_by="test" + ) + db_session.add(claim) + await db_session.flush() + + with patch.object( + domains, "lookup_txt_records", AsyncMock(return_value=[domains.txt_record_value(claim)]) + ): + with pytest.raises(domains.DomainClaimError) as raised: + await domains.verify_claim(db_session, claim) + + assert raised.value.status_code == 409 + # Only the savepoint rolled back: the request can still read and write. + await db_session.refresh(claim) + assert claim.verified_at is None + assert claim.last_checked_at is not None + assert await db_session.scalar( + select(OrganizationDomain.id).where(OrganizationDomain.id == claim.id) + ) == claim.id diff --git a/api/tests/unit/services/test_identity_provider_token_expiry.py b/api/tests/unit/services/test_identity_provider_token_expiry.py index a7cc2906a5..a8b096eebd 100644 --- a/api/tests/unit/services/test_identity_provider_token_expiry.py +++ b/api/tests/unit/services/test_identity_provider_token_expiry.py @@ -2,6 +2,7 @@ announced once.""" from datetime import datetime, timedelta, timezone +from types import SimpleNamespace from unittest.mock import AsyncMock, patch from uuid import uuid4 @@ -101,9 +102,9 @@ async def test_disabled_connections_are_skipped(db_session, connection): assert await _notices_for(db_session, connection) == [] -def _notice(*, announce: bool) -> ExpiryNotice: +def _notice(*, announce: bool, identity_provider_id=None) -> ExpiryNotice: return ExpiryNotice( - identity_provider_id=uuid4(), + identity_provider_id=identity_provider_id or uuid4(), identity_provider_name="Contoso Entra", vendor="entra", organization_id=uuid4(), @@ -115,38 +116,71 @@ def _notice(*, announce: bool) -> ExpiryNotice: ) +class _Notifications: + """The notification service's admin notifications, in memory.""" + + def __init__(self) -> None: + self.raised: dict[str, dict] = {} + self.dismissed: list[str] = [] + + async def find_admin_notification_by_metadata(self, metadata, category=None): + for notification_id, held in self.raised.items(): + if all(held.get(key) == value for key, value in metadata.items()): + return SimpleNamespace(id=notification_id) + return None + + async def create_notification(self, *, user_id, request, for_admins, initial_status): + self.raised[f"n{len(self.raised)}"] = request.metadata + + async def dismiss_notification(self, notification_id, user_id): + self.dismissed.append(notification_id) + del self.raised[notification_id] + + @pytest.mark.asyncio async def test_delivery_announces_new_stages_and_keeps_one_notification(): - service = AsyncMock() - service.find_admin_notification_by_title.side_effect = [None, object()] + service = _Notifications() emit = AsyncMock() + connection_id = uuid4() with ( patch("src.services.notification_service.get_notification_service", return_value=service), patch("src.services.events.builtins.emit_identity_provider_token_expiry", emit), ): - await deliver_expiry_notices([_notice(announce=True), _notice(announce=False)]) + await deliver_expiry_notices( + [ + _notice(announce=True, identity_provider_id=connection_id), + _notice(announce=False, identity_provider_id=connection_id), + ] + ) assert emit.await_count == 1 assert emit.await_args.kwargs["stage"] == "expiring" # The second notice's notification already exists, so only one is raised. - assert service.create_notification.await_count == 1 - metadata = service.create_notification.await_args.kwargs["request"].metadata + assert len(service.raised) == 1 + (metadata,) = service.raised.values() assert metadata["action_url"].startswith("/users/identity-providers/") @pytest.mark.asyncio -async def test_rotation_dismisses_outstanding_warnings(): +async def test_same_named_connections_keep_their_own_warnings(): from src.services.identity_provisioning.expiry import clear_expiry_notifications - expired = AsyncMock() - expired.id = "n-expired" - service = AsyncMock() - service.find_admin_notification_by_title.side_effect = ( - lambda title, category: expired if title.startswith("SCIM Token Expired") else None - ) - with patch( - "src.services.notification_service.get_notification_service", return_value=service + service = _Notifications() + rotated, other = uuid4(), uuid4() + with ( + patch("src.services.notification_service.get_notification_service", return_value=service), + patch("src.services.events.builtins.emit_identity_provider_token_expiry", AsyncMock()), ): - await clear_expiry_notifications("Contoso Entra") - - service.dismiss_notification.assert_awaited_once_with("n-expired", user_id="system") + # Two organizations name their connection the same. + await deliver_expiry_notices( + [ + _notice(announce=True, identity_provider_id=rotated), + _notice(announce=True, identity_provider_id=other), + ] + ) + assert len(service.raised) == 2 + + await clear_expiry_notifications(rotated) + + (remaining,) = service.raised.values() + assert remaining["identity_provider_id"] == str(other) diff --git a/api/tests/unit/services/test_identity_provisioning_core.py b/api/tests/unit/services/test_identity_provisioning_core.py index 22bd535e83..4d4d49c932 100644 --- a/api/tests/unit/services/test_identity_provisioning_core.py +++ b/api/tests/unit/services/test_identity_provisioning_core.py @@ -247,3 +247,203 @@ async def test_deleting_a_user_who_signed_in_disables_and_unlinks(setup): ) is None ) + + +async def _privileged_role(session) -> Role: + from src.models.orm.users import RolePermission + + role = Role(name=f"Helpdesk Admin {uuid4().hex[:8]}", created_by="test") + session.add(role) + await session.flush() + session.add(RolePermission(role_id=role.id, permission="users.readwrite")) + await session.flush() + return role + + +async def _make_privileged(session, user_id) -> None: + role = await _privileged_role(session) + session.add(UserRole(user_id=user_id, role_id=role.id, assigned_by="test")) + await session.flush() + + +@pytest.mark.asyncio +async def test_refuses_to_adopt_a_privileged_user(setup): + session = setup["session"] + email = f"admin@{setup['domain']}" + admin = User(email=email, organization_id=setup["organization"].id) + session.add(admin) + await session.flush() + await _make_privileged(session, admin.id) + + with pytest.raises(ProvisioningError) as raised: + await Provisioner(session, setup["connection"]).create_user(_record(email)) + + assert raised.value.kind is ProvisioningErrorKind.CONFLICT + + +@pytest.mark.asyncio +async def test_a_user_who_becomes_privileged_is_left_to_bifrost(setup): + session, connection = setup["session"], setup["connection"] + provisioner = Provisioner(session, connection) + link = await provisioner.create_user(_record(f"promoted@{setup['domain']}")) + group = await provisioner.create_group( + GroupRecord(display_name="Techs", external_id="g1", attributes={}), [link.id] + ) + await _map_and_apply(session, connection, group.id, []) + await _make_privileged(session, link.user_id) + + with pytest.raises(ProvisioningError) as raised: + await provisioner.replace_user(link, _record(f"promoted@{setup['domain']}", active=False)) + assert raised.value.kind is ProvisioningErrorKind.CONFLICT + with pytest.raises(ProvisioningError): + await provisioner.delete_user(link) + + # Losing the mapped group would deactivate anyone else. + await provisioner.set_group_members(group, []) + assert (await _user(session, link)).is_active + + +@pytest.mark.asyncio +async def test_a_mapped_role_that_turns_privileged_is_not_granted(setup): + from src.models.orm.audit import AuditLog + from src.models.orm.users import RolePermission + + session, connection, role = setup["session"], setup["connection"], setup["role"] + provisioner = Provisioner(session, connection) + first = await provisioner.create_user(_record(f"first@{setup['domain']}")) + group = await provisioner.create_group( + GroupRecord(display_name="Techs", external_id="g1", attributes={}), [first.id] + ) + await _map_and_apply(session, connection, group.id, [role.id]) + # The role gains a privileged permission after it was mapped. + session.add(RolePermission(role_id=role.id, permission="users.readwrite")) + await session.flush() + + later = await provisioner.create_user(_record(f"later@{setup['domain']}")) + await provisioner.set_group_members(group, [first.id, later.id]) + await Provisioner(session, connection).apply_pending([later.id]) + + assert ( + await session.scalar( + select(UserRole).where(UserRole.user_id == later.user_id, UserRole.role_id == role.id) + ) + is None + ) + refusal = await session.scalar( + select(AuditLog).where( + AuditLog.action == "identity_provider.role_grant_refused", + AuditLog.resource_id == later.user_id, + ) + ) + assert refusal is not None + + +@pytest.mark.asyncio +async def test_emails_match_without_regard_to_case(setup): + session = setup["session"] + stored = f"Mixed.Case@{setup['domain']}" + session.add(User(email=stored, organization_id=setup["other"].id)) + await session.flush() + + # The same address in another org, spelled in lowercase, is still taken. + with pytest.raises(ProvisioningError) as raised: + await Provisioner(session, setup["connection"]).create_user(_record(stored.lower())) + + assert raised.value.kind is ProvisioningErrorKind.CONFLICT + + +@pytest.mark.asyncio +async def test_user_names_are_unique_without_regard_to_case(setup): + session = setup["session"] + provisioner = Provisioner(session, setup["connection"]) + await provisioner.create_user(_record(f"pat@{setup['domain']}")) + duplicate = UserRecord( + user_name=f"PAT@{setup['domain']}", + email=f"pat.other@{setup['domain']}", + external_id="ext-other", + display_name="Pat Other", + active=True, + attributes={}, + ) + + with pytest.raises(ProvisioningError) as raised: + await provisioner.create_user(duplicate) + + assert raised.value.kind is ProvisioningErrorKind.CONFLICT + + +@pytest.mark.asyncio +async def test_a_replace_that_omits_active_keeps_the_user_inactive(setup): + session = setup["session"] + provisioner = Provisioner(session, setup["connection"]) + email = f"paused@{setup['domain']}" + link = await provisioner.create_user(_record(email, active=False)) + record = _record(email) + + await provisioner.replace_user( + link, + UserRecord( + user_name=record.user_name, + email=record.email, + external_id=record.external_id, + display_name=record.display_name, + active=None, + attributes=record.attributes, + ), + ) + await Provisioner(session, setup["connection"]).apply_pending([link.id]) + + assert not link.active + assert not (await _user(session, link)).is_active + + +@pytest.mark.asyncio +async def test_a_managed_user_cannot_be_moved_to_another_org(setup): + from shared.sdk_users import UserServiceError, bulk_update_users, update_user + from src.models.contracts.users import BulkUserOperation + from tests.helpers.authorization import admin_caller + + session = setup["session"] + link = await Provisioner(session, setup["connection"]).create_user( + _record(f"mover@{setup['domain']}") + ) + + with pytest.raises(UserServiceError) as raised: + await update_user( + session, + admin_caller(), + user_id=str(link.user_id), + fields={"organization_id"}, + organization_id=setup["other"].id, + ) + assert raised.value.status_code == 409 + assert "Contoso Entra" in raised.value.detail + + result = await bulk_update_users( + session, + admin_caller(), + BulkUserOperation( + user_ids=[link.user_id], operation="move_org", organization_id=setup["other"].id + ), + ) + assert [failure.user_id for failure in result.failed] == [link.user_id] + assert (await _user(session, link)).organization_id == setup["organization"].id + + +@pytest.mark.asyncio +async def test_a_role_with_privileged_permissions_cannot_be_mapped(setup): + from fastapi import HTTPException + + from src.services.identity_providers import set_group_mapping + from tests.helpers.authorization import admin_caller + + session, connection = setup["session"], setup["connection"] + group = await Provisioner(session, connection).create_group( + GroupRecord(display_name="Admins", external_id="g-admins", attributes={}), [] + ) + role = await _privileged_role(session) + + with pytest.raises(HTTPException) as raised: + await set_group_mapping(session, admin_caller(), connection.id, group.id, [role.id]) + + assert raised.value.status_code == 422 diff --git a/api/tests/unit/services/test_identity_provisioning_scim.py b/api/tests/unit/services/test_identity_provisioning_scim.py index 8050b625ad..eb1c6fb473 100644 --- a/api/tests/unit/services/test_identity_provisioning_scim.py +++ b/api/tests/unit/services/test_identity_provisioning_scim.py @@ -94,6 +94,50 @@ def test_patch_on_an_unknown_extension_is_kept_verbatim(): assert scim.user_record(resource, unknown).attributes[CUSTOM] == {"costCenter": "43"} +def test_a_replace_without_active_leaves_it_unsaid(): + resource, unknown = scim.parse_user( + {"schemas": [scim.USER_SCHEMA], "userName": "pat@contoso.example"} + ) + + assert scim.user_record(resource, unknown).active is None + + +def test_an_unsupported_verb_on_an_unknown_extension_changes_nothing(): + link = _link(**{CUSTOM: {"costCenter": "42"}}) + + with pytest.raises(ScimError) as raised: + scim.apply_user_patch( + link, _patch({"op": "merge", "path": f"{CUSTOM}:costCenter", "value": "43"}) + ) + + assert raised.value.status == 400 + assert link.attributes[CUSTOM] == {"costCenter": "42"} + + +@pytest.mark.parametrize( + "operation", + [ + {"op": "replace", "path": f"{CUSTOM}:costCenter", "value": "43"}, + {"op": "add", "value": {CUSTOM: {"costCenter": "43"}}}, + ], +) +def test_a_patch_into_an_extension_that_is_not_an_object_is_invalid(operation): + with pytest.raises(ScimError) as raised: + scim.apply_user_patch(_link(**{CUSTOM: "not an object"}), _patch(operation)) + + assert raised.value.status == 400 + + +def test_a_patch_that_removes_the_user_name_is_invalid(): + with pytest.raises(ScimError) as raised: + resource, unknown = scim.apply_user_patch( + _link(), _patch({"op": "remove", "path": "userName"}) + ) + scim.user_record(resource, unknown) + + assert raised.value.status == 400 + + def test_patch_cannot_change_the_id(): with pytest.raises(ScimError) as raised: scim.apply_user_patch(_link(), _patch({"op": "replace", "path": "id", "value": "x"})) diff --git a/client/src/pages/IdentityProviderDetail.test.tsx b/client/src/pages/IdentityProviderDetail.test.tsx index 81093ead2f..e6ddc40464 100644 --- a/client/src/pages/IdentityProviderDetail.test.tsx +++ b/client/src/pages/IdentityProviderDetail.test.tsx @@ -1,5 +1,5 @@ import { beforeEach, describe, expect, it, vi } from "vitest"; -import { Route, Routes } from "react-router-dom"; +import { Route, Routes, useLocation } from "react-router-dom"; import { renderWithProviders, screen } from "@/test-utils"; import { @@ -109,14 +109,26 @@ function provider(overrides: Partial = {}): IdentityProvider { }; } +/** Shows the current history entry's state, as a reload would see it. */ +function HistoryState() { + return ( + + {JSON.stringify(useLocation().state)} + + ); +} + function renderPage(state?: unknown) { return renderWithProviders( - - } - /> - , + <> + + } + /> + + + , { initialEntries: [ { pathname: "/users/identity-providers/idp-1", state }, @@ -171,6 +183,10 @@ describe("IdentityProviderDetail", () => { }); expect(screen.getByText("bfidp_one_time")).toBeInTheDocument(); + // A reload or back navigation can't show it again. + expect(screen.getByLabelText("History state")).toHaveTextContent( + "null", + ); await user.click( screen.getByRole("button", { name: "Dismiss secret" }), ); diff --git a/client/src/pages/IdentityProviderDetail.tsx b/client/src/pages/IdentityProviderDetail.tsx index 16036afb91..870dc87b93 100644 --- a/client/src/pages/IdentityProviderDetail.tsx +++ b/client/src/pages/IdentityProviderDetail.tsx @@ -1,4 +1,4 @@ -import { useState } from "react"; +import { useEffect, useState } from "react"; import { Link, useLocation, useNavigate, useParams } from "react-router-dom"; import { AlertCircle, ArrowLeft, Pencil, Trash2 } from "lucide-react"; import { toast } from "sonner"; @@ -100,11 +100,17 @@ export function IdentityProviderDetail() { } | null )?.issuedToken ?? null, ); - const dismissToken = () => { - setIssuedToken(null); - // Don't show the token again after a reload or back navigation. - navigate(location.pathname, { replace: true, state: null }); - }; + const hasTokenInHistory = Boolean( + (location.state as { issuedToken?: unknown } | null)?.issuedToken, + ); + useEffect(() => { + // The token now lives only in this page's state: take it out of the + // history entry so a reload or back navigation can't show it again. + if (hasTokenInHistory) { + navigate(location.pathname, { replace: true, state: null }); + } + }, [hasTokenInHistory, location.pathname, navigate]); + const dismissToken = () => setIssuedToken(null); const provider = query.data; if (query.isLoading) { diff --git a/docs/plans/2026-11-16-scim-identity-providers.md b/docs/plans/2026-11-16-scim-identity-providers.md index e142e25630..be3e52393d 100644 --- a/docs/plans/2026-11-16-scim-identity-providers.md +++ b/docs/plans/2026-11-16-scim-identity-providers.md @@ -80,6 +80,8 @@ Out of scope for v1: | 10 | Provision / update / deprovision / group-membership changes emit platform events. | | 11 | External (portal) users: a per-connection **Provision as External users** setting, off by default. It applies to users the connection creates; an adopted user keeps their current setting. An External user (`User.is_external`) keeps the same org cascade but is not covered by `access_level="authenticated"` sharing; they see only what a role or an explicit grant gives them. | | 12 | The provisioning core is protocol-independent. Connections have `source_type` = `scim` \| `google_directory`. The UI is the same for both: groups seen, "N/N mapped", last sync, credential expiry. | +| 13 | Privileged access is decided in Bifrost, never by a directory. A privileged user (`privileged_user_ids`, Platform Admin included) is not adopted; once a linked user becomes privileged, SCIM writes to them return `409` and the access pass leaves them alone. A role carrying privileged permissions can't be mapped, and is refused at grant time too (`identity_provider.role_grant_refused`) in case it gained them after it was mapped. | +| 14 | A user stays in their connection's organization. Moving an IdP-managed user to another organization (directly, in bulk, or by promoting them to Platform Admin) returns `409`; remove them from the directory first. Otherwise the connection would control, and grant its mapped roles to, a user in another organization. | ## Data model From 149c8becfa196c8076bfd47e02585a63be559465 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 22:40:20 -0500 Subject: [PATCH 15/26] Close the races and gaps from the second review round - Moves lock the user row before checking for a link, as adoption does, and check the move's real destination (an explicit organization wins over promotion). The provisioner also leaves alone a linked user who is no longer in its organization. - Privilege decisions read role permissions under a share lock that a permission edit (which locks the role for update) waits on, both for the user being changed and for each role being granted. - Adoption refuses to choose between accounts whose emails differ only in letter case. - Conflict logs carry the SQLSTATE and constraint, not the statement, its parameters or the key values. - Failed SCIM authentication is limited per connection rather than by X-Forwarded-For, which the client controls. - Rotation reads the token it replaces under the issuing lock, so a concurrent rotation's token is the one reported. - The expiry check raises its warnings under a key-share lock on the connections before committing, so a rotation can't land between the check and the warning; events still follow the commit. - user.groups_changed reports the user as the change leaves them. - Deeply nested bodies and non-object extension values answer 400, and a guard refusal during a mapping change answers 409. Adds two-session tests for adoption against a move, a grant against a permission edit, and two rotations at once. Co-Authored-By: Claude Opus 5.5 --- api/shared/sdk_users.py | 25 +- api/src/core/transaction_conflicts.py | 29 ++- .../identity_provider_token_expiry.py | 6 +- api/src/routers/scim.py | 30 ++- api/src/services/identity_providers.py | 23 +- .../services/identity_provisioning/core.py | 95 +++++-- .../services/identity_provisioning/expiry.py | 40 ++- .../services/identity_provisioning/scim.py | 13 +- .../services/identity_provisioning/tokens.py | 18 +- .../unit/routers/test_scim_authentication.py | 13 +- .../test_identity_provider_concurrency.py | 239 ++++++++++++++++++ .../test_identity_provider_token_expiry.py | 17 +- .../test_identity_provisioning_core.py | 78 ++++++ .../test_identity_provisioning_scim.py | 28 ++ 14 files changed, 570 insertions(+), 84 deletions(-) create mode 100644 api/tests/unit/services/test_identity_provider_concurrency.py diff --git a/api/shared/sdk_users.py b/api/shared/sdk_users.py index 18f1917e0a..a9d1d1942b 100644 --- a/api/shared/sdk_users.py +++ b/api/shared/sdk_users.py @@ -668,14 +668,25 @@ async def update_user( and db_user.id in await _platform_operator_holders(session, [db_user.id]) ): raise UserServiceError(409, OPERATOR_MOVE_MESSAGE) - # Promotion moves the user to the provider organization. - destination = PROVIDER_ORG_ID if is_superuser else organization_id - if destination is not None and destination != db_user.organization_id: + # Where the writes below leave the user: an explicit organization wins + # over promotion, which moves them to the provider organization. + destination = ( + organization_id + if organization_id is not None + else PROVIDER_ORG_ID if is_superuser else db_user.organization_id + ) + if destination != db_user.organization_id: + from src.models import User as UserORM from src.services.identity_providers import ( managed_user_move_message, managing_connections, ) + # Adoption links a user while holding this row lock, so a link + # committed by then is seen and a later one sees the new org. + await session.execute( + select(UserORM.id).where(UserORM.id == db_user.id).with_for_update() + ) managing = await managing_connections(session, [db_user.id]) if db_user.id in managing: raise UserServiceError(409, managed_user_move_message(managing[db_user.id])) @@ -914,6 +925,14 @@ async def bulk_update_users( managing_connections, ) + # Lock the users first, in id order, as adoption does before linking: + # a link committed by then is seen, and a later adoption sees the move. + await session.execute( + select(UserORM.id) + .where(UserORM.id.in_(list(users_by_id))) + .order_by(UserORM.id) + .with_for_update() + ) managing = await managing_connections(session, list(users_by_id)) else: managing = {} diff --git a/api/src/core/transaction_conflicts.py b/api/src/core/transaction_conflicts.py index e49cc646f0..25c5efce6b 100644 --- a/api/src/core/transaction_conflicts.py +++ b/api/src/core/transaction_conflicts.py @@ -25,6 +25,21 @@ logger = logging.getLogger(__name__) +def _log(kind: str, request: Request, exc: Exception) -> None: + """Log what happened without the statement, its parameters or the key + values in PostgreSQL's message: those carry directory data.""" + error = getattr(exc, "orig", None) or exc + cause = error.__cause__ or error + logger.warning( + "%s on %s %s; nothing was saved (sqlstate=%s, constraint=%s)", + kind, + request.method, + request.url.path, + getattr(error, "sqlstate", None), + getattr(cause, "constraint_name", None), + ) + + def conflict_aware_route( on_conflict: Callable[[], Exception], *, @@ -44,21 +59,11 @@ async def route(request: Request) -> Response: return await handler(request) except Exception as exc: if on_duplicate is not None and is_unique_violation(exc): - logger.warning( - "Unique violation on %s %s; nothing was saved: %s", - request.method, - request.url.path, - exc, - ) + _log("Unique violation", request, exc) raise on_duplicate() from None if not is_transaction_conflict(exc): raise - logger.warning( - "Transaction conflict on %s %s; nothing was saved: %s", - request.method, - request.url.path, - exc, - ) + _log("Transaction conflict", request, exc) raise on_conflict() from None return route diff --git a/api/src/jobs/schedulers/identity_provider_token_expiry.py b/api/src/jobs/schedulers/identity_provider_token_expiry.py index 3f47f058d1..b287c4a1bf 100644 --- a/api/src/jobs/schedulers/identity_provider_token_expiry.py +++ b/api/src/jobs/schedulers/identity_provider_token_expiry.py @@ -5,8 +5,9 @@ from src.core.database import get_session_factory from src.services.identity_provisioning.expiry import ( + announce_expiry_events, collect_expiry_notices, - deliver_expiry_notices, + raise_expiry_notifications, ) logger = logging.getLogger(__name__) @@ -17,8 +18,9 @@ async def check_identity_provider_token_expiry() -> dict: session_factory = get_session_factory() async with session_factory() as db: notices = await collect_expiry_notices(db, datetime.now(timezone.utc)) + await raise_expiry_notifications(notices) await db.commit() - await deliver_expiry_notices(notices) + await announce_expiry_events(notices) summary = { "expiring": sum(1 for n in notices if n.stage == "expiring"), "expired": sum(1 for n in notices if n.stage == "expired"), diff --git a/api/src/routers/scim.py b/api/src/routers/scim.py index 6a1bae0a78..df3d2bcb91 100644 --- a/api/src/routers/scim.py +++ b/api/src/routers/scim.py @@ -18,7 +18,7 @@ from sqlalchemy import func, select from src.core.db_deps import DbSession -from src.core.rate_limit import RateLimiter, get_client_ip +from src.core.rate_limit import RateLimiter from src.core.transaction_conflicts import conflict_aware_route from src.models.orm.identity_providers import ( IdentityProvider, @@ -55,8 +55,9 @@ def _duplicate() -> ScimError: # Generous: a large Entra or Okta cycle sends bursts of user and membership # changes. Applied per authenticated connection. _RATE_LIMIT = RateLimiter(max_requests=1200, window_seconds=60) -# Requests with a bad token are limited per client, apart from the -# connection's own quota, so they can't crowd out the identity provider. +# Requests with a bad token have their own quota per connection, so they +# can't crowd out the identity provider's valid token. Keyed by the URL's +# connection, not the client address: X-Forwarded-For is client-controlled. _AUTH_FAILURE_LIMIT = RateLimiter(max_requests=60, window_seconds=60) _MAX_BODY_BYTES = 1024 * 1024 @@ -95,7 +96,7 @@ async def scim_connection( else None ) if connection is None or connection.source_type != "scim": - await _limit(_AUTH_FAILURE_LIMIT, "scim_auth_failure", get_client_ip(request)) + await _limit(_AUTH_FAILURE_LIMIT, "scim_auth_failure", identity_provider_id) raise ScimError(401, "Invalid or expired token") await _limit(_RATE_LIMIT, "scim", str(connection.id)) return connection @@ -124,10 +125,29 @@ async def _json_body(request: Request) -> Any: if len(raw) > _MAX_BODY_BYTES: raise ScimError(413, "The request body is too large", "tooLarge") try: - return json.loads(raw) + body = json.loads(raw) # RecursionError: nesting deeper than the parser can follow. except (ValueError, RecursionError): raise ScimError(400, "The request body is not valid JSON", "invalidSyntax") from None + if _too_deep(body): + raise ScimError(400, "The request body is nested too deeply", "invalidSyntax") + return body + + +# SCIM resources nest a few levels; the parsing and copying that follow +# recurse, so refuse far deeper input here. +_MAX_JSON_DEPTH = 32 + + +def _too_deep(value: Any) -> bool: + stack = [(value, 1)] + while stack: + item, depth = stack.pop() + if depth > _MAX_JSON_DEPTH: + return True + children = item.values() if isinstance(item, dict) else item if isinstance(item, list) else () + stack.extend((child, depth + 1) for child in children) + return False # Past any real directory; keeps the offset inside PostgreSQL's bigint. diff --git a/api/src/services/identity_providers.py b/api/src/services/identity_providers.py index d90e2027c6..caa1e403de 100644 --- a/api/src/services/identity_providers.py +++ b/api/src/services/identity_providers.py @@ -51,7 +51,11 @@ ) from src.services.authorization.privilege import is_privileged_principal from src.services.identity_provisioning import domains, tokens -from src.services.identity_provisioning.core import UNMAPPABLE_ROLE_IDS, Provisioner +from src.services.identity_provisioning.core import ( + UNMAPPABLE_ROLE_IDS, + ProvisioningError, + Provisioner, +) from src.services.identity_provisioning.expiry import clear_expiry_notifications def scim_base_url(identity_provider_id: UUID) -> str: @@ -382,6 +386,9 @@ async def rotate_token( connection = await _connection( session, caller, identity_provider_id, operation="identityproviders.token.rotate" ) + # Read the token being replaced under the lock issuing takes, so a + # concurrent rotation's token is the one reported. + await tokens.lock_tokens(session, connection.id) previous = await tokens.current_token(session, connection.id) raw, token = await tokens.issue_token( session, connection, lifetime_days=lifetime_days, created_by=caller.principal.email @@ -507,6 +514,16 @@ async def _authorize_mapped_roles( raise HTTPException(exc.status_code, exc.detail) from None +async def _reconcile_mapping_change(provisioner: Provisioner) -> None: + """Apply a mapping change to every linked user. An account guard can + refuse a resulting deactivation (the SSO recovery admin, say); that + refusal answers the mapping change with 409.""" + try: + await provisioner.reconcile_all() + except ProvisioningError as exc: + raise HTTPException(409, exc.detail) from None + + async def set_group_mapping( session: AsyncSession, caller: Caller, @@ -554,7 +571,7 @@ async def set_group_mapping( "role_ids": sorted(str(r) for r in wanted), }, ) - await provisioner.reconcile_all() + await _reconcile_mapping_change(provisioner) (response,) = [g for g in await list_groups(session, caller, connection.id) if g.id == group.id] return response, provisioner @@ -580,7 +597,7 @@ async def delete_group_mapping( resource_id=connection.id, details={"group": group.display_name}, ) - await provisioner.reconcile_all() + await _reconcile_mapping_change(provisioner) return provisioner diff --git a/api/src/services/identity_provisioning/core.py b/api/src/services/identity_provisioning/core.py index 2dad3d9065..7ac51b0d34 100644 --- a/api/src/services/identity_provisioning/core.py +++ b/api/src/services/identity_provisioning/core.py @@ -64,7 +64,7 @@ IdentityProviderGroupRole, IdentityProviderUser, ) -from src.models.orm.users import User, UserRole +from src.models.orm.users import Role, User, UserRole from src.services.audit_context import ActorContext from src.services.identity_provisioning.domains import email_domain, verified_domains @@ -81,6 +81,7 @@ "This user holds privileged access, which is managed in Bifrost, not by an identity provider" ) PRIVILEGED_ROLE_MESSAGE = "Roles with privileged permissions aren't granted by identity providers" +OTHER_ORG_USER_MESSAGE = "This user no longer belongs to this connection's organization" class ProvisioningErrorKind(enum.Enum): @@ -246,12 +247,24 @@ async def create_user(self, record: UserRecord) -> IdentityProviderUser: await self._require_claimed_domain(email) # Lock a user being adopted before deciding it may be. - user = await self.session.scalar( - select(User) - .where(func.lower(User.email) == email) - .with_for_update() - .execution_options(populate_existing=True) - ) + matches = ( + await self.session.scalars( + select(User) + .where(func.lower(User.email) == email) + .order_by(User.id) + .with_for_update() + .execution_options(populate_existing=True) + ) + ).all() + if len(matches) > 1: + # Accounts that differ only in letter case: adopting either + # would be a guess. + raise ProvisioningError( + ProvisioningErrorKind.CONFLICT, + "Several accounts share this email in different letter cases; " + "resolve them in Bifrost first", + ) + user = matches[0] if matches else None created_user = user is None link_active = record.active is not False if user is None: @@ -291,7 +304,7 @@ async def replace_user( await self.hold_policy() await self._refuse_duplicate_link(record, exclude_link_id=link.id) user = await self._lock_user(link.user_id) - await self._refuse_privileged(user) + await self._refuse_unmanaged(user) email = normalize_email(record.email) if email != user.email: await self._require_claimed_domain(email) @@ -318,7 +331,7 @@ async def replace_user( async def delete_user(self, link: IdentityProviderUser) -> None: await self.hold_policy() user = await self._lock_user(link.user_id) - await self._refuse_privileged(user) + await self._refuse_unmanaged(user) if link.created_user and user.last_login is None: from shared.run_cancellation import cancel_unstarted_runs_for_deleted_user @@ -526,11 +539,11 @@ async def _reconcile_access( ).all(): memberships.setdefault(link_id, set()).add(group_id) - privileged = await self._privileged(users.keys()) + unmanaged = await self._unmanaged(users.values()) - now = datetime.now(timezone.utc) - for link in sorted(links, key=lambda link: link.user_id): - user = users[link.user_id] + def queue_group_event(user: User, link: IdentityProviderUser) -> None: + # Built once the user's access is settled, so it reports the + # user as the change leaves them. if group is not None and group_changes and link.id in group_changes: self.pending_events.append( ( @@ -545,10 +558,15 @@ async def _reconcile_access( }, ) ) - if user.id in privileged: + + now = datetime.now(timezone.utc) + for link in sorted(links, key=lambda link: link.user_id): + user = users[link.user_id] + if user.id in unmanaged: # Membership is still recorded, but the directory decides - # nothing about a privileged user's access. + # nothing about this user's access. link.access_pending_since = None + queue_group_event(user, link) continue mapped = memberships.get(link.id, set()) & mapped_roles.keys() desired_roles: set[UUID] = set() @@ -580,6 +598,7 @@ async def _reconcile_access( await self.session.flush() self._mark_stale(user.id, change.roles_added | change.roles_removed) + queue_group_event(user, link) if change.activated or change.roles_added or change.roles_removed: self.pending_events.append( ( @@ -655,7 +674,9 @@ async def _grant_roles(self, user: User, desired: set[UUID]) -> set[UUID]: added: set[UUID] = set() missing = await self._missing_roles(user, desired) # A mapped role checked at mapping time can gain privileged - # permissions later, so check again at every grant. + # permissions later, so check again at every grant, under the lock + # a permission edit waits on. + await self._lock_roles_shared(missing) roles = await load_roles(self.session, missing) for role_id in sorted(missing): if role_id in roles and is_privileged_principal(roles[role_id].permissions): @@ -790,7 +811,7 @@ async def _require_adoptable(self, user: User) -> None: raise ProvisioningError( ProvisioningErrorKind.CONFLICT, "This email belongs to a system account" ) - await self._refuse_privileged(user) + await self._refuse_unmanaged(user) linked = await self.session.scalar( select(IdentityProviderUser.identity_provider_id).where( IdentityProviderUser.user_id == user.id @@ -802,14 +823,42 @@ async def _require_adoptable(self, user: User) -> None: "This user is already managed by an identity provider", ) - async def _privileged(self, user_ids: Iterable[UUID]) -> frozenset[UUID]: - from src.services.authorization.enforce import privileged_user_ids + async def _lock_roles_shared(self, role_ids: Iterable[UUID]) -> None: + """Share-lock role rows before reading their permissions. A + permission edit locks its role for update first, so it waits for + this decision rather than changing the role under it.""" + ids = set(role_ids) + if ids: + await self.session.execute( + select(Role.id).where(Role.id.in_(ids)).order_by(Role.id).with_for_update(read=True) + ) - return await privileged_user_ids(self.session, list(user_ids)) + async def _unmanaged(self, users: Iterable[User]) -> dict[UUID, str]: + """``user_id -> reason`` for the locked ``users`` this connection must + leave alone: privileged ones, and any no longer in its organization + (moves are refused, so this only guards the invariant).""" + from src.services.authorization.enforce import privileged_user_ids - async def _refuse_privileged(self, user: User) -> None: - if user.is_superuser or user.id in await self._privileged([user.id]): - raise ProvisioningError(ProvisioningErrorKind.CONFLICT, PRIVILEGED_USER_MESSAGE) + users = list(users) + held = set( + await self.session.scalars( + select(UserRole.role_id).where(UserRole.user_id.in_([u.id for u in users])) + ) + ) + await self._lock_roles_shared(held | {u.base_role_id for u in users if u.base_role_id}) + privileged = await privileged_user_ids(self.session, [u.id for u in users]) + unmanaged: dict[UUID, str] = {} + for user in users: + if user.organization_id != self.connection.organization_id: + unmanaged[user.id] = OTHER_ORG_USER_MESSAGE + elif user.is_superuser or user.id in privileged: + unmanaged[user.id] = PRIVILEGED_USER_MESSAGE + return unmanaged + + async def _refuse_unmanaged(self, user: User) -> None: + reason = (await self._unmanaged([user])).get(user.id) + if reason is not None: + raise ProvisioningError(ProvisioningErrorKind.CONFLICT, reason) async def _refuse_duplicate_link( self, record: UserRecord, *, exclude_link_id: UUID | None diff --git a/api/src/services/identity_provisioning/expiry.py b/api/src/services/identity_provisioning/expiry.py index 6c6249b1fe..a1b7650840 100644 --- a/api/src/services/identity_provisioning/expiry.py +++ b/api/src/services/identity_provisioning/expiry.py @@ -58,7 +58,19 @@ class ExpiryNotice: async def collect_expiry_notices(session: AsyncSession, now: datetime) -> list[ExpiryNotice]: """Each enabled connection whose current token needs attention. Marks - newly reached stages on the tokens; the caller commits, then delivers.""" + newly reached stages on the tokens. + + The caller raises the notifications, then commits, then announces the + events. The connection rows stay key-share locked until the commit, so + a rotation (which locks them for update) either lands before this reads + the tokens or waits until the warnings are up, then dismisses them. + """ + await session.execute( + select(IdentityProvider.id) + .where(IdentityProvider.is_enabled.is_(True)) + .order_by(IdentityProvider.id) + .with_for_update(key_share=True) + ) rows = ( await session.execute( select(IdentityProvider, IdentityProviderToken, Organization.name) @@ -125,17 +137,11 @@ async def clear_expiry_notifications(identity_provider_id: UUID) -> None: await service.dismiss_notification(existing.id, user_id="system") -async def deliver_expiry_notices(notices: list[ExpiryNotice]) -> None: - """Send events for new stages and keep admin notifications up.""" - from src.models.contracts.notifications import ( - NotificationCategory, - NotificationCreate, - NotificationStatus, - ) +async def announce_expiry_events(notices: list[ExpiryNotice]) -> None: + """Send an event for each newly reached stage, after the commit that + recorded it.""" from src.services.events.builtins import emit_identity_provider_token_expiry - from src.services.notification_service import get_notification_service - service = get_notification_service() for notice in notices: if notice.announce: await emit_identity_provider_token_expiry( @@ -150,6 +156,20 @@ async def deliver_expiry_notices(notices: list[ExpiryNotice]) -> None: expires_at=notice.expires_at, stage=notice.stage, ) + + +async def raise_expiry_notifications(notices: list[ExpiryNotice]) -> None: + """Keep one admin notification up per connection and stage (before the + commit; see :func:`collect_expiry_notices`).""" + from src.models.contracts.notifications import ( + NotificationCategory, + NotificationCreate, + NotificationStatus, + ) + from src.services.notification_service import get_notification_service + + service = get_notification_service() + for notice in notices: title = notification_title(notice) key = _notification_key(notice.identity_provider_id, notice.stage) if await service.find_admin_notification_by_metadata( diff --git a/api/src/services/identity_provisioning/scim.py b/api/src/services/identity_provisioning/scim.py index 6752cff919..c405ebabcd 100644 --- a/api/src/services/identity_provisioning/scim.py +++ b/api/src/services/identity_provisioning/scim.py @@ -201,6 +201,12 @@ def _extension_object(unknown: dict[str, Any], schema: str) -> dict[str, Any]: return target +def _extension_value(schema: str, value: Any) -> dict[str, Any]: + if not isinstance(value, dict): + raise ScimError(400, f"A value for the extension '{schema}' must be an object", "invalidValue") + return value + + def _apply_unknown_extension_ops( operations: list[dict[str, Any]], unknown: dict[str, Any] ) -> list[dict[str, Any]]: @@ -229,15 +235,14 @@ def _apply_unknown_extension_ops( unknown.pop(schema, None) elif attribute: target[attribute] = value - elif isinstance(value, dict): - target.update(value) + else: + target.update(_extension_value(schema, value)) continue if not path and isinstance(value, dict): known_value = {} for key, item in value.items(): if key.startswith("urn:") and key not in KNOWN_USER_SCHEMAS: - if isinstance(item, dict): - _extension_object(unknown, key).update(item) + _extension_object(unknown, key).update(_extension_value(key, item)) else: known_value[key] = item if not known_value: diff --git a/api/src/services/identity_provisioning/tokens.py b/api/src/services/identity_provisioning/tokens.py index 23d7f358ba..95afd90099 100644 --- a/api/src/services/identity_provisioning/tokens.py +++ b/api/src/services/identity_provisioning/tokens.py @@ -53,6 +53,17 @@ async def current_token( ) +async def lock_tokens(session: AsyncSession, identity_provider_id: UUID) -> None: + """Serialize token changes for one connection on its row. Otherwise two + rotations at once each shorten only the tokens they can see, and both + new tokens keep their full lifetime.""" + await session.execute( + select(IdentityProvider.id) + .where(IdentityProvider.id == identity_provider_id) + .with_for_update() + ) + + async def issue_token( session: AsyncSession, connection: IdentityProvider, @@ -63,12 +74,7 @@ async def issue_token( """Create a new token; earlier live tokens keep working for the overlap.""" if lifetime_days not in ALLOWED_LIFETIME_DAYS: raise ValueError(f"lifetime_days must be one of {ALLOWED_LIFETIME_DAYS}") - # Rotations of one connection serialize here. Otherwise two at once - # each shorten only the tokens they can see, and both new tokens keep - # their full lifetime. - await session.execute( - select(IdentityProvider.id).where(IdentityProvider.id == connection.id).with_for_update() - ) + await lock_tokens(session, connection.id) now = datetime.now(timezone.utc) overlap_end = now + ROTATION_OVERLAP await session.execute( diff --git a/api/tests/unit/routers/test_scim_authentication.py b/api/tests/unit/routers/test_scim_authentication.py index ed05865c00..ce29a88298 100644 --- a/api/tests/unit/routers/test_scim_authentication.py +++ b/api/tests/unit/routers/test_scim_authentication.py @@ -11,27 +11,24 @@ def _request(token: str) -> SimpleNamespace: - return SimpleNamespace( - headers={"authorization": f"Bearer {token}"}, - client=SimpleNamespace(host="203.0.113.9"), - ) + return SimpleNamespace(headers={"authorization": f"Bearer {token}"}) @pytest.mark.asyncio -async def test_a_bad_token_counts_against_the_client_not_the_connection(): +async def test_a_bad_token_counts_against_failures_not_the_connections_quota(): + connection_id = str(uuid4()) connection_quota, failure_quota = AsyncMock(), AsyncMock() with ( patch.object(scim_router._RATE_LIMIT, "check", connection_quota), patch.object(scim_router._AUTH_FAILURE_LIMIT, "check", failure_quota), patch.object(scim_router.tokens, "authenticate", AsyncMock(return_value=None)), - patch.object(scim_router, "get_client_ip", return_value="203.0.113.9"), ): with pytest.raises(ScimError) as raised: - await scim_router.scim_connection(str(uuid4()), _request("bfidp_junk"), AsyncMock()) + await scim_router.scim_connection(connection_id, _request("bfidp_junk"), AsyncMock()) assert raised.value.status == 401 connection_quota.assert_not_awaited() - failure_quota.assert_awaited_once_with("scim_auth_failure", "203.0.113.9") + failure_quota.assert_awaited_once_with("scim_auth_failure", connection_id) @pytest.mark.asyncio diff --git a/api/tests/unit/services/test_identity_provider_concurrency.py b/api/tests/unit/services/test_identity_provider_concurrency.py new file mode 100644 index 0000000000..a1db7e6576 --- /dev/null +++ b/api/tests/unit/services/test_identity_provider_concurrency.py @@ -0,0 +1,239 @@ +"""Identity provisioning against a concurrent transaction, with committed data. + +Each test holds a lock in one session, starts the competing change in +another, checks that it waits, then lets the first commit and checks the +second decides from the committed state. +""" + +import asyncio +from datetime import datetime, timezone +from unittest.mock import AsyncMock, patch +from uuid import uuid4 + +import pytest +import pytest_asyncio +from sqlalchemy import delete, select + +from src.models.orm.identity_providers import ( + IdentityProvider, + IdentityProviderGroupMapping, + IdentityProviderGroupRole, + IdentityProviderToken, + IdentityProviderUser, + OrganizationDomain, +) +from src.models.orm.organizations import Organization +from src.models.orm.users import Role, RolePermission, User, UserRole +from src.services.identity_provisioning.core import ( + GroupRecord, + ProvisioningError, + ProvisioningErrorKind, + Provisioner, + UserRecord, +) +from tests.helpers.authorization import admin_caller + +# Long enough that a change which doesn't wait would have finished. +_STILL_WAITING = 0.5 + + +def _record(email: str) -> UserRecord: + return UserRecord( + user_name=email, + email=email, + external_id=f"ext-{email}", + display_name="Pat Example", + active=True, + attributes={}, + ) + + +@pytest_asyncio.fixture +async def committed(async_session_factory): + suffix = uuid4().hex[:10] + domain = f"race-{suffix}.example" + async with async_session_factory() as session: + organization = Organization(name=f"Race {suffix}", created_by="test") + other = Organization(name=f"Elsewhere {suffix}", created_by="test") + session.add_all([organization, other]) + await session.flush() + connection = IdentityProvider( + organization_id=organization.id, + name="Race Entra", + source_type="scim", + vendor="entra", + created_by="test", + ) + role = Role(name=f"Race Tech {suffix}", created_by="test") + session.add_all( + [ + connection, + role, + OrganizationDomain( + organization_id=organization.id, + domain=domain, + verification_token="t", + verified_at=datetime.now(timezone.utc), + created_by="test", + ), + ] + ) + await session.commit() + data = { + "domain": domain, + "organization_id": organization.id, + "other_id": other.id, + "connection_id": connection.id, + "role_id": role.id, + } + yield data + from src.models.orm.audit import AuditLog + + organization_ids = [organization.id, other.id] + async with async_session_factory() as session: + await session.execute(delete(User).where(User.email.like(f"%@{domain}"))) + await session.execute(delete(Role).where(Role.id == role.id)) + await session.execute(delete(AuditLog).where(AuditLog.organization_id.in_(organization_ids))) + await session.execute(delete(Organization).where(Organization.id.in_(organization_ids))) + await session.commit() + + +async def _waits(task: asyncio.Task) -> bool: + await asyncio.sleep(_STILL_WAITING) + return not task.done() + + +@pytest.mark.asyncio +async def test_adoption_after_a_committed_move_is_refused(async_session_factory, committed): + from shared.sdk_users import update_user + + email = f"mover@{committed['domain']}" + async with async_session_factory() as session: + user = User(email=email, organization_id=committed["organization_id"]) + session.add(user) + await session.commit() + user_id = user.id + + async def adopt() -> None: + async with async_session_factory() as session: + connection = await session.get(IdentityProvider, committed["connection_id"]) + assert connection is not None + await Provisioner(session, connection).create_user(_record(email)) + await session.commit() + + async with asyncio.timeout(30): + async with async_session_factory() as mover: + await update_user( + mover, + admin_caller(), + user_id=str(user_id), + fields={"organization_id"}, + organization_id=committed["other_id"], + ) + adoption = asyncio.create_task(adopt()) + assert await _waits(adoption) + await mover.commit() + with pytest.raises(ProvisioningError) as raised: + await adoption + + assert raised.value.kind is ProvisioningErrorKind.CONFLICT + async with async_session_factory() as session: + assert ( + await session.scalar( + select(IdentityProviderUser.id).where(IdentityProviderUser.user_id == user_id) + ) + is None + ) + + +@pytest.mark.asyncio +async def test_a_grant_waits_for_a_permission_edit_and_refuses_the_new_privilege( + async_session_factory, committed +): + from shared.account_guards import _lock_roles + + async with async_session_factory() as session: + connection = await session.get(IdentityProvider, committed["connection_id"]) + assert connection is not None + provisioner = Provisioner(session, connection) + group = await provisioner.create_group( + GroupRecord(display_name="Techs", external_id="g-race", attributes={}), [] + ) + session.add(IdentityProviderGroupMapping(group_id=group.id, created_by="test")) + await session.flush() + session.add(IdentityProviderGroupRole(group_id=group.id, role_id=committed["role_id"])) + await session.flush() + link = await provisioner.create_user(_record(f"hire@{committed['domain']}")) + await provisioner.set_group_members(group, [link.id]) + await session.commit() + link_id, user_id = link.id, link.user_id + + async def apply_grant() -> None: + async with async_session_factory() as session: + connection = await session.get(IdentityProvider, committed["connection_id"]) + assert connection is not None + await Provisioner(session, connection).apply_pending([link_id]) + await session.commit() + + async with asyncio.timeout(30): + async with async_session_factory() as editor: + # What a permission edit does: lock the role, then change it. + await _lock_roles(editor, {committed["role_id"]}) + editor.add(RolePermission(role_id=committed["role_id"], permission="users.readwrite")) + await editor.flush() + grant = asyncio.create_task(apply_grant()) + assert await _waits(grant) + await editor.commit() + await grant + + async with async_session_factory() as session: + assert ( + await session.scalar( + select(UserRole.role_id).where( + UserRole.user_id == user_id, UserRole.role_id == committed["role_id"] + ) + ) + is None + ) + + +@pytest.mark.asyncio +async def test_concurrent_rotations_leave_one_full_lifetime_token( + async_session_factory, committed +): + from src.services.identity_providers import rotate_token + + async def rotate(session): + with patch( + "src.services.identity_providers.clear_expiry_notifications", AsyncMock() + ): + return await rotate_token(session, admin_caller(), committed["connection_id"], 365) + + async def second_rotation(): + async with async_session_factory() as session: + issued = await rotate(session) + await session.commit() + return issued + + async with asyncio.timeout(30): + async with async_session_factory() as session: + first = await rotate(session) + second_task = asyncio.create_task(second_rotation()) + assert await _waits(second_task) + await session.commit() + second = await second_task + + async with async_session_factory() as session: + tokens = { + token.token_hint: token + for token in await session.scalars( + select(IdentityProviderToken).where( + IdentityProviderToken.identity_provider_id == committed["connection_id"] + ) + ) + } + # The second rotation shortened the first's token and reported it. + shortened = tokens[first.info.hint].expires_at + assert shortened < first.info.expires_at + assert second.previous_tokens_expire_at == shortened + assert tokens[second.info.hint].expires_at == second.info.expires_at diff --git a/api/tests/unit/services/test_identity_provider_token_expiry.py b/api/tests/unit/services/test_identity_provider_token_expiry.py index a8b096eebd..8fcc4adc6f 100644 --- a/api/tests/unit/services/test_identity_provider_token_expiry.py +++ b/api/tests/unit/services/test_identity_provider_token_expiry.py @@ -13,8 +13,9 @@ from src.models.orm.organizations import Organization from src.services.identity_provisioning.expiry import ( ExpiryNotice, + announce_expiry_events, collect_expiry_notices, - deliver_expiry_notices, + raise_expiry_notifications, token_stage, ) @@ -146,12 +147,12 @@ async def test_delivery_announces_new_stages_and_keeps_one_notification(): patch("src.services.notification_service.get_notification_service", return_value=service), patch("src.services.events.builtins.emit_identity_provider_token_expiry", emit), ): - await deliver_expiry_notices( - [ - _notice(announce=True, identity_provider_id=connection_id), - _notice(announce=False, identity_provider_id=connection_id), - ] - ) + notices = [ + _notice(announce=True, identity_provider_id=connection_id), + _notice(announce=False, identity_provider_id=connection_id), + ] + await raise_expiry_notifications(notices) + await announce_expiry_events(notices) assert emit.await_count == 1 assert emit.await_args.kwargs["stage"] == "expiring" @@ -172,7 +173,7 @@ async def test_same_named_connections_keep_their_own_warnings(): patch("src.services.events.builtins.emit_identity_provider_token_expiry", AsyncMock()), ): # Two organizations name their connection the same. - await deliver_expiry_notices( + await raise_expiry_notifications( [ _notice(announce=True, identity_provider_id=rotated), _notice(announce=True, identity_provider_id=other), diff --git a/api/tests/unit/services/test_identity_provisioning_core.py b/api/tests/unit/services/test_identity_provisioning_core.py index 4d4d49c932..e728a8a419 100644 --- a/api/tests/unit/services/test_identity_provisioning_core.py +++ b/api/tests/unit/services/test_identity_provisioning_core.py @@ -447,3 +447,81 @@ async def test_a_role_with_privileged_permissions_cannot_be_mapped(setup): await set_group_mapping(session, admin_caller(), connection.id, group.id, [role.id]) assert raised.value.status_code == 422 + + +@pytest.mark.asyncio +async def test_promotion_with_an_explicit_org_is_checked_at_that_org(setup): + from shared.sdk_users import UserServiceError, update_user + from tests.helpers.authorization import admin_caller + + session = setup["session"] + link = await Provisioner(session, setup["connection"]).create_user( + _record(f"promoted@{setup['domain']}") + ) + + # The explicit organization wins over promotion's provider org. + with pytest.raises(UserServiceError) as raised: + await update_user( + session, + admin_caller(), + user_id=str(link.user_id), + fields={"is_superuser", "organization_id"}, + is_superuser=True, + organization_id=setup["other"].id, + ) + + assert raised.value.status_code == 409 + + +@pytest.mark.asyncio +async def test_refuses_to_guess_between_accounts_that_differ_in_case(setup): + session = setup["session"] + email = f"twin@{setup['domain']}" + session.add_all( + [ + User(email=email, organization_id=setup["organization"].id), + User(email=email.upper(), organization_id=setup["organization"].id), + ] + ) + await session.flush() + + with pytest.raises(ProvisioningError) as raised: + await Provisioner(session, setup["connection"]).create_user(_record(email)) + + assert raised.value.kind is ProvisioningErrorKind.CONFLICT + + +@pytest.mark.asyncio +async def test_a_group_event_reports_the_user_as_the_change_leaves_them(setup): + session, connection = setup["session"], setup["connection"] + provisioner = Provisioner(session, connection) + link = await provisioner.create_user(_record(f"leaver@{setup['domain']}")) + group = await provisioner.create_group( + GroupRecord(display_name="Techs", external_id="g1", attributes={}), [link.id] + ) + await _map_and_apply(session, connection, group.id, []) + provisioner.pending_events.clear() + + await provisioner.set_group_members(group, []) + + (event,) = [body for topic, body in provisioner.pending_events if topic == "user.groups_changed"] + assert event["group"]["change"] == "removed" + assert event["user"]["is_active"] is False + + +@pytest.mark.asyncio +async def test_a_linked_user_outside_the_connections_org_is_left_alone(setup): + session = setup["session"] + provisioner = Provisioner(session, setup["connection"]) + email = f"strayed@{setup['domain']}" + link = await provisioner.create_user(_record(email)) + # Moves are refused; this guards the invariant if one happened anyway. + user = await _user(session, link) + user.organization_id = setup["other"].id + await session.flush() + + with pytest.raises(ProvisioningError) as raised: + await provisioner.replace_user(link, _record(email, active=False)) + + assert raised.value.kind is ProvisioningErrorKind.CONFLICT + assert (await _user(session, link)).is_active diff --git a/api/tests/unit/services/test_identity_provisioning_scim.py b/api/tests/unit/services/test_identity_provisioning_scim.py index eb1c6fb473..cafb3097b1 100644 --- a/api/tests/unit/services/test_identity_provisioning_scim.py +++ b/api/tests/unit/services/test_identity_provisioning_scim.py @@ -128,6 +128,34 @@ def test_a_patch_into_an_extension_that_is_not_an_object_is_invalid(operation): assert raised.value.status == 400 +@pytest.mark.parametrize( + "operation", + [ + {"op": "replace", "path": CUSTOM, "value": "not an object"}, + {"op": "add", "value": {CUSTOM: "not an object"}}, + ], +) +def test_a_whole_extension_value_must_be_an_object(operation): + with pytest.raises(ScimError) as raised: + scim.apply_user_patch(_link(**{CUSTOM: {"costCenter": "42"}}), _patch(operation)) + + assert raised.value.status == 400 + + +def test_a_body_nested_past_any_scim_resource_is_refused(): + from src.routers.scim import _too_deep + + shallow: dict = {"userName": "pat@contoso.example", "name": {"givenName": "Pat"}} + deep: dict = {} + level = deep + for _ in range(64): + level["x"] = {} + level = level["x"] + + assert not _too_deep(shallow) + assert _too_deep(deep) + + def test_a_patch_that_removes_the_user_name_is_invalid(): with pytest.raises(ScimError) as raised: resource, unknown = scim.apply_user_patch( From 050fa4176797e07ac95503edbc2f273369890ddf Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 23:16:39 -0500 Subject: [PATCH 16/26] Make privileged linked users revoke-only and fix lock order Owner decision: a directory can still take access away from a linked user who became privileged (deactivate, unlink, remove roles it granted), since Bifrost's privileged permission set is broad and offboarding must keep working. It still never adopts a privileged user, re-activates, renames or re-emails one, or grants a privileged role. From the adversarial review: - The users API locks the user row before any guard locks a role (single update, delete, and every bulk operation), the order identity provisioning uses, so an admin and a directory deactivating the same user can't deadlock. - The move test now holds adoption's lock with the link uncommitted, so it fails if the move doesn't wait and look; it covers bulk moves too. - A role deleted during a grant is skipped instead of failing on its foreign key. - Bad-token limits key on the parsed connection id, so other spellings of the same UUID share one quota. - Adoption that omits active keeps the user's current state. Co-Authored-By: Claude Opus 5.5 --- api/shared/sdk_users.py | 36 +++++---- api/src/routers/scim.py | 4 +- .../services/identity_provisioning/core.py | 48 +++++++++--- .../test_identity_provider_concurrency.py | 76 +++++++++++-------- .../test_identity_provisioning_core.py | 53 +++++++++++-- .../2026-11-16-scim-identity-providers.md | 2 +- 6 files changed, 153 insertions(+), 66 deletions(-) diff --git a/api/shared/sdk_users.py b/api/shared/sdk_users.py index a9d1d1942b..77a0fcf9ad 100644 --- a/api/shared/sdk_users.py +++ b/api/shared/sdk_users.py @@ -182,6 +182,14 @@ def __init__(self, status_code: int, detail: str) -> None: RESERVED_EMAIL_MESSAGE = "This email domain is reserved" +async def _lock_user_row(session: AsyncSession, user_id: UUID) -> None: + """Lock the user before any guard locks a role: identity provisioning + takes the same order (user, then roles), so the two can't deadlock.""" + from src.models import User as UserORM + + await session.execute(select(UserORM.id).where(UserORM.id == user_id).with_for_update()) + + def refuse_identity_management(user: "UserORM") -> None: """Identities aren't edited, deleted or bulk-changed as people.""" from shared.identities import is_identity @@ -653,6 +661,7 @@ async def update_user( if db_user.is_system: raise UserServiceError(403, "System user cannot be modified") refuse_identity_management(db_user) + await _lock_user_row(session, db_user.id) if email is not None: refuse_reserved_email(email) _authorize_update( @@ -676,17 +685,13 @@ async def update_user( else PROVIDER_ORG_ID if is_superuser else db_user.organization_id ) if destination != db_user.organization_id: - from src.models import User as UserORM from src.services.identity_providers import ( managed_user_move_message, managing_connections, ) - # Adoption links a user while holding this row lock, so a link - # committed by then is seen and a later one sees the new org. - await session.execute( - select(UserORM.id).where(UserORM.id == db_user.id).with_for_update() - ) + # Adoption links a user while holding the row lock taken above, so + # a link committed by then is seen and a later one sees the new org. managing = await managing_connections(session, [db_user.id]) if db_user.id in managing: raise UserServiceError(409, managed_user_move_message(managing[db_user.id])) @@ -819,6 +824,7 @@ async def delete_user( if db_user.is_system: raise UserServiceError(403, "System user cannot be deleted") refuse_identity_management(db_user) + await _lock_user_row(session, db_user.id) require_operation(caller, "users.delete", org_target(db_user.organization_id)) require_unprotected(caller, db_user.id in await privileged_user_ids(session, [db_user.id]), db_user.organization_id) @@ -911,7 +917,15 @@ async def bulk_update_users( succeeded: list[UUID] = [] failed: list[BulkUserFailure] = [] - rows = await session.execute(select(UserORM).where(UserORM.id.in_(request.user_ids))) + # Lock the users first, in id order: before any guard locks a role, as + # identity provisioning does, and before a move checks for a link, as + # adoption locks before linking. + rows = await session.execute( + select(UserORM) + .where(UserORM.id.in_(request.user_ids)) + .order_by(UserORM.id) + .with_for_update() + ) users_by_id = {u.id: u for u in rows.scalars().all()} held = await held_permissions_by_user(session, list(users_by_id)) operators = ( @@ -925,14 +939,6 @@ async def bulk_update_users( managing_connections, ) - # Lock the users first, in id order, as adoption does before linking: - # a link committed by then is seen, and a later adoption sees the move. - await session.execute( - select(UserORM.id) - .where(UserORM.id.in_(list(users_by_id))) - .order_by(UserORM.id) - .with_for_update() - ) managing = await managing_connections(session, list(users_by_id)) else: managing = {} diff --git a/api/src/routers/scim.py b/api/src/routers/scim.py index df3d2bcb91..14a490d4f3 100644 --- a/api/src/routers/scim.py +++ b/api/src/routers/scim.py @@ -96,7 +96,9 @@ async def scim_connection( else None ) if connection is None or connection.source_type != "scim": - await _limit(_AUTH_FAILURE_LIMIT, "scim_auth_failure", identity_provider_id) + # The parsed id: one connection, however its UUID is spelled. + key = str(connection_id) if connection_id is not None else "invalid" + await _limit(_AUTH_FAILURE_LIMIT, "scim_auth_failure", key) raise ScimError(401, "Invalid or expired token") await _limit(_RATE_LIMIT, "scim", str(connection.id)) return connection diff --git a/api/src/services/identity_provisioning/core.py b/api/src/services/identity_provisioning/core.py index 7ac51b0d34..49cc753bd6 100644 --- a/api/src/services/identity_provisioning/core.py +++ b/api/src/services/identity_provisioning/core.py @@ -10,9 +10,12 @@ another organization is a conflict. Emails and userNames match without regard to case. - Privileged access is decided in Bifrost, never by a directory: a - privileged user is not adopted, changed or deleted (a conflict), the - access pass leaves one alone, and a role carrying privileged permissions - is never granted. + privileged user is never adopted, a role carrying privileged permissions + is never granted, and a linked user who becomes privileged is revoke-only + (the directory can still deactivate or unlink them and remove roles it + granted, but not re-activate, rename, re-email or grant). +- A connection acts only on users in its organization; one that has left + it is left alone. - The identity provider owns name, email and active for linked users. - Roles come from group mappings. Provisioning only removes the role assignments it created (``UserRole.identity_provider_id``), so a manual @@ -266,7 +269,8 @@ async def create_user(self, record: UserRecord) -> IdentityProviderUser: ) user = matches[0] if matches else None created_user = user is None - link_active = record.active is not False + # Unsaid, a new user starts active and an adopted one keeps its state. + link_active = record.active if record.active is not None else (user is None or user.is_active) if user is None: # Insert in the final state: a new user carries no token, so the # insert leaves the authority epoch alone where a later flip of @@ -304,9 +308,11 @@ async def replace_user( await self.hold_policy() await self._refuse_duplicate_link(record, exclude_link_id=link.id) user = await self._lock_user(link.user_id) - await self._refuse_unmanaged(user) + revoke_only = await self._revoke_only(user) email = normalize_email(record.email) - if email != user.email: + if email != user.email.lower(): + if revoke_only: + raise ProvisioningError(ProvisioningErrorKind.CONFLICT, PRIVILEGED_USER_MESSAGE) await self._require_claimed_domain(email) taken = await self.session.scalar( select(User.id).where(func.lower(User.email) == email, User.id != user.id) @@ -331,7 +337,8 @@ async def replace_user( async def delete_user(self, link: IdentityProviderUser) -> None: await self.hold_policy() user = await self._lock_user(link.user_id) - await self._refuse_unmanaged(user) + # Removing access is always the directory's call, privileged or not. + await self._revoke_only(user) if link.created_user and user.last_login is None: from shared.run_cancellation import cancel_unstarted_runs_for_deleted_user @@ -562,12 +569,15 @@ def queue_group_event(user: User, link: IdentityProviderUser) -> None: now = datetime.now(timezone.utc) for link in sorted(links, key=lambda link: link.user_id): user = users[link.user_id] - if user.id in unmanaged: - # Membership is still recorded, but the directory decides - # nothing about this user's access. + reason = unmanaged.get(user.id) + if reason == OTHER_ORG_USER_MESSAGE: + # Membership is still recorded, but this connection decides + # nothing about a user outside its organization. link.access_pending_since = None queue_group_event(user, link) continue + # A privileged user's access only ever shrinks from here. + revoke_only = reason is not None mapped = memberships.get(link.id, set()) & mapped_roles.keys() desired_roles: set[UUID] = set() for group_id in mapped: @@ -580,7 +590,9 @@ def queue_group_event(user: User, link: IdentityProviderUser) -> None: await self._set_active(user, False) change.deactivated = True change.roles_removed = await self._remove_granted_roles(user.id, desired_roles) - if defer_grants: + if revoke_only: + link.access_pending_since = None + elif defer_grants: waiting = ( (desired_active and not user.is_active) or bool(await self._missing_roles(user, desired_roles)) @@ -679,7 +691,10 @@ async def _grant_roles(self, user: User, desired: set[UUID]) -> set[UUID]: await self._lock_roles_shared(missing) roles = await load_roles(self.session, missing) for role_id in sorted(missing): - if role_id in roles and is_privileged_principal(roles[role_id].permissions): + if role_id not in roles: + # Deleted since the mapping was read: nothing to grant. + continue + if is_privileged_principal(roles[role_id].permissions): await self._refuse_grant(user, role_id, PRIVILEGED_ROLE_MESSAGE) continue try: @@ -860,6 +875,15 @@ async def _refuse_unmanaged(self, user: User) -> None: if reason is not None: raise ProvisioningError(ProvisioningErrorKind.CONFLICT, reason) + async def _revoke_only(self, user: User) -> bool: + """Whether the directory may only take access away from this + linked user (they're privileged); refuses one outside the + connection's organization.""" + reason = (await self._unmanaged([user])).get(user.id) + if reason == OTHER_ORG_USER_MESSAGE: + raise ProvisioningError(ProvisioningErrorKind.CONFLICT, reason) + return reason is not None + async def _refuse_duplicate_link( self, record: UserRecord, *, exclude_link_id: UUID | None ) -> None: diff --git a/api/tests/unit/services/test_identity_provider_concurrency.py b/api/tests/unit/services/test_identity_provider_concurrency.py index a1db7e6576..46fa2ed4b7 100644 --- a/api/tests/unit/services/test_identity_provider_concurrency.py +++ b/api/tests/unit/services/test_identity_provider_concurrency.py @@ -19,15 +19,12 @@ IdentityProviderGroupMapping, IdentityProviderGroupRole, IdentityProviderToken, - IdentityProviderUser, OrganizationDomain, ) from src.models.orm.organizations import Organization from src.models.orm.users import Role, RolePermission, User, UserRole from src.services.identity_provisioning.core import ( GroupRecord, - ProvisioningError, - ProvisioningErrorKind, Provisioner, UserRecord, ) @@ -103,10 +100,39 @@ async def _waits(task: asyncio.Task) -> bool: return not task.done() -@pytest.mark.asyncio -async def test_adoption_after_a_committed_move_is_refused(async_session_factory, committed): - from shared.sdk_users import update_user +async def _move(session, user_id, organization_id, *, bulk: bool) -> bool: + """Move the user as an administrator would; whether it was allowed.""" + from shared.sdk_users import UserServiceError, bulk_update_users, update_user + from src.models.contracts.users import BulkUserOperation + + if bulk: + result = await bulk_update_users( + session, + admin_caller(), + BulkUserOperation( + user_ids=[user_id], operation="move_org", organization_id=organization_id + ), + ) + return not result.failed + try: + await update_user( + session, + admin_caller(), + user_id=str(user_id), + fields={"organization_id"}, + organization_id=organization_id, + ) + except UserServiceError as exc: + assert exc.status_code == 409 + return False + return True + +@pytest.mark.asyncio +@pytest.mark.parametrize("bulk", [False, True]) +async def test_a_move_waits_for_an_adoption_and_is_refused( + async_session_factory, committed, bulk +): email = f"mover@{committed['domain']}" async with async_session_factory() as session: user = User(email=email, organization_id=committed["organization_id"]) @@ -114,36 +140,26 @@ async def test_adoption_after_a_committed_move_is_refused(async_session_factory, await session.commit() user_id = user.id - async def adopt() -> None: + async def move() -> bool: async with async_session_factory() as session: - connection = await session.get(IdentityProvider, committed["connection_id"]) - assert connection is not None - await Provisioner(session, connection).create_user(_record(email)) + allowed = await _move(session, user_id, committed["other_id"], bulk=bulk) await session.commit() + return allowed async with asyncio.timeout(30): - async with async_session_factory() as mover: - await update_user( - mover, - admin_caller(), - user_id=str(user_id), - fields={"organization_id"}, - organization_id=committed["other_id"], - ) - adoption = asyncio.create_task(adopt()) - assert await _waits(adoption) - await mover.commit() - with pytest.raises(ProvisioningError) as raised: - await adoption + async with async_session_factory() as adopter: + connection = await adopter.get(IdentityProvider, committed["connection_id"]) + assert connection is not None + # Adoption holds the user's row lock with the link not yet committed. + await Provisioner(adopter, connection).create_user(_record(email)) + moving = asyncio.create_task(move()) + assert await _waits(moving) + await adopter.commit() + assert await moving is False - assert raised.value.kind is ProvisioningErrorKind.CONFLICT async with async_session_factory() as session: - assert ( - await session.scalar( - select(IdentityProviderUser.id).where(IdentityProviderUser.user_id == user_id) - ) - is None - ) + moved = await session.get(User, user_id) + assert moved is not None and moved.organization_id == committed["organization_id"] @pytest.mark.asyncio diff --git a/api/tests/unit/services/test_identity_provisioning_core.py b/api/tests/unit/services/test_identity_provisioning_core.py index e728a8a419..379c8826ee 100644 --- a/api/tests/unit/services/test_identity_provisioning_core.py +++ b/api/tests/unit/services/test_identity_provisioning_core.py @@ -282,25 +282,64 @@ async def test_refuses_to_adopt_a_privileged_user(setup): @pytest.mark.asyncio -async def test_a_user_who_becomes_privileged_is_left_to_bifrost(setup): +async def test_a_user_who_becomes_privileged_can_only_lose_access(setup): session, connection = setup["session"], setup["connection"] provisioner = Provisioner(session, connection) - link = await provisioner.create_user(_record(f"promoted@{setup['domain']}")) + email = f"promoted@{setup['domain']}" + link = await provisioner.create_user(_record(email)) group = await provisioner.create_group( GroupRecord(display_name="Techs", external_id="g1", attributes={}), [link.id] ) await _map_and_apply(session, connection, group.id, []) await _make_privileged(session, link.user_id) + # No new email for a privileged user. with pytest.raises(ProvisioningError) as raised: - await provisioner.replace_user(link, _record(f"promoted@{setup['domain']}", active=False)) + await provisioner.replace_user(link, _record(f"renamed@{setup['domain']}")) assert raised.value.kind is ProvisioningErrorKind.CONFLICT - with pytest.raises(ProvisioningError): - await provisioner.delete_user(link) - # Losing the mapped group would deactivate anyone else. + # Offboarding still works: losing the mapped group deactivates. await provisioner.set_group_members(group, []) - assert (await _user(session, link)).is_active + assert not (await _user(session, link)).is_active + + # But the directory can't turn them back on. + await provisioner.set_group_members(group, [link.id]) + await Provisioner(session, connection).apply_pending([link.id]) + assert not (await _user(session, link)).is_active + assert link.access_pending_since is None + + await provisioner.delete_user(link) + assert ( + await session.scalar( + select(IdentityProviderUser.id).where(IdentityProviderUser.id == link.id) + ) + is None + ) + + +@pytest.mark.asyncio +async def test_adoption_that_omits_active_keeps_the_users_state(setup): + session = setup["session"] + email = f"disabled@{setup['domain']}" + existing = User(email=email, organization_id=setup["organization"].id, is_active=False) + session.add(existing) + await session.flush() + record = _record(email) + + link = await Provisioner(session, setup["connection"]).create_user( + UserRecord( + user_name=record.user_name, + email=record.email, + external_id=record.external_id, + display_name=record.display_name, + active=None, + attributes=record.attributes, + ) + ) + await Provisioner(session, setup["connection"]).apply_pending([link.id]) + + assert not link.active + assert not (await _user(session, link)).is_active @pytest.mark.asyncio diff --git a/docs/plans/2026-11-16-scim-identity-providers.md b/docs/plans/2026-11-16-scim-identity-providers.md index be3e52393d..f8dd47b705 100644 --- a/docs/plans/2026-11-16-scim-identity-providers.md +++ b/docs/plans/2026-11-16-scim-identity-providers.md @@ -80,7 +80,7 @@ Out of scope for v1: | 10 | Provision / update / deprovision / group-membership changes emit platform events. | | 11 | External (portal) users: a per-connection **Provision as External users** setting, off by default. It applies to users the connection creates; an adopted user keeps their current setting. An External user (`User.is_external`) keeps the same org cascade but is not covered by `access_level="authenticated"` sharing; they see only what a role or an explicit grant gives them. | | 12 | The provisioning core is protocol-independent. Connections have `source_type` = `scim` \| `google_directory`. The UI is the same for both: groups seen, "N/N mapped", last sync, credential expiry. | -| 13 | Privileged access is decided in Bifrost, never by a directory. A privileged user (`privileged_user_ids`, Platform Admin included) is not adopted; once a linked user becomes privileged, SCIM writes to them return `409` and the access pass leaves them alone. A role carrying privileged permissions can't be mapped, and is refused at grant time too (`identity_provider.role_grant_refused`) in case it gained them after it was mapped. | +| 13 | Privileged access is decided in Bifrost, never by a directory. A privileged user (`privileged_user_ids`, Platform Admin included) is never adopted (`409`). A linked user who later becomes privileged is **revoke-only**: the directory can still deactivate or unlink them and remove roles it granted (offboarding always works), but can't re-activate, rename, re-email or grant them anything. A role carrying privileged permissions can't be mapped, and is refused at grant time too (`identity_provider.role_grant_refused`) in case it gained them after it was mapped. Note how broad `PRIVILEGED_PERMISSIONS` is (for example `tables.readwrite`, `executions.readwrite`, `code.execute`): roles carrying those are assigned in Bifrost. | | 14 | A user stays in their connection's organization. Moving an IdP-managed user to another organization (directly, in bulk, or by promoting them to Platform Admin) returns `409`; remove them from the directory first. Otherwise the connection would control, and grant its mapped roles to, a user in another organization. | ## Data model From 985ea00497c2b2aa348d71157d6215f4dfe124f6 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 23:27:18 -0500 Subject: [PATCH 17/26] Share-lock roles in read-only guards and close the last races From the third review round: - The assignment and External-transition guards only read role permissions, so they share-lock the role (shared.account_guards. share_lock_roles) instead of locking it for update. A permission edit still locks for update and waits for them; two readers no longer deadlock upgrading a lock both share, which External-user grants hit. Provisioning uses the same helper. - Updating or deleting a user rereads it under its row lock before authorizing, and bulk operations load their users locked, so a concurrent move can't be authorized against the old organization. - Mapping refuses a privileged role under a share lock held to the commit, so a role can't turn privileged between the check and the save. - The expiry check takes FOR KEY SHARE (what key_share alone didn't emit) and reads tokens only for the connections it locked. - POST and PUT refuse a non-object value for an unknown extension. Adds two-session tests: two grants of one role to External users don't wait for each other, and a mapping that races a permission edit is refused. Co-Authored-By: Claude Opus 5.5 --- api/shared/account_guards.py | 22 ++++- api/shared/sdk_users.py | 23 +++-- api/src/services/identity_providers.py | 16 +++- .../services/identity_provisioning/core.py | 17 +--- .../services/identity_provisioning/expiry.py | 16 ++-- .../services/identity_provisioning/scim.py | 2 +- .../test_identity_provider_concurrency.py | 88 +++++++++++++++++++ 7 files changed, 155 insertions(+), 29 deletions(-) diff --git a/api/shared/account_guards.py b/api/shared/account_guards.py index 40a7ebcb7c..1197cac0ed 100644 --- a/api/shared/account_guards.py +++ b/api/shared/account_guards.py @@ -87,7 +87,7 @@ async def assert_user_can_be_external(session: AsyncSession, user_id: UUID) -> N ) ).scalars(), } - await _lock_roles(session, role_ids) + await share_lock_roles(session, role_ids) if await _roles_are_privileged(session, role_ids): raise AccountGuardError(EXTERNAL_PRIVILEGED_ROLE_MESSAGE) @@ -108,7 +108,7 @@ async def assert_role_assignment_allowed_for_user( ) if not is_external: return - await _lock_roles(session, {role_id}) + await share_lock_roles(session, {role_id}) if await _roles_are_privileged(session, {role_id}): raise AccountGuardError(EXTERNAL_PRIVILEGED_ROLE_MESSAGE) @@ -145,6 +145,24 @@ async def _lock_roles(session: AsyncSession, role_ids: Iterable[UUID]) -> None: ) +async def share_lock_roles(session: AsyncSession, role_ids: Iterable[UUID]) -> None: + """Share-lock roles to read their permissions. A permission edit locks + its role for update (:func:`assert_role_permissions_allowed`), so it + waits for the reader; readers don't wait for each other, and a reader + never needs to upgrade, which two readers of one role would deadlock on. + """ + from src.models import Role as RoleORM + + ids = set(role_ids) + if ids: + await session.execute( + select(RoleORM.id) + .where(RoleORM.id.in_(ids)) + .order_by(RoleORM.id) + .with_for_update(read=True) + ) + + async def _external_role_holders(session: AsyncSession, role_id: UUID) -> set[UUID]: from src.models import User as UserORM from src.models import UserRole as UserRoleORM diff --git a/api/shared/sdk_users.py b/api/shared/sdk_users.py index 77a0fcf9ad..6da4632061 100644 --- a/api/shared/sdk_users.py +++ b/api/shared/sdk_users.py @@ -182,12 +182,22 @@ def __init__(self, status_code: int, detail: str) -> None: RESERVED_EMAIL_MESSAGE = "This email domain is reserved" -async def _lock_user_row(session: AsyncSession, user_id: UUID) -> None: - """Lock the user before any guard locks a role: identity provisioning - takes the same order (user, then roles), so the two can't deadlock.""" +async def _lock_user_row(session: AsyncSession, user: "UserORM") -> None: + """Lock the user and read it again, before deciding anything from it. + + Taken before any guard locks a role, the order identity provisioning + uses (user, then roles), so the two can't deadlock. The reread means + authorization sees the user as a concurrent change (a move, say) left + them, not as first loaded. + """ from src.models import User as UserORM - await session.execute(select(UserORM.id).where(UserORM.id == user_id).with_for_update()) + await session.execute( + select(UserORM) + .where(UserORM.id == user.id) + .with_for_update() + .execution_options(populate_existing=True) + ) def refuse_identity_management(user: "UserORM") -> None: @@ -661,7 +671,7 @@ async def update_user( if db_user.is_system: raise UserServiceError(403, "System user cannot be modified") refuse_identity_management(db_user) - await _lock_user_row(session, db_user.id) + await _lock_user_row(session, db_user) if email is not None: refuse_reserved_email(email) _authorize_update( @@ -824,7 +834,7 @@ async def delete_user( if db_user.is_system: raise UserServiceError(403, "System user cannot be deleted") refuse_identity_management(db_user) - await _lock_user_row(session, db_user.id) + await _lock_user_row(session, db_user) require_operation(caller, "users.delete", org_target(db_user.organization_id)) require_unprotected(caller, db_user.id in await privileged_user_ids(session, [db_user.id]), db_user.organization_id) @@ -925,6 +935,7 @@ async def bulk_update_users( .where(UserORM.id.in_(request.user_ids)) .order_by(UserORM.id) .with_for_update() + .execution_options(populate_existing=True) ) users_by_id = {u.id: u for u in rows.scalars().all()} held = await held_permissions_by_user(session, list(users_by_id)) diff --git a/api/src/services/identity_providers.py b/api/src/services/identity_providers.py index caa1e403de..f79b421001 100644 --- a/api/src/services/identity_providers.py +++ b/api/src/services/identity_providers.py @@ -500,8 +500,6 @@ async def _authorize_mapped_roles( missing = role_ids - roles.keys() if missing: raise HTTPException(422, f"Role {sorted(missing, key=str)[0]} not found") - if any(is_privileged_principal(role.permissions) for role in roles.values()): - raise HTTPException(422, "Roles with privileged permissions can't be mapped from a group") require_permission(caller, "roleassignments.readwrite", organization_id) reach = permitted_organizations(caller, "roleassignments.readwrite") boundaries = default_boundaries(organization_id) @@ -524,6 +522,19 @@ async def _reconcile_mapping_change(provisioner: Provisioner) -> None: raise HTTPException(409, exc.detail) from None +async def _refuse_privileged_roles(session: AsyncSession, role_ids: set[UUID]) -> None: + """Refuse mapping a role with privileged permissions. Decided under + share locks a permission edit waits on, held to the commit, so a role + can't turn privileged between this check and the mapping's save.""" + from shared.account_guards import share_lock_roles + from src.services.user_role_assignments import load_roles + + await share_lock_roles(session, role_ids) + roles = await load_roles(session, role_ids) + if any(is_privileged_principal(role.permissions) for role in roles.values()): + raise HTTPException(422, "Roles with privileged permissions can't be mapped from a group") + + async def set_group_mapping( session: AsyncSession, caller: Caller, @@ -543,6 +554,7 @@ async def set_group_mapping( # also takes, so it can't vanish before the mapping is written. await provisioner.hold_policy(exclusive=True) group = await _group(session, connection, group_id) + await _refuse_privileged_roles(session, wanted) mapping = await session.get(IdentityProviderGroupMapping, group.id) if mapping is None: session.add( diff --git a/api/src/services/identity_provisioning/core.py b/api/src/services/identity_provisioning/core.py index 49cc753bd6..4e25d09ece 100644 --- a/api/src/services/identity_provisioning/core.py +++ b/api/src/services/identity_provisioning/core.py @@ -52,6 +52,7 @@ from sqlalchemy import delete, func, or_, select, text from sqlalchemy.ext.asyncio import AsyncSession +from shared.account_guards import share_lock_roles from shared.builtin_roles import ( BASE_ROLE_IDS, DECRYPTION_ROLE_ID, @@ -67,7 +68,7 @@ IdentityProviderGroupRole, IdentityProviderUser, ) -from src.models.orm.users import Role, User, UserRole +from src.models.orm.users import User, UserRole from src.services.audit_context import ActorContext from src.services.identity_provisioning.domains import email_domain, verified_domains @@ -688,7 +689,7 @@ async def _grant_roles(self, user: User, desired: set[UUID]) -> set[UUID]: # A mapped role checked at mapping time can gain privileged # permissions later, so check again at every grant, under the lock # a permission edit waits on. - await self._lock_roles_shared(missing) + await share_lock_roles(self.session, missing) roles = await load_roles(self.session, missing) for role_id in sorted(missing): if role_id not in roles: @@ -838,16 +839,6 @@ async def _require_adoptable(self, user: User) -> None: "This user is already managed by an identity provider", ) - async def _lock_roles_shared(self, role_ids: Iterable[UUID]) -> None: - """Share-lock role rows before reading their permissions. A - permission edit locks its role for update first, so it waits for - this decision rather than changing the role under it.""" - ids = set(role_ids) - if ids: - await self.session.execute( - select(Role.id).where(Role.id.in_(ids)).order_by(Role.id).with_for_update(read=True) - ) - async def _unmanaged(self, users: Iterable[User]) -> dict[UUID, str]: """``user_id -> reason`` for the locked ``users`` this connection must leave alone: privileged ones, and any no longer in its organization @@ -860,7 +851,7 @@ async def _unmanaged(self, users: Iterable[User]) -> dict[UUID, str]: select(UserRole.role_id).where(UserRole.user_id.in_([u.id for u in users])) ) ) - await self._lock_roles_shared(held | {u.base_role_id for u in users if u.base_role_id}) + await share_lock_roles(self.session, held | {u.base_role_id for u in users if u.base_role_id}) privileged = await privileged_user_ids(self.session, [u.id for u in users]) unmanaged: dict[UUID, str] = {} for user in users: diff --git a/api/src/services/identity_provisioning/expiry.py b/api/src/services/identity_provisioning/expiry.py index a1b7650840..89b04c07b0 100644 --- a/api/src/services/identity_provisioning/expiry.py +++ b/api/src/services/identity_provisioning/expiry.py @@ -65,11 +65,15 @@ async def collect_expiry_notices(session: AsyncSession, now: datetime) -> list[E a rotation (which locks them for update) either lands before this reads the tokens or waits until the warnings are up, then dismisses them. """ - await session.execute( - select(IdentityProvider.id) - .where(IdentityProvider.is_enabled.is_(True)) - .order_by(IdentityProvider.id) - .with_for_update(key_share=True) + locked = list( + await session.scalars( + select(IdentityProvider.id) + .where(IdentityProvider.is_enabled.is_(True)) + .order_by(IdentityProvider.id) + # FOR KEY SHARE: SCIM requests' last_request_at updates don't + # wait on it; only rotation's FOR UPDATE does. + .with_for_update(read=True, key_share=True) + ) ) rows = ( await session.execute( @@ -80,6 +84,8 @@ async def collect_expiry_notices(session: AsyncSession, now: datetime) -> list[E ) .join(Organization, Organization.id == IdentityProvider.organization_id) .where( + # Only the connections locked above, enabled since or not. + IdentityProvider.id.in_(locked), IdentityProvider.is_enabled.is_(True), Organization.is_active.is_(True), IdentityProviderToken.revoked_at.is_(None), diff --git a/api/src/services/identity_provisioning/scim.py b/api/src/services/identity_provisioning/scim.py index c405ebabcd..199b46ffef 100644 --- a/api/src/services/identity_provisioning/scim.py +++ b/api/src/services/identity_provisioning/scim.py @@ -127,7 +127,7 @@ def _split_unknown_extensions( unknown: dict[str, Any] = {} for key, value in body.items(): if key.startswith("urn:") and key not in known: - unknown[key] = value + unknown[key] = _extension_value(key, value) else: kept[key] = value if "schemas" in kept and isinstance(kept["schemas"], list): diff --git a/api/tests/unit/services/test_identity_provider_concurrency.py b/api/tests/unit/services/test_identity_provider_concurrency.py index 46fa2ed4b7..4983282986 100644 --- a/api/tests/unit/services/test_identity_provider_concurrency.py +++ b/api/tests/unit/services/test_identity_provider_concurrency.py @@ -253,3 +253,91 @@ async def second_rotation(): assert shortened < first.info.expires_at assert second.previous_tokens_expire_at == shortened assert tokens[second.info.hint].expires_at == second.info.expires_at + + +async def _pending_hires(async_session_factory, committed, count: int) -> list: + """Linked users waiting for the mapped role, on a connection that + provisions External users.""" + async with async_session_factory() as session: + connection = await session.get(IdentityProvider, committed["connection_id"]) + assert connection is not None + connection.provision_as_external = True + provisioner = Provisioner(session, connection) + group = await provisioner.create_group( + GroupRecord(display_name="Portal", external_id="g-portal", attributes={}), [] + ) + session.add(IdentityProviderGroupMapping(group_id=group.id, created_by="test")) + await session.flush() + session.add(IdentityProviderGroupRole(group_id=group.id, role_id=committed["role_id"])) + await session.flush() + links = [ + await provisioner.create_user(_record(f"portal{i}@{committed['domain']}")) + for i in range(count) + ] + await provisioner.set_group_members(group, [link.id for link in links]) + await session.commit() + return [link.id for link in links] + + +@pytest.mark.asyncio +async def test_grants_of_one_role_to_external_users_dont_wait_for_each_other( + async_session_factory, committed +): + first, second = await _pending_hires(async_session_factory, committed, 2) + + async def apply(link_id) -> None: + async with async_session_factory() as session: + connection = await session.get(IdentityProvider, committed["connection_id"]) + assert connection is not None + await Provisioner(session, connection).apply_pending([link_id]) + await session.commit() + + async with asyncio.timeout(30): + async with async_session_factory() as session: + connection = await session.get(IdentityProvider, committed["connection_id"]) + assert connection is not None + await Provisioner(session, connection).apply_pending([first]) + # Holding that grant open, a second grant of the same role goes + # through: both only share-lock the role, so neither can be left + # waiting to upgrade a lock the other shares. + await asyncio.wait_for(apply(second), timeout=_STILL_WAITING * 4) + await session.commit() + + +@pytest.mark.asyncio +async def test_a_mapping_waits_for_a_permission_edit_and_refuses_the_new_privilege( + async_session_factory, committed +): + from fastapi import HTTPException + + from shared.account_guards import _lock_roles + from src.services.identity_providers import set_group_mapping + + async with async_session_factory() as session: + connection = await session.get(IdentityProvider, committed["connection_id"]) + assert connection is not None + group = await Provisioner(session, connection).create_group( + GroupRecord(display_name="Admins", external_id="g-admins", attributes={}), [] + ) + await session.commit() + group_id = group.id + + async def map_group() -> None: + async with async_session_factory() as session: + await set_group_mapping( + session, admin_caller(), committed["connection_id"], group_id, [committed["role_id"]] + ) + await session.commit() + + async with asyncio.timeout(30): + async with async_session_factory() as editor: + await _lock_roles(editor, {committed["role_id"]}) + editor.add(RolePermission(role_id=committed["role_id"], permission="users.readwrite")) + await editor.flush() + mapping = asyncio.create_task(map_group()) + assert await _waits(mapping) + await editor.commit() + with pytest.raises(HTTPException) as raised: + await mapping + + assert raised.value.status_code == 422 From 767a269e84e74c7fa741dfa6cdb3853aa51db128 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Tue, 17 Nov 2026 23:40:03 -0500 Subject: [PATCH 18/26] Unlink privileged users on DELETE and avoid the admin lock upgrade From the adversarial confirmation pass: - A SCIM DELETE of a privileged user unlinks and deactivates them through the account guards; only an unprivileged, never-signed-in account the connection created is removed outright. Before, the hard delete skipped the last-admin and SSO-recovery checks. - Provisioning no longer share-locks built-in roles (their permissions can't change). The last-admin guard locks Platform Admin for update during a deactivation, and a share lock taken first would have to be upgraded, deadlocking two concurrent admin offboardings. - Extension attributes sent flattened at the top level, as Entra does for custom attributes (":": value), are filed under their schema (merged with the nested form for enterprise) instead of refused. Adds a two-session test that an admin and a directory deactivating the same user don't deadlock (single and bulk), which fails with the old role-before-user order. Co-Authored-By: Claude Opus 5.5 --- .../services/identity_provisioning/core.py | 17 +++++-- .../services/identity_provisioning/scim.py | 22 +++++++-- .../test_identity_provider_concurrency.py | 47 +++++++++++++++++++ .../test_identity_provisioning_core.py | 5 ++ .../test_identity_provisioning_scim.py | 25 +++++----- 5 files changed, 98 insertions(+), 18 deletions(-) diff --git a/api/src/services/identity_provisioning/core.py b/api/src/services/identity_provisioning/core.py index 4e25d09ece..c0af9ece19 100644 --- a/api/src/services/identity_provisioning/core.py +++ b/api/src/services/identity_provisioning/core.py @@ -55,6 +55,7 @@ from shared.account_guards import share_lock_roles from shared.builtin_roles import ( BASE_ROLE_IDS, + BUILTIN_ROLE_IDS, DECRYPTION_ROLE_ID, PLATFORM_ADMIN_ROLE_ID, PLATFORM_OPERATOR_ROLE_ID, @@ -339,8 +340,11 @@ async def delete_user(self, link: IdentityProviderUser) -> None: await self.hold_policy() user = await self._lock_user(link.user_id) # Removing access is always the directory's call, privileged or not. - await self._revoke_only(user) - if link.created_user and user.last_login is None: + revoke_only = await self._revoke_only(user) + # Removing an account outright is for one nobody has used and that + # carries nothing Bifrost granted; a privileged one is unlinked and + # deactivated, through the account guards like any deactivation. + if link.created_user and user.last_login is None and not revoke_only: from shared.run_cancellation import cancel_unstarted_runs_for_deleted_user body = self._user_event_body(user, link) @@ -851,7 +855,14 @@ async def _unmanaged(self, users: Iterable[User]) -> dict[UUID, str]: select(UserRole.role_id).where(UserRole.user_id.in_([u.id for u in users])) ) ) - await share_lock_roles(self.session, held | {u.base_role_id for u in users if u.base_role_id}) + # Built-in roles are left out: their permissions can't be edited. + # One of them, Platform Admin, is locked for update by the last-admin + # guard a deactivation runs; a share lock taken here would have to be + # upgraded, and two deactivations at once would deadlock on it. + await share_lock_roles( + self.session, + (held | {u.base_role_id for u in users if u.base_role_id}) - BUILTIN_ROLE_IDS, + ) privileged = await privileged_user_ids(self.session, [u.id for u in users]) unmanaged: dict[UUID, str] = {} for user in users: diff --git a/api/src/services/identity_provisioning/scim.py b/api/src/services/identity_provisioning/scim.py index 199b46ffef..8bcca1e6bd 100644 --- a/api/src/services/identity_provisioning/scim.py +++ b/api/src/services/identity_provisioning/scim.py @@ -122,14 +122,28 @@ def _from_scim_exception(exc: SCIMException) -> ScimError: def _split_unknown_extensions( body: dict[str, Any], known: frozenset[str] ) -> tuple[dict[str, Any], dict[str, Any]]: - """Separate extension objects Bifrost has no schema for.""" + """Separate extension objects Bifrost has no schema for. + + An attribute sent flattened, as ``":": value`` + (Entra does this for custom extension attributes), is filed under its + schema like the nested form. + """ kept: dict[str, Any] = {} unknown: dict[str, Any] = {} for key, value in body.items(): - if key.startswith("urn:") and key not in known: - unknown[key] = _extension_value(key, value) - else: + if not key.startswith("urn:"): kept[key] = value + elif key in known: + # Merged with any of its attributes already filed flattened. + kept[key] = {**_extension_object(kept, key), **_extension_value(key, value)} + elif isinstance(value, dict): + _extension_object(unknown, key).update(value) + else: + schema, _, attribute = key.rpartition(":") + if not schema.startswith("urn:") or not attribute: + raise ScimError(400, f"A value for the extension '{key}' must be an object", "invalidValue") + target = kept if schema in known else unknown + _extension_object(target, schema)[attribute] = value if "schemas" in kept and isinstance(kept["schemas"], list): kept["schemas"] = [s for s in kept["schemas"] if s in known] return kept, unknown diff --git a/api/tests/unit/services/test_identity_provider_concurrency.py b/api/tests/unit/services/test_identity_provider_concurrency.py index 4983282986..130102292f 100644 --- a/api/tests/unit/services/test_identity_provider_concurrency.py +++ b/api/tests/unit/services/test_identity_provider_concurrency.py @@ -100,6 +100,53 @@ async def _waits(task: asyncio.Task) -> bool: return not task.done() +@pytest.mark.asyncio +@pytest.mark.parametrize("bulk", [False, True]) +async def test_an_admin_and_a_directory_deactivating_one_user_dont_deadlock( + async_session_factory, committed, bulk +): + from shared.account_guards import refuse_last_active_platform_admin_removal + from shared.sdk_users import bulk_update_users, update_user + from src.models.contracts.users import BulkUserOperation + + async with async_session_factory() as session: + user = User(email=f"leaver@{committed['domain']}", organization_id=committed["organization_id"]) + session.add(user) + await session.commit() + user_id = user.id + + async def deactivate() -> None: + async with async_session_factory() as session: + if bulk: + await bulk_update_users( + session, + admin_caller(), + BulkUserOperation(user_ids=[user_id], operation="set_active", is_active=False), + ) + else: + await update_user( + session, + admin_caller(), + user_id=str(user_id), + fields={"is_active"}, + is_active=False, + ) + await session.commit() + + async with asyncio.timeout(30): + async with async_session_factory() as directory: + # What a directory deactivation does: lock the user, then run + # the last-admin guard, which locks Platform Admin for update. + await directory.execute(select(User.id).where(User.id == user_id).with_for_update()) + admin = asyncio.create_task(deactivate()) + assert await _waits(admin) + # Had the admin taken the role lock before the user, this would + # close a cycle and PostgreSQL would abort one side. + await refuse_last_active_platform_admin_removal(directory, {user_id}) + await directory.commit() + await admin + + async def _move(session, user_id, organization_id, *, bulk: bool) -> bool: """Move the user as an administrator would; whether it was allowed.""" from shared.sdk_users import UserServiceError, bulk_update_users, update_user diff --git a/api/tests/unit/services/test_identity_provisioning_core.py b/api/tests/unit/services/test_identity_provisioning_core.py index 379c8826ee..8ccaa37d6e 100644 --- a/api/tests/unit/services/test_identity_provisioning_core.py +++ b/api/tests/unit/services/test_identity_provisioning_core.py @@ -308,6 +308,9 @@ async def test_a_user_who_becomes_privileged_can_only_lose_access(setup): assert not (await _user(session, link)).is_active assert link.access_pending_since is None + # A DELETE unlinks: the account was created by this connection and has + # never signed in, but it's privileged, so it isn't removed outright. + user_id = link.user_id await provisioner.delete_user(link) assert ( await session.scalar( @@ -315,6 +318,8 @@ async def test_a_user_who_becomes_privileged_can_only_lose_access(setup): ) is None ) + user = await session.get(User, user_id) + assert user is not None and not user.is_active @pytest.mark.asyncio diff --git a/api/tests/unit/services/test_identity_provisioning_scim.py b/api/tests/unit/services/test_identity_provisioning_scim.py index cafb3097b1..bb553d696a 100644 --- a/api/tests/unit/services/test_identity_provisioning_scim.py +++ b/api/tests/unit/services/test_identity_provisioning_scim.py @@ -114,18 +114,21 @@ def test_an_unsupported_verb_on_an_unknown_extension_changes_nothing(): assert link.attributes[CUSTOM] == {"costCenter": "42"} -@pytest.mark.parametrize( - "operation", - [ - {"op": "replace", "path": f"{CUSTOM}:costCenter", "value": "43"}, - {"op": "add", "value": {CUSTOM: {"costCenter": "43"}}}, - ], -) -def test_a_patch_into_an_extension_that_is_not_an_object_is_invalid(operation): - with pytest.raises(ScimError) as raised: - scim.apply_user_patch(_link(**{CUSTOM: "not an object"}), _patch(operation)) +def test_flattened_extension_attributes_are_filed_under_their_schema(): + resource, unknown = scim.parse_user( + { + "schemas": [scim.USER_SCHEMA, ENTERPRISE], + "userName": "pat@contoso.example", + f"{ENTERPRISE}:department": "Support", + ENTERPRISE: {"employeeNumber": "42"}, + f"{CUSTOM}:costCenter": "7", + } + ) + record = scim.user_record(resource, unknown) - assert raised.value.status == 400 + assert record.attributes[ENTERPRISE]["department"] == "Support" + assert record.attributes[ENTERPRISE]["employeeNumber"] == "42" + assert record.attributes[CUSTOM] == {"costCenter": "7"} @pytest.mark.parametrize( From 1356564a4ee35da8304e71f9394afab22ad3b6a6 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Wed, 18 Nov 2026 00:12:55 -0500 Subject: [PATCH 19/26] Lock users first in role assignment writes and fix suite failures From Sol's confirmation pass: - Role assignment writes (PUT, base-role change, SDK assign/unassign) lock the user before the last-admin guard locks Platform Admin, the order identity provisioning uses. Adds a two-session test that a Platform Admin demotion and a directory deactivation don't deadlock. - An update or delete whose user was deleted while it waited for the lock answers 404 instead of failing on a stale object. Found by running the full unit suite: - Provisioning no longer reads is_superuser directly (privileged_user_ids already counts Platform Admin) or passes it to the User constructor; update_user's extra read is registered in the elevated-check inventory. - The new notification lookup is registered as a plain Redis read. - The import-boundary test lists identity_providers among the users contract's dependencies (UserPublic.identity_source). Co-Authored-By: Claude Opus 5.5 --- api/shared/sdk_roles.py | 9 +++- api/shared/sdk_users.py | 5 +- .../services/identity_provisioning/core.py | 4 +- api/src/services/user_role_assignments.py | 28 ++++++++--- .../test_identity_provider_concurrency.py | 50 +++++++++++++++++++ api/tests/unit/test_access_list.py | 6 ++- .../unit/test_model_import_boundaries.py | 4 +- 7 files changed, 93 insertions(+), 13 deletions(-) diff --git a/api/shared/sdk_roles.py b/api/shared/sdk_roles.py index 140c1d36ea..0c3481146a 100644 --- a/api/shared/sdk_roles.py +++ b/api/shared/sdk_roles.py @@ -682,7 +682,14 @@ async def _require_assignment_target( from src.services.authorization.privilege import is_privileged_principal from src.services.user_role_assignments import RoleAssignmentError, check_role_change - user = await session.get(UserORM, user_id) + # Lock the user before any guard locks a role, the order identity + # provisioning uses, so the two can't deadlock. + user = await session.scalar( + select(UserORM) + .where(UserORM.id == user_id) + .with_for_update() + .execution_options(populate_existing=True) + ) if user is None: return None require_operation(caller, operation, org_target(user.organization_id)) diff --git a/api/shared/sdk_users.py b/api/shared/sdk_users.py index 6da4632061..96fc616b58 100644 --- a/api/shared/sdk_users.py +++ b/api/shared/sdk_users.py @@ -192,12 +192,15 @@ async def _lock_user_row(session: AsyncSession, user: "UserORM") -> None: """ from src.models import User as UserORM - await session.execute( + locked = await session.scalar( select(UserORM) .where(UserORM.id == user.id) .with_for_update() .execution_options(populate_existing=True) ) + if locked is None: + # Deleted (by a directory, say) while this request waited. + raise UserServiceError(404, "User not found") def refuse_identity_management(user: "UserORM") -> None: diff --git a/api/src/services/identity_provisioning/core.py b/api/src/services/identity_provisioning/core.py index c0af9ece19..c83515330a 100644 --- a/api/src/services/identity_provisioning/core.py +++ b/api/src/services/identity_provisioning/core.py @@ -868,7 +868,8 @@ async def _unmanaged(self, users: Iterable[User]) -> dict[UUID, str]: for user in users: if user.organization_id != self.connection.organization_id: unmanaged[user.id] = OTHER_ORG_USER_MESSAGE - elif user.is_superuser or user.id in privileged: + # privileged_user_ids counts the Platform Admin role too. + elif user.id in privileged: unmanaged[user.id] = PRIVILEGED_USER_MESSAGE return unmanaged @@ -961,7 +962,6 @@ async def _new_user(self, email: str, record: UserRecord, *, active: bool) -> Us name=record.display_name, hashed_password="", is_active=active, - is_superuser=False, is_external=self.connection.provision_as_external, # The identity provider vouches for the address; the person signs # in through SSO, which marks the account registered. diff --git a/api/src/services/user_role_assignments.py b/api/src/services/user_role_assignments.py index 0bfa002546..ff129c87a9 100644 --- a/api/src/services/user_role_assignments.py +++ b/api/src/services/user_role_assignments.py @@ -314,8 +314,21 @@ def is_privileged(self) -> bool: return is_privileged_principal(self.held) -async def _load_target(session: AsyncSession, user_id: UUID) -> AssignmentTarget | None: - user = await session.get(User, user_id) +async def _load_target( + session: AsyncSession, user_id: UUID, *, lock: bool = False +) -> AssignmentTarget | None: + if lock: + # A write locks the user before any guard locks a role (the + # last-admin check locks Platform Admin), the order identity + # provisioning uses, so the two can't deadlock. + user = await session.scalar( + select(User) + .where(User.id == user_id) + .with_for_update() + .execution_options(populate_existing=True) + ) + else: + user = await session.get(User, user_id) if user is None: return None held = await held_permissions_by_user(session, [user_id]) @@ -492,14 +505,15 @@ async def _response( async def require_assignment_target( - session: AsyncSession, caller: Caller, user_id: UUID, operation: str + session: AsyncSession, caller: Caller, user_id: UUID, operation: str, *, lock: bool = False ) -> AssignmentTarget: """The user ``operation`` is about, once ``caller`` may perform it at - their organization (404 when they do not exist).""" + their organization (404 when they do not exist). A write passes + ``lock`` to lock the user first.""" # Reach anywhere first: a caller with none learns nothing about which # users exist. operation_reach(caller, operation) - target = await _load_target(session, user_id) + target = await _load_target(session, user_id, lock=lock) if target is None: raise RoleAssignmentError(404, "User not found") require_operation(caller, operation, org_target(target.user.organization_id)) @@ -545,7 +559,7 @@ async def replace_role_assignments( from src.core.cache import invalidate_role_users from src.services.audit import emit_audit - target = await require_assignment_target(session, caller, user_id, PUT_OPERATION) + target = await require_assignment_target(session, caller, user_id, PUT_OPERATION, lock=True) user = target.user if is_system_account(user.id): raise RoleAssignmentError(422, SYSTEM_ACCOUNT_ROLE_MESSAGE) @@ -718,7 +732,7 @@ async def set_base_role_assignment( from shared.sdk_users import UserServiceError, set_user_base_role from src.services.audit import emit_audit - target = await require_assignment_target(session, caller, user_id, PUT_OPERATION) + target = await require_assignment_target(session, caller, user_id, PUT_OPERATION, lock=True) user = target.user if user.id == caller.principal.user_id: raise RoleAssignmentError(400, "Cannot change your own roles via bulk action") diff --git a/api/tests/unit/services/test_identity_provider_concurrency.py b/api/tests/unit/services/test_identity_provider_concurrency.py index 130102292f..4abf53b950 100644 --- a/api/tests/unit/services/test_identity_provider_concurrency.py +++ b/api/tests/unit/services/test_identity_provider_concurrency.py @@ -100,6 +100,56 @@ async def _waits(task: asyncio.Task) -> bool: return not task.done() +@pytest.mark.asyncio +async def test_a_platform_admin_demotion_and_a_directory_deactivation_dont_deadlock( + async_session_factory, committed +): + from shared.account_guards import refuse_last_active_platform_admin_removal + from shared.builtin_roles import PLATFORM_ADMIN_ROLE_ID, USER_ROLE_ID + from src.models.contracts.role_assignments import UserRoleAssignmentsUpdate + from src.services.user_role_assignments import replace_role_assignments + + async with async_session_factory() as session: + # Two Platform Admins, so removing one is allowed. + admins = [ + User( + email=f"admin{i}@{committed['domain']}", + organization_id=committed["organization_id"], + is_superuser=True, + ) + for i in range(2) + ] + session.add_all(admins) + await session.flush() + session.add_all( + UserRole(user_id=admin.id, role_id=PLATFORM_ADMIN_ROLE_ID, assigned_by="test") + for admin in admins + ) + await session.commit() + user_id = admins[0].id + + async def demote() -> None: + async with async_session_factory() as session: + await replace_role_assignments( + session, + admin_caller(), + user_id=user_id, + request=UserRoleAssignmentsUpdate(base_role_id=USER_ROLE_ID, additional=[]), + ) + await session.commit() + + async with asyncio.timeout(30): + async with async_session_factory() as directory: + await directory.execute(select(User.id).where(User.id == user_id).with_for_update()) + admin = asyncio.create_task(demote()) + assert await _waits(admin) + # Had the demotion locked Platform Admin before the user, this + # would close a cycle and PostgreSQL would abort one side. + await refuse_last_active_platform_admin_removal(directory, {user_id}) + await directory.commit() + await admin + + @pytest.mark.asyncio @pytest.mark.parametrize("bulk", [False, True]) async def test_an_admin_and_a_directory_deactivating_one_user_dont_deadlock( diff --git a/api/tests/unit/test_access_list.py b/api/tests/unit/test_access_list.py index 16ca05b946..02f33a0e69 100644 --- a/api/tests/unit/test_access_list.py +++ b/api/tests/unit/test_access_list.py @@ -1959,7 +1959,7 @@ def scanned_functions(entry_functions) -> dict[str, list[tuple[str, str]]]: "shared.sdk_users.bulk_update_users": (1, "current_role_derivation"), "shared.sdk_users.create_user": (4, "current_role_derivation"), "shared.sdk_users.list_users": (2, "current_role_derivation"), - "shared.sdk_users.update_user": (8, "current_role_derivation"), + "shared.sdk_users.update_user": (9, "current_role_derivation"), "shared.sdk_workflow_execution.execute_sdk_workflow": (5, "current_role_derivation"), "shared.system_account_guard.is_system_account": (1, "actor_facts"), "shared.unattended_execution.create_unattended_context": (2, "actor_facts"), @@ -2255,6 +2255,10 @@ def scanned_functions(entry_functions) -> dict[str, list[tuple[str, str]]]: 2, "De-duplicates an admin notification before creating one.", ), + "src.services.notification_service.NotificationService.find_admin_notification_by_metadata": ( + 2, + "De-duplicates an admin notification by an id where titles repeat.", + ), "src.services.notification_service.NotificationService.update_notification": ( 1, "Rewrites a notification's status for its writer.", diff --git a/api/tests/unit/test_model_import_boundaries.py b/api/tests/unit/test_model_import_boundaries.py index 4ad911d3b7..f7eaf3dd16 100644 --- a/api/tests/unit/test_model_import_boundaries.py +++ b/api/tests/unit/test_model_import_boundaries.py @@ -144,8 +144,10 @@ def test_contract_package_exports_are_lazy_and_resolve_public_symbols() -> None: "src.models.contracts.organizations", ] assert result["user_create_name"] == "UserCreate" - # users imports role_assignments for RoleUserSummary.boundaries. + # users imports role_assignments for RoleUserSummary.boundaries and + # identity_providers for UserPublic.identity_source. assert result["loaded_after_user"] == [ + "src.models.contracts.identity_providers", "src.models.contracts.organizations", "src.models.contracts.role_assignments", "src.models.contracts.users", From a55129bd2cd30ae972ba28447f5d59d29fbbc04d Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Wed, 18 Nov 2026 10:21:10 -0500 Subject: [PATCH 20/26] Match the design doc's UI section to the reviewed screens Co-Authored-By: Claude Opus 5.5 --- docs/plans/2026-11-16-scim-identity-providers.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/plans/2026-11-16-scim-identity-providers.md b/docs/plans/2026-11-16-scim-identity-providers.md index f8dd47b705..524d7c1a50 100644 --- a/docs/plans/2026-11-16-scim-identity-providers.md +++ b/docs/plans/2026-11-16-scim-identity-providers.md @@ -265,9 +265,9 @@ connection. ## UI -- Users view gains an **Identity Providers** tab. The list columns are Name, - Organization, Vendor, Verified Domains, Provisioning Status, Credential - Expiry, Users and Groups Mapped. +- Users view gains an **Identity Providers** tab. The list columns are Name + (with the vendor beneath it), Organization, Email Domains, Provisioning, + Token Expiry, Users and Groups Mapped. - The detail page has: - an expiry banner (expired or within 14 days); - a provisioning card (status, Tenant URL with copy, token hint and expiry, @@ -276,8 +276,10 @@ connection. - a group-mapping table (group → mapping status → roles; edit). - The edit dialog uses a left-hand section nav: General, Provisioning, Group Mapping. -- The users list gains an Identity Source column in both the wide and narrow - layouts. Managed fields are locked on user detail. +- The users list shows each user's identity source beneath their name, in + both the wide and narrow layouts. A new hire waiting for the access pass is + listed with an **Activating** status instead of Inactive. Managed fields are + locked on user detail. ## Phases @@ -291,7 +293,7 @@ connection. headers, and the result goes in the docs. Proxy header trust stays out of scope: it isn't a standard, and every proxy does it differently. 2. **UI:** list, detail and edit dialog, the domains card, the token reveal, - the mapping editor, and the Identity Source column; vitest plus one + the mapping editor, and the identity source label; vitest plus one Playwright happy path. 3. **Credential expiry warnings (done):** an hourly scheduled task checks each enabled connection's current token. Superseded tokens are ignored, so the From 830cff527bb1d6ec9b47011b6831c2357fc7cd13 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Wed, 18 Nov 2026 12:38:26 -0500 Subject: [PATCH 21/26] Stop advertising the SCIM password attribute Bifrost drops a password on arrival and reports changePassword as unsupported, so /Schemas no longer lists it. Microsoft's SCIM Validator refused to run against the endpoint while it did. Co-Authored-By: Claude Opus 5.5 --- api/src/services/identity_provisioning/scim.py | 7 +++++++ .../unit/services/test_identity_provisioning_scim.py | 8 ++++++++ 2 files changed, 15 insertions(+) diff --git a/api/src/services/identity_provisioning/scim.py b/api/src/services/identity_provisioning/scim.py index 8bcca1e6bd..4fcdb02e5d 100644 --- a/api/src/services/identity_provisioning/scim.py +++ b/api/src/services/identity_provisioning/scim.py @@ -609,6 +609,13 @@ def schemas(base_url: str) -> list[dict[str, Any]]: result = [] for model in (User, EnterpriseUser, Group): body = _dump(model.to_schema()) + # Discovery lists only what Bifrost keeps; a password is dropped on + # arrival, and advertising it makes Entra's validator generate one. + body["attributes"] = [ + attribute + for attribute in body["attributes"] + if attribute["name"] not in _NEVER_STORED + ] body["meta"] = {"resourceType": "Schema", "location": f"{base_url}/Schemas/{body['id']}"} result.append(body) return result diff --git a/api/tests/unit/services/test_identity_provisioning_scim.py b/api/tests/unit/services/test_identity_provisioning_scim.py index bb553d696a..347018cf73 100644 --- a/api/tests/unit/services/test_identity_provisioning_scim.py +++ b/api/tests/unit/services/test_identity_provisioning_scim.py @@ -58,6 +58,14 @@ def test_entra_create_keeps_extensions_and_drops_the_password(): assert "password" not in record.attributes +def test_discovery_does_not_advertise_the_password_it_drops(): + user = next(s for s in scim.schemas("https://bifrost.example") if s["id"] == scim.USER_SCHEMA) + names = {attribute["name"] for attribute in user["attributes"]} + + assert "password" not in names + assert {"userName", "emails", "active"} <= names + + def test_user_without_email_or_email_username_is_refused(): resource, unknown = scim.parse_user({"userName": "pat"}) From 9bb6c6652fa45a48531e9fd9bc18d8b509ecb844 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Wed, 18 Nov 2026 12:40:23 -0500 Subject: [PATCH 22/26] Add the Product Updates disposition and draft entry for SCIM providers Co-Authored-By: Claude Opus 5.5 --- product-updates/dispositions/pr-1464.json | 15 ++++++ .../182c8b4f-4cb8-467c-bc96-0c24b16106b7.md | 46 +++++++++++++++++++ 2 files changed, 61 insertions(+) create mode 100644 product-updates/dispositions/pr-1464.json create mode 100644 product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md diff --git a/product-updates/dispositions/pr-1464.json b/product-updates/dispositions/pr-1464.json new file mode 100644 index 0000000000..0a30ca8355 --- /dev/null +++ b/product-updates/dispositions/pr-1464.json @@ -0,0 +1,15 @@ +{ + "key": "pr:1464", + "classification": "highlight", + "entry_ids": [ + "182c8b4f-4cb8-467c-bc96-0c24b16106b7" + ], + "reason": "New user-facing feature: SCIM provisioning and group-to-role mapping per organization.", + "review": { + "status": "draft", + "evidence": [ + "https://github.com/gobifrost/bifrost/pull/1464", + "https://github.com/gobifrost/bifrost/issues/1463" + ] + } +} diff --git a/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md b/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md new file mode 100644 index 0000000000..17f605023b --- /dev/null +++ b/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md @@ -0,0 +1,46 @@ +---json +{ + "id": "182c8b4f-4cb8-467c-bc96-0c24b16106b7", + "revision": 1, + "published_at": "2026-11-18T17:39:34Z", + "title": "Provision Users From Your Identity Provider", + "visibility": "highlight", + "area": "Administration", + "type": "New", + "action_required": false, + "security_review": { + "status": "required", + "review_ref": "https://github.com/gobifrost/bifrost/pull/1464" + }, + "sources": [ + { + "pr": 1464, + "role": "implementation" + } + ], + "contributors": [ + { + "login": "MTG-Thomas", + "profile_url": "https://github.com/MTG-Thomas", + "source_pr": 1464 + } + ], + "eligibility": { + "requires_prs": [ + 1464 + ], + "requires_commits": [] + }, + "assets": [], + "review": { + "status": "draft", + "evidence": [ + "https://github.com/gobifrost/bifrost/pull/1464", + "Draft prose; awaiting exact-text approval. The installation-relative route /users/identity-providers was verified in client/src/App.tsx." + ] + }, + "in_app": true +} +--- + +Connect a SCIM directory, such as Microsoft Entra ID, to an organization in [Identity Providers](/users/identity-providers). Bifrost creates, updates and deactivates its users and maps directory groups to roles. Organizations verify their email domains with a DNS record first, and directories never grant privileged roles. From 5ebe88bc8260eeb88c4e6a09f5acc2bdefcadb84 Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Wed, 18 Nov 2026 12:53:13 -0500 Subject: [PATCH 23/26] Sync the plugin skill mirror and address CodeQL findings in the tests Co-Authored-By: Claude Opus 5.5 --- api/tests/e2e/api/test_scim_provisioning.py | 9 ++++++--- .../services/test_identity_provider_access_pass.py | 4 +--- .../services/test_identity_provider_concurrency.py | 8 ++++---- .../bifrost-build/generated/openapi-digest.md | 13 +++++++++++++ .../skills/bifrost-build/generated/operations.md | 13 +++++++++++++ 5 files changed, 37 insertions(+), 10 deletions(-) diff --git a/api/tests/e2e/api/test_scim_provisioning.py b/api/tests/e2e/api/test_scim_provisioning.py index 1fb222d06d..542bff4c68 100644 --- a/api/tests/e2e/api/test_scim_provisioning.py +++ b/api/tests/e2e/api/test_scim_provisioning.py @@ -98,13 +98,15 @@ async def test_scim_user_and_group_lifecycle(e2e_client, platform_admin, db_sess unclaimed = {**entra_user, "userName": "pat@unclaimed.example", "externalId": "x", "emails": [{"value": "pat@unclaimed.example", "primary": True}]} await _verify_domain_out_of_band(db_session, scim_connection["claim_id"]) - assert e2e_client.post(f"{path}/Users", headers=scim, json=unclaimed).status_code == 400 + refused = e2e_client.post(f"{path}/Users", headers=scim, json=unclaimed) + assert refused.status_code == 400 created = e2e_client.post(f"{path}/Users", headers=scim, json=entra_user) body = _ok(created, 201) assert created.headers["location"].endswith(f"/Users/{body['id']}") assert body["active"] is True and body[ENTERPRISE]["department"] == "Support" - assert _ok(e2e_client.post(f"{path}/Users", headers=scim, json=entra_user), 409)["scimType"] == "uniqueness" + duplicate = e2e_client.post(f"{path}/Users", headers=scim, json=entra_user) + assert _ok(duplicate, 409)["scimType"] == "uniqueness" found = _ok(e2e_client.get(f"{path}/Users", headers=scim, params={"filter": f'userName eq "{email.upper()}"'})) assert [r["id"] for r in found["Resources"]] == [body["id"]] @@ -155,7 +157,8 @@ async def test_scim_user_and_group_lifecycle(e2e_client, platform_admin, db_sess })) assert patched["active"] is False - assert e2e_client.delete(f"{path}/Users/{body['id']}", headers=new_scim).status_code == 204 + deleted = e2e_client.delete(f"{path}/Users/{body['id']}", headers=new_scim) + assert deleted.status_code == 204 db_session.expire_all() assert await db_session.get(User, user_id) is None finally: diff --git a/api/tests/unit/services/test_identity_provider_access_pass.py b/api/tests/unit/services/test_identity_provider_access_pass.py index cfd8673552..6c67fad438 100644 --- a/api/tests/unit/services/test_identity_provider_access_pass.py +++ b/api/tests/unit/services/test_identity_provider_access_pass.py @@ -317,8 +317,6 @@ async def deadlocked(self, link_ids): @pytest.mark.asyncio async def test_the_oldest_grant_goes_first_across_connections(setup, monkeypatch): - import src.services.identity_provisioning.access_pass as access_pass - session = setup["session"] other = IdentityProvider( organization_id=setup["connection"].organization_id, @@ -338,7 +336,7 @@ async def test_the_oldest_grant_goes_first_across_connections(setup, monkeypatch await Provisioner(session, connection).replace_user( link, _record(link.user_name, name=renamed) ) - monkeypatch.setattr(access_pass, "BATCH_LIMIT", 1) + monkeypatch.setattr("src.services.identity_provisioning.access_pass.BATCH_LIMIT", 1) await run_access_pass(session) diff --git a/api/tests/unit/services/test_identity_provider_concurrency.py b/api/tests/unit/services/test_identity_provider_concurrency.py index 4abf53b950..1d6763e3ef 100644 --- a/api/tests/unit/services/test_identity_provider_concurrency.py +++ b/api/tests/unit/services/test_identity_provider_concurrency.py @@ -147,7 +147,7 @@ async def demote() -> None: # would close a cycle and PostgreSQL would abort one side. await refuse_last_active_platform_admin_removal(directory, {user_id}) await directory.commit() - await admin + await asyncio.gather(admin) @pytest.mark.asyncio @@ -194,7 +194,7 @@ async def deactivate() -> None: # close a cycle and PostgreSQL would abort one side. await refuse_last_active_platform_admin_removal(directory, {user_id}) await directory.commit() - await admin + await asyncio.gather(admin) async def _move(session, user_id, organization_id, *, bulk: bool) -> bool: @@ -297,7 +297,7 @@ async def apply_grant() -> None: grant = asyncio.create_task(apply_grant()) assert await _waits(grant) await editor.commit() - await grant + await asyncio.gather(grant) async with async_session_factory() as session: assert ( @@ -435,6 +435,6 @@ async def map_group() -> None: assert await _waits(mapping) await editor.commit() with pytest.raises(HTTPException) as raised: - await mapping + await asyncio.gather(mapping) assert raised.value.status_code == 422 diff --git a/plugins/bifrost/skills/bifrost-build/generated/openapi-digest.md b/plugins/bifrost/skills/bifrost-build/generated/openapi-digest.md index 27671bf71c..1c50c59078 100644 --- a/plugins/bifrost/skills/bifrost-build/generated/openapi-digest.md +++ b/plugins/bifrost/skills/bifrost-build/generated/openapi-digest.md @@ -300,6 +300,19 @@ | POST | `/api/identities` | | DELETE | `/api/identities/{identity_id}` | | PATCH | `/api/identities/{identity_id}` | +| GET | `/api/identity-providers` | +| POST | `/api/identity-providers` | +| GET | `/api/identity-providers/domains` | +| POST | `/api/identity-providers/domains` | +| DELETE | `/api/identity-providers/domains/{claim_id}` | +| POST | `/api/identity-providers/domains/{claim_id}/verify` | +| DELETE | `/api/identity-providers/{identity_provider_id}` | +| GET | `/api/identity-providers/{identity_provider_id}` | +| PATCH | `/api/identity-providers/{identity_provider_id}` | +| GET | `/api/identity-providers/{identity_provider_id}/groups` | +| DELETE | `/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping` | +| PUT | `/api/identity-providers/{identity_provider_id}/groups/{group_id}/mapping` | +| POST | `/api/identity-providers/{identity_provider_id}/token` | | GET | `/api/integrations` | | POST | `/api/integrations` | | GET | `/api/integrations/by-name/{name}` | diff --git a/plugins/bifrost/skills/bifrost-build/generated/operations.md b/plugins/bifrost/skills/bifrost-build/generated/operations.md index 379f030983..af2a90f811 100644 --- a/plugins/bifrost/skills/bifrost-build/generated/operations.md +++ b/plugins/bifrost/skills/bifrost-build/generated/operations.md @@ -187,3 +187,16 @@ caller's own items. | `workspace.files.editor.folder.create` | — | — | `repository.readwrite` | | `workspace.files.editor.delete` | — | — | `repository.readwrite` | | `workspace.files.editor.rename` | — | — | `repository.readwrite` | +| `identityproviders.domains.list` | — | — | `identityproviders.read` | +| `identityproviders.domains.create` | — | — | `identityproviders.readwrite` | +| `identityproviders.domains.verify` | — | — | `identityproviders.readwrite` | +| `identityproviders.domains.delete` | — | — | `identityproviders.readwrite` | +| `identityproviders.list` | — | — | `identityproviders.read` | +| `identityproviders.create` | — | — | `identityproviders.readwrite` | +| `identityproviders.get` | — | — | `identityproviders.read` | +| `identityproviders.update` | — | — | `identityproviders.readwrite` | +| `identityproviders.delete` | — | — | `identityproviders.readwrite` | +| `identityproviders.token.rotate` | — | — | `identityproviders.readwrite` | +| `identityproviders.groups.list` | — | — | `identityproviders.read` | +| `identityproviders.groups.map` | — | — | `identityproviders.readwrite`, `roleassignments.readwrite` | +| `identityproviders.groups.unmap` | — | — | `identityproviders.readwrite` | From 235b04b802af3dc0337bb747d2d93d33e915f57b Mon Sep 17 00:00:00 2001 From: Thomas <87775189+MTG-Thomas@users.noreply.github.com> Date: Wed, 18 Nov 2026 13:15:09 -0500 Subject: [PATCH 24/26] Record approval of the SCIM providers Product Updates entry Co-Authored-By: Claude Opus 5.5 --- product-updates/dispositions/pr-1464.json | 5 +++-- .../entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/product-updates/dispositions/pr-1464.json b/product-updates/dispositions/pr-1464.json index 0a30ca8355..16802bffed 100644 --- a/product-updates/dispositions/pr-1464.json +++ b/product-updates/dispositions/pr-1464.json @@ -6,10 +6,11 @@ ], "reason": "New user-facing feature: SCIM provisioning and group-to-role mapping per organization.", "review": { - "status": "draft", + "status": "approved", "evidence": [ "https://github.com/gobifrost/bifrost/pull/1464", - "https://github.com/gobifrost/bifrost/issues/1463" + "https://github.com/gobifrost/bifrost/issues/1463", + "MTG-Thomas approved the highlight entry's exact text on 2026-11-18." ] } } diff --git a/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md b/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md index 17f605023b..2aa7824980 100644 --- a/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md +++ b/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md @@ -33,10 +33,11 @@ }, "assets": [], "review": { - "status": "draft", + "status": "approved", "evidence": [ "https://github.com/gobifrost/bifrost/pull/1464", - "Draft prose; awaiting exact-text approval. The installation-relative route /users/identity-providers was verified in client/src/App.tsx." + "MTG-Thomas approved the exact title and body in the owning T3 conversation on 2026-11-18 (\"I approve of that entry\").", + "The installation-relative route /users/identity-providers was verified in client/src/App.tsx." ] }, "in_app": true From 976e379c2dbe1236664c3e254305acf53102d05c Mon Sep 17 00:00:00 2001 From: Jack Musick Date: Wed, 18 Nov 2026 16:16:41 -0500 Subject: [PATCH 25/26] fix: preserve SCIM group access policy and audit role deltas --- .../services/identity_provisioning/core.py | 27 +++++----- api/tests/e2e/api/test_scim_provisioning.py | 4 ++ .../test_identity_provider_access_pass.py | 19 +++++-- .../test_identity_provisioning_core.py | 51 ++++++++++++++++++- product-updates/dispositions/pr-1464.json | 2 +- .../182c8b4f-4cb8-467c-bc96-0c24b16106b7.md | 2 +- 6 files changed, 84 insertions(+), 21 deletions(-) diff --git a/api/src/services/identity_provisioning/core.py b/api/src/services/identity_provisioning/core.py index c83515330a..98c288bc85 100644 --- a/api/src/services/identity_provisioning/core.py +++ b/api/src/services/identity_provisioning/core.py @@ -452,20 +452,14 @@ async def set_group_members( ) async def delete_group(self, group: IdentityProviderGroup) -> None: - # Deleting a mapped group deletes its mapping: a policy change. + # Directory lifecycle changes cannot remove an administrator's access policy. await self.hold_policy(exclusive=True) - affected = await self.group_member_ids(group.id) - mapped = await self._mapped_group_roles() - if mapped.keys() == {group.id}: - # Deleting the last mapped group lifts the mapped-group - # requirement from every linked user, members or not. - affected = set( - await self.session.scalars( - select(IdentityProviderUser.id).where( - IdentityProviderUser.identity_provider_id == self.connection.id - ) - ) + if group.id in await self._mapped_group_roles(): + raise ProvisioningError( + ProvisioningErrorKind.CONFLICT, + "Unmap this group in Bifrost before deleting it", ) + affected = await self.group_member_ids(group.id) await self.session.delete(group) await self.session.flush() await self._reconcile_access(affected, defer_grants=True) @@ -614,6 +608,15 @@ def queue_group_event(user: User, link: IdentityProviderUser) -> None: link.access_pending_since = None await self.session.flush() + if change.roles_added or change.roles_removed: + await self._audit( + "identity_provider.user_roles_changed", + user, + { + "roles_added": sorted(str(r) for r in change.roles_added), + "roles_removed": sorted(str(r) for r in change.roles_removed), + }, + ) self._mark_stale(user.id, change.roles_added | change.roles_removed) queue_group_event(user, link) if change.activated or change.roles_added or change.roles_removed: diff --git a/api/tests/e2e/api/test_scim_provisioning.py b/api/tests/e2e/api/test_scim_provisioning.py index 542bff4c68..d872b7df09 100644 --- a/api/tests/e2e/api/test_scim_provisioning.py +++ b/api/tests/e2e/api/test_scim_provisioning.py @@ -135,6 +135,10 @@ async def test_scim_user_and_group_lifecycle(e2e_client, platform_admin, db_sess assert grant is not None assert str(grant.identity_provider_id) == scim_connection["connection"]["id"] + refused_delete = e2e_client.delete(f"{path}/Groups/{group['id']}", headers=scim) + assert refused_delete.status_code == 409, refused_delete.text + assert "Unmap" in refused_delete.json()["detail"] + # Okta removes members with a value list rather than a path filter. removed = e2e_client.patch(f"{path}/Groups/{group['id']}", headers=scim, json={ "schemas": [PATCH], "Operations": [{"op": "remove", "path": "members", "value": [{"value": body["id"]}]}], diff --git a/api/tests/unit/services/test_identity_provider_access_pass.py b/api/tests/unit/services/test_identity_provider_access_pass.py index 6c67fad438..cb20e0f816 100644 --- a/api/tests/unit/services/test_identity_provider_access_pass.py +++ b/api/tests/unit/services/test_identity_provider_access_pass.py @@ -24,7 +24,13 @@ from src.models.orm.organizations import Organization from src.models.orm.users import Role, User, UserRole from src.services.identity_provisioning.access_pass import run_access_pass -from src.services.identity_provisioning.core import GroupRecord, Provisioner, UserRecord +from src.services.identity_provisioning.core import ( + GroupRecord, + Provisioner, + ProvisioningError, + ProvisioningErrorKind, + UserRecord, +) _CLEAR_FLAG = text("SELECT set_config('bifrost.authz_epoch_bumped', '', true)") _EPOCH_UPDATES = text( @@ -241,7 +247,7 @@ async def test_a_rename_waits_for_the_pass(setup): @pytest.mark.asyncio -async def test_deleting_the_last_mapped_group_releases_users_outside_it(setup): +async def test_deleting_a_mapped_group_preserves_the_access_requirement(setup): session = setup["session"] group = await _mapped_group(setup) provisioner = Provisioner(session, setup["connection"]) @@ -249,12 +255,15 @@ async def test_deleting_the_last_mapped_group_releases_users_outside_it(setup): outsider = await provisioner.create_user(_record(f"outsider@{setup['domain']}")) assert not (await _refreshed(session, outsider))[0].is_active - # Deleting a group changes policy: its own exclusive-lock transaction. - await Provisioner(session, setup["connection"]).delete_group(group) + with pytest.raises(ProvisioningError) as raised: + await Provisioner(session, setup["connection"]).delete_group(group) + assert raised.value.kind is ProvisioningErrorKind.CONFLICT + mapping = await session.get(IdentityProviderGroupMapping, group.id) + assert mapping is not None await run_access_pass(session) user, link = await _refreshed(session, outsider) - assert user.is_active and link.access_pending_since is None + assert not user.is_active and link.access_pending_since is None @pytest.mark.asyncio diff --git a/api/tests/unit/services/test_identity_provisioning_core.py b/api/tests/unit/services/test_identity_provisioning_core.py index 8ccaa37d6e..b0dc5bd5fe 100644 --- a/api/tests/unit/services/test_identity_provisioning_core.py +++ b/api/tests/unit/services/test_identity_provisioning_core.py @@ -185,8 +185,14 @@ async def test_a_manual_grant_of_a_mapped_role_survives_unmapping(setup): ) await _map_and_apply(session, setup["connection"], group.id, [role.id]) - # Deleting a group changes policy: its own exclusive-lock transaction. - await Provisioner(session, setup["connection"]).delete_group(group) + # An administrator explicitly unmaps before the directory deletes the group. + mapper = Provisioner(session, setup["connection"]) + await mapper.hold_policy(exclusive=True) + mapping = await session.get(IdentityProviderGroupMapping, group.id) + await session.delete(mapping) + await session.flush() + await mapper.reconcile_all() + await mapper.delete_group(group) manual = await session.scalar( select(UserRole).where(UserRole.user_id == link.user_id, UserRole.role_id == role.id) @@ -569,3 +575,44 @@ async def test_a_linked_user_outside_the_connections_org_is_left_alone(setup): assert raised.value.kind is ProvisioningErrorKind.CONFLICT assert (await _user(session, link)).is_active + + +@pytest.mark.asyncio +async def test_role_deltas_are_audited_for_an_already_active_user(setup): + from src.models.orm.audit import AuditLog + + session, connection, role = setup["session"], setup["connection"], setup["role"] + provisioner = Provisioner(session, connection) + link = await provisioner.create_user(_record(f"roles@{setup['domain']}")) + # Membership in an access-only group keeps activity true throughout. + access = await provisioner.create_group( + GroupRecord(display_name="Access", external_id="access", attributes={}), [link.id] + ) + await _map_and_apply(session, connection, access.id, []) + techs = await provisioner.create_group( + GroupRecord(display_name="Techs", external_id="techs", attributes={}), [] + ) + await _map_and_apply(session, connection, techs.id, [role.id]) + await provisioner.set_group_members(techs, [link.id]) + await provisioner.apply_pending([link.id]) + await provisioner.set_group_members(techs, []) + user = await _user(session, link) + assert user.is_active + + rows = ( + await session.scalars( + select(AuditLog).where( + AuditLog.resource_id == link.user_id, + AuditLog.action == "identity_provider.user_roles_changed", + ) + ) + ).all() + assert len(rows) == 2 + assert { + (tuple(row.details["roles_added"]), tuple(row.details["roles_removed"])) for row in rows + } == {((str(role.id),), ()), ((), (str(role.id),))} + for row in rows: + assert row.source == "identity_provider" + assert row.organization_id == connection.organization_id + assert row.user_id is None + assert row.details["identity_provider_id"] == str(connection.id) diff --git a/product-updates/dispositions/pr-1464.json b/product-updates/dispositions/pr-1464.json index 16802bffed..8386f3c8f1 100644 --- a/product-updates/dispositions/pr-1464.json +++ b/product-updates/dispositions/pr-1464.json @@ -10,7 +10,7 @@ "evidence": [ "https://github.com/gobifrost/bifrost/pull/1464", "https://github.com/gobifrost/bifrost/issues/1463", - "MTG-Thomas approved the highlight entry's exact text on 2026-11-18." + "Jack Musick approved the exact title and body in the native SCIM Provisioning and Organization Roles conversation, ordinal 9, on 2026-11-18." ] } } diff --git a/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md b/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md index 2aa7824980..733c69db3e 100644 --- a/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md +++ b/product-updates/entries/182c8b4f-4cb8-467c-bc96-0c24b16106b7.md @@ -36,7 +36,7 @@ "status": "approved", "evidence": [ "https://github.com/gobifrost/bifrost/pull/1464", - "MTG-Thomas approved the exact title and body in the owning T3 conversation on 2026-11-18 (\"I approve of that entry\").", + "Jack Musick approved the exact title and body in the native SCIM Provisioning and Organization Roles conversation, ordinal 9, on 2026-11-18.", "The installation-relative route /users/identity-providers was verified in client/src/App.tsx." ] }, From bfc5d5302702df631cd88d43f544d02b7d292119 Mon Sep 17 00:00:00 2001 From: Jack Musick Date: Wed, 18 Nov 2026 17:03:33 -0500 Subject: [PATCH 26/26] test: include identity provider tasks in scheduler diagnostics --- api/tests/e2e/platform/test_scheduler_diagnostics.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/api/tests/e2e/platform/test_scheduler_diagnostics.py b/api/tests/e2e/platform/test_scheduler_diagnostics.py index 5a8d9470fe..801ac51463 100644 --- a/api/tests/e2e/platform/test_scheduler_diagnostics.py +++ b/api/tests/e2e/platform/test_scheduler_diagnostics.py @@ -68,6 +68,8 @@ def test_admin_sees_every_schedule_and_durable_maintenance_handoffs( "usage_reporting", "logo_thumbnail_backfill", "execution_worker_cleanup", + "identity_provider_token_expiry", + "identity_provider_access_pass", } assert set(tasks) == expected assert tasks["usage_reporting"]["execution_mode"] == "leader"