Repository navigation
feat: admin API and dashboard for organization encryption transitions - #9446
Conversation
https://linear.app/speakeasy/issue/AIM-1007/feat-admin-api-and-dashboard-for-organization-encryption-transitions Staff can now see where each organization stands in moving its OAuth client secrets and session tokens from the platform key to its own key, and drive the move from the admin dashboard. Admin API (gram admin): - getOrganizationEncryption reports the key configuration and, for each use case that moves, the transition state, live counts of waiting, unreadable and unattributed values, the last backfill and verification, and why finalize would be refused. - listOrganizationEncryptionTransitions lists organizations by state for one use case, counting organizations that were never enrolled as platform_key. - enroll, rollback and run act on one organization. Run backfills and verifies in dual_write, and finishes clearing platform-key copies in org_kms. - finalize requires a typed confirmation and the number of unreadable values the administrator acknowledged. It refuses when that number no longer matches. Once org_kms is recorded it starts the run that clears the platform-key copies; if that cannot start, the hourly run finishes it. - Writes are keyed on the organization id alone, and every change is audited to the staff member, including the clear the run performs. - The methods are unavailable unless organization encryption is configured. Runs also need Temporal. Admin dashboard: - An Encryption section on the organization page, with a card per use case and the enroll, run, roll back and finalize actions. The finalize dialog states what is destroyed, that rollback is no longer possible, and the unreadable count, which the administrator must acknowledge again if it changes. The page refreshes while a backfill or clear may be running. - An Encryption transitions page lists organizations by state and enrolls a selected batch, or the next N in platform_key, one at a time at about one a second. Failures are reported without stopping the rest, and the batch can be stopped. Shared helper changes: - Finalize also refuses while the organization's key is suspended, and checks the acknowledged unreadable count under the state row lock. - A clear that loses a race with another clear reports success instead of an error. - The on-demand run's activity timeout fits a full clear (8 minutes). Admin MCP: no tools added. Transitions are rare staff-only steps and finalize is destructive, so they stay deliberate dashboard actions. Temporal actions/month: unchanged per namespace except 2 to 4 per on-demand run, including one per finalize. It doesn't scale with organizations.
🦋 Changeset detectedLatest commit: 2df9579 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Demo: screenshotsCaptured against a local stack with organization encryption enabled. The red rings mark the new UI. Encryption transitions page, with its new admin sidebar entry: Encryption section on an organization page, in dual write with finalize blocked and the blocker listed: After finalize, once the background run cleared the platform-key copies: |
Demo: bulk enrollWhat it shows:
02-bulk-enroll-trimmed.webm |
Demo: finalizeWhat it shows:
04-finalize-trimmed.webm |
There was a problem hiding this comment.
cubic analysis
4 issues found across 79 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="client/admin/src/sdk/src/models/components/finalizeorganizationencryptionrequestbody.ts">
<violation number="1" location="client/admin/src/sdk/src/models/components/finalizeorganizationencryptionrequestbody.ts:42">
P2: This schema omits the API’s non-negative constraint, so callers can pass a negative acknowledgement count and receive a server-side 400 after the SDK accepts and sends it. Add a `z.gte(0)` check to keep outbound validation aligned with the contract.</violation>
</file>
<file name="server/internal/admin/organizationencryption.go">
<violation number="1" location="server/internal/admin/organizationencryption.go:184">
P1: According to linked Linear issue AIM-1007, `oauth-session-tokens` enrollment must be blocked until AIM-1004 is deployed in every OAuth-serving process. This path has no rollout/readiness check, so an unupdated process can keep relying on platform copies while finalization destroys them; add the gate before allowing the first enrollment.</violation>
<violation number="2" location="server/internal/admin/organizationencryption.go:224">
P2: According to linked Linear issue AIM-1007, every admin action, including “run the backfill now,” must be audited and attributed to the staff member. A dual-write run started here has no audit record because the actor is ignored by the backfill path; record an internal staff-attributed run event when starting the workflow.</violation>
</file>
<file name="client/admin/src/sdk/src/react-query/adminListOrganizationEncryptionTransitions.core.ts">
<violation number="1" location="client/admin/src/sdk/src/react-query/adminListOrganizationEncryptionTransitions.core.ts:52">
P3: The query key does not normalize the request defaults, so omitted pagination and explicit `{ limit: 50, offset: 0 }` create separate cache entries for the same API request. Normalize both values in the key.</violation>
</file>
Linked issue analysis
Linked issue: AIM-1007: feat: admin API and dashboard for organization encryption transitions
| Status | Acceptance criteria | Notes |
|---|---|---|
| ✅ | Show configuration and transition progress for moving use cases | The API reports configuration, transition state, live counts, and backfill and verification history for both OAuth use cases. |
| ✅ | Support enrollment, rollback, backfill, and finalize actions | The API and organization page expose these actions, with finalize blockers reported and checked before the transition. |
| ✅ | Enroll organizations sequentially and continue after individual failures | Bulk enrollment runs one organization at a time and reports failures without stopping the remaining enrollments. |
| ✅ | Finalize safely, acknowledge unreadable values, and resume clearing | Finalize rechecks the acknowledged unreadable count under lock, records org_kms first, and supports resuming an incomplete clear. |
| ✅ | Provide organization and transition dashboards with deliberate confirmation | The dashboard adds per-organization controls, state-based listings, and a dialog requiring count acknowledgement and the typed phrase. |
| ✅ | Audit transition actions to the staff member internally | The PR says actions, including clearing, carry the staff actor and are recorded as internal audit events. |
| ❌ | Check OAuth token-writer rollout before enrollment | The issue requires checking AIM-1004 on every OAuth-serving process before enrollment, but the PR description shows no such check. |
This PR changes a database schema or migrates data. Ultrareviews find 2.4x more serious bugs than standard reviews. Comment @cubic-dev-ai ultrareview to run one.
View guided diff | Turn on auto-fix | Re-trigger cubic
| return nil, err | ||
| } | ||
|
|
||
| if _, err := s.orgEncryption.Transitions.Enroll(ctx, organizationID, move.UseCase, encryptionActor(ctx)); err != nil { |
There was a problem hiding this comment.
P1: According to linked Linear issue AIM-1007, oauth-session-tokens enrollment must be blocked until AIM-1004 is deployed in every OAuth-serving process. This path has no rollout/readiness check, so an unupdated process can keep relying on platform copies while finalization destroys them; add the gate before allowing the first enrollment.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At server/internal/admin/organizationencryption.go, line 184:
<comment>According to linked Linear issue AIM-1007, `oauth-session-tokens` enrollment must be blocked until AIM-1004 is deployed in every OAuth-serving process. This path has no rollout/readiness check, so an unupdated process can keep relying on platform copies while finalization destroys them; add the gate before allowing the first enrollment.</comment>
<file context>
@@ -0,0 +1,500 @@
+ return nil, err
+ }
+
+ if _, err := s.orgEncryption.Transitions.Enroll(ctx, organizationID, move.UseCase, encryptionActor(ctx)); err != nil {
+ return nil, mapOrganizationEncryptionError(ctx, logger, err, "enroll")
+ }
</file context>
There was a problem hiding this comment.
Declining: AIM-1004 (sealing OAuth session tokens with organization keys) is already merged and deployed to every process that serves OAuth, so there is no rollout state left for a gate to check. A code gate would also have nothing reliable to read, since deployment state is not stored anywhere the admin process can see.
| FinalizeOrganizationEncryptionRequestBody | ||
| > = z.pipe( | ||
| z.object({ | ||
| acknowledgedUnreadableValues: z.int(), |
There was a problem hiding this comment.
P2: This schema omits the API’s non-negative constraint, so callers can pass a negative acknowledgement count and receive a server-side 400 after the SDK accepts and sends it. Add a z.gte(0) check to keep outbound validation aligned with the contract.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At client/admin/src/sdk/src/models/components/finalizeorganizationencryptionrequestbody.ts, line 42:
<comment>This schema omits the API’s non-negative constraint, so callers can pass a negative acknowledgement count and receive a server-side 400 after the SDK accepts and sends it. Add a `z.gte(0)` check to keep outbound validation aligned with the contract.</comment>
<file context>
@@ -0,0 +1,65 @@
+ FinalizeOrganizationEncryptionRequestBody
+ > = z.pipe(
+ z.object({
+ acknowledgedUnreadableValues: z.int(),
+ confirmation: z.string(),
+ organizationId: z.string(),
</file context>
| acknowledgedUnreadableValues: z.int(), | |
| acknowledgedUnreadableValues: z.int().check(z.gte(0)), |
There was a problem hiding this comment.
Declining: this file is generated by the Speakeasy SDK generator from the OpenAPI document and cannot be edited by hand. The minimum is declared in the design (Minimum(0) on acknowledged_unreadable_values), Goa validates it on the server, and the dashboard only ever sends the non-negative count it read from the API.
| return nil, oops.E(oops.CodeConflict, nil, "the platform-key copies are already cleared; there is nothing left to run").LogInfo(ctx, logger) | ||
| } | ||
|
|
||
| if err := s.orgEncryption.Runs.StartOrganizationEncryptionRun(ctx, organizationID, move.UseCase, encryptionActor(ctx)); err != nil { |
There was a problem hiding this comment.
P2: According to linked Linear issue AIM-1007, every admin action, including “run the backfill now,” must be audited and attributed to the staff member. A dual-write run started here has no audit record because the actor is ignored by the backfill path; record an internal staff-attributed run event when starting the workflow.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At server/internal/admin/organizationencryption.go, line 224:
<comment>According to linked Linear issue AIM-1007, every admin action, including “run the backfill now,” must be audited and attributed to the staff member. A dual-write run started here has no audit record because the actor is ignored by the backfill path; record an internal staff-attributed run event when starting the workflow.</comment>
<file context>
@@ -0,0 +1,500 @@
+ return nil, oops.E(oops.CodeConflict, nil, "the platform-key copies are already cleared; there is nothing left to run").LogInfo(ctx, logger)
+ }
+
+ if err := s.orgEncryption.Runs.StartOrganizationEncryptionRun(ctx, organizationID, move.UseCase, encryptionActor(ctx)); err != nil {
+ return nil, oops.E(oops.CodeUnexpected, err, "error starting the organization encryption run").LogError(ctx, logger)
+ }
</file context>
There was a problem hiding this comment.
Declining by design: "Run backfill now" changes no transition state, so it records no audit event. The work it can lead to that does change state is already attributed to the staff member: the clear it finishes in org_kms carries the requesting actor through the workflow params into ClearPlatformCopies, which records organization_encryption_transition:clear_platform_copies with that actor. Enroll, rollback and finalize are each audited to the staff member as well.
| limit: request?.limit, | ||
| offset: request?.offset, |
There was a problem hiding this comment.
P3: The query key does not normalize the request defaults, so omitted pagination and explicit { limit: 50, offset: 0 } create separate cache entries for the same API request. Normalize both values in the key.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At client/admin/src/sdk/src/react-query/adminListOrganizationEncryptionTransitions.core.ts, line 52:
<comment>The query key does not normalize the request defaults, so omitted pagination and explicit `{ limit: 50, offset: 0 }` create separate cache entries for the same API request. Normalize both values in the key.</comment>
<file context>
@@ -0,0 +1,92 @@
+ queryKey: queryKeyAdminListOrganizationEncryptionTransitions({
+ useCase: request?.useCase,
+ state: request?.state,
+ limit: request?.limit,
+ offset: request?.offset,
+ }),
</file context>
| limit: request?.limit, | |
| offset: request?.offset, | |
| limit: request?.limit ?? 50, | |
| offset: request?.offset ?? 0, |
There was a problem hiding this comment.
Declining: this file is generated by the Speakeasy SDK generator and cannot be edited by hand. The dashboard always passes explicit limit and offset values for list queries, and invalidates by the shared key prefix after changes, so the two forms never coexist in practice.
https://linear.app/speakeasy/issue/AIM-1007/feat-admin-api-and-dashboard-for-organization-encryption-transitions - gram admin builds organization encryption whether or not the application encryption key is set, so the transition methods no longer depend on it. Only the remote session service still needs both. - Finalize requires the acknowledged unreadable count instead of treating it as optional. - The configuration response declares its mode and state enums. - Bulk enroll uses the use case the URL names, and "select next" pages past disabled organizations until it has enough. - The finalize dialog's own button follows blockers that appear while it is open, and the dialog scrolls on short screens. - The command palette finds the encryption pages by rollback, roll back and finalize. - Tests prove enrollment runs one organization at a time, that the organization-key copy still opens after clearing, and the exact finalize blocker messages.



https://linear.app/speakeasy/issue/AIM-1007/feat-admin-api-and-dashboard-for-organization-encryption-transitions
Stacked on #9311 (AIM-1006, the background backfill and verification). Review and merge that first; this PR then retargets
main.Summary
Staff can now see where each organization stands in moving its OAuth client secrets and session tokens from the platform key to its own key, and drive that move from the admin dashboard.
Admin API (
server/design/admin/organizationencryption.go,server/internal/admin/organizationencryption.go)getOrganizationEncryption: the key configuration and, for each use case that moves (oauth-client-secrets,oauth-session-tokens), the transition state, live counts of waiting, unreadable and unattributed values, the last backfill and verification, and the reasons finalize would be refused.tailscale-gateway-stateis left out because it never had platform-key copies.listOrganizationEncryptionTransitions: organizations by state for one use case, with counts per state. Organizations that were never enrolled count asplatform_key.enrollOrganizationEncryption,rollbackOrganizationEncryption,runOrganizationEncryption: run starts an on-demand Temporal run that backfills and verifies indual_write, or finishes clearing the platform-key copies inorg_kms.finalizeOrganizationEncryption: requires the typed phrasedestroy platform-key copiesand the number of unreadable values the admin acknowledged.Transitions.Finalizecompares that number under the state row lock and refuses if it changed. Afterorg_kmsis recorded, the handler starts the clearing run; if it cannot start, the hourly run finishes the clear and the response says so.gram adminwires this only when organization encryption is configured; otherwise the methods return 503. Without Temporal, only run is unavailable.Admin dashboard (
client/admin)Shared helper (
server/internal/orgencryption,server/internal/background)FinalizeRequest.ExpectedUnreadable, and finalize refuses while the organization's key is suspended.ClearPlatformCopiesreports success when another clear (such as the hourly resume) finished first, instead of a wrappedno rowserror.Migrator.RunOnDemandbacks the on-demand workflow, whose activity timeout now fits a full clear (8 minutes).CountStoresandFinalizeBlockersfor the admin view.FinalizeBlockersis advisory; finalize checks again under the lock.Motivation
Moving an organization to its own key ends with an irreversible step: finalize destroys the platform-key copies. Staff need to see each organization's progress, enroll organizations in controlled batches (each enroll can mint a KMS key), and finalize deliberately once the backfill and verification are clean. The ordering, fencing and audit events already live in the shared helper (AIM-1001), so this PR calls it rather than reimplementing them, and keeps finalize synchronous so the acknowledged count is checked at the moment of the change. Clearing runs as a Temporal run so it survives admin restarts, and bulk enroll runs from the dashboard so a long batch never depends on one HTTP request.
Admin MCP parity: no tools added. Transitions are rare, staff-only platform steps, and finalize is destructive, so they stay deliberate dashboard actions behind a typed confirmation. The decision is recorded in the design file.
Temporal actions/month ≈ 2 to 4 per on-demand run (including one per finalize) on top of AIM-1006's ~1,440 per namespace, scales with fixed (staff actions only, not organizations).
Gutternote review steps
Setup: preview deployments run
gram adminwithout organization encryption, so in a preview both pages report that organization encryption is not available on this server. The demo below shows the flows against a local stack with it enabled./encryption-transitions). Expect the use case and state filters, counts per state, Batch size, Select next in platform key and Enroll selected (N) above a table of organizations./organizations/<org>/encryption). Expect a Key card and a card per use case with its state badge, counts, last backfill and verification. In dual write, Finalize is disabled and the blockers are listed.destroy platform-key copies. After finalize, the badge shows CLEARING and then ORGANIZATION KEY with the copies cleared.Also check: the 503 message when organization encryption is not configured, and the command palette entries for Encryption (organization) and Encryption transitions.
Summary by cubic
Staff can now see where each organization stands in moving its OAuth client secrets and session tokens from the platform key to its own key, and drive the move from the admin dashboard.
Admin API
gram adminbuilds organization encryption independently of the application encryption key; methods return 503 unless organization encryption is configured, and run additionally requires Temporal.finalizerequires the typed phrasedestroy platform-key copiesand an acknowledged unreadable count, re-checked under the state row lock; it refuses while the organization's key is suspended.org_kms, it starts a run that clears the platform-key copies; if that run cannot start, the hourly background run finishes the clear.Admin dashboard
Written for commit 2df9579. Summary will update on new commits.
🚀 Preview Environment (PR #9446)
Preview URL: https://pr-9446.dev.getgram.ai
Gram Preview Bot