Skip to content

Memoize env-keyed subprocess probes through one helper: composer, python and ruby each define the same env key and drifted on when an answer is kept #1888

Closed

Description

[agent] Filed by the scheduled architecture audit (audit-core). Register: register.

Kind: refactor. Register ID: A32.

Problem

Three crawlers memoize a subprocess answer (composer config, the Python site query, gem env), keyed on the sorted process environment plus the cwd. The key function is byte-identical in all three:

let mut vars: Vec<_> = std::env::vars_os().collect();
vars.sort();
(vars, std::env::current_dir().ok())
  • composer: EnvKeyedMemo + env_key, composer_crawler.rs L477-L550.`` It is a single slot, and it keeps an answer only if the environment is unchanged after the subprocess returns.
  • python: KeyedMemo<K, V> + site_query_key, python_crawler.rs L2765-L2817.```` It is a single slot and stores every answer (let value = run()?; *lock() = Some((key, value.clone()))). It has no env-unchanged check, so an answer computed while another thread changed the environment is filed under the old key.
  • ruby: GemEnvMemo (a HashMap of OnceCells) + gem_env_key, ruby_crawler.rs L1012-L1023 and L1625-L1650.`````` It keeps every key it has ever seen, never evicts, single-flights per key, and has composer's env-unchanged check.

The drift (python's missing env-unchanged check) comes from reading the code; I did not execute it. It only matters when tests or the CLI change env concurrently.

Proposed change

  • Add one utils::process::EnvKeyedMemo<V> with an env_key(). Use a single slot and composer's rule: keep an answer only when the env is unchanged after the run.
  • Composer and python use it directly.
  • Ruby's async single-flight can wrap the same key type and keep-rule. Alternatively, ruby's map can become the single slot, since no caller needs more than one environment at a time.
  • Delete the three key functions and the two memo structs.

Size

3 files plus utils/process.rs, about −60 lines.

Acceptance criteria

  • Behavior is unchanged. Composer's get_or_run_keyed tests, python's memo test (python_crawler.rs ~L8126) and ruby's memoize_gem_env_homes tests must stay green and move to the shared helper.
  • One test covers the keep rule: an answer computed while the env changed is not cached.

Activity

  1. added
    arch-auditFiled by a scheduled architecture audit routine (see the architecture review discussion)
    refactorStructural change: duplicated code or logic, missing abstraction, layering, dead code
    on Oct 30, 2026
  2. mikolalysenko commented on Oct 30, 2026

    @mikolalysenko
    CollaboratorAuthor

    [agent] Triaged: priority:p1 (touches the Python and RubyGems crawlers as well as Composer). Confirmed on main: the three env-key functions (composer_crawler::env_key, python_crawler::site_query_key, ruby_crawler::gem_env_key) and the two memo structs are present as described. No duplicate or open PR found.


    Generated by Claude Code

  3. mikolalysenko commented on Oct 30, 2026

    @mikolalysenko
    CollaboratorAuthor

    [agent] Claiming this issue (shared root cause: three crawlers each hand-write the same env-keyed subprocess memo). Branch: agent/fix-env-keyed-memo. Claim-ID: 2026-10-30T08:20:54Z-02821a


    Generated by Claude Code

  4. mikolalysenko commented on Oct 30, 2026

    @mikolalysenko
    CollaboratorAuthor

    [agent] Draft PR: #1895


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:triagedarch-auditFiled by a scheduled architecture audit routine (see the architecture review discussion)priority:p1refactorStructural change: duplicated code or logic, missing abstraction, layering, dead code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions