Skip to content

Optional local clinical NLI training and export tooling - #3618

Draft
maziyarpanahi wants to merge 1 commit into
masterfrom
feature/v3-nli-checkpoint
Draft

maziyarpanahi wants to merge 1 commit into
masterfrom
feature/v3-nli-checkpoint

Conversation

@maziyarpanahi

@maziyarpanahi maziyarpanahi commented Sep 29, 2026 •

Copy link
Copy Markdown
Owner

Description

Add actual local training and offline export/evaluation commands for a small
clinical NLI encoder, with source-disjoint public/synthetic data and
fail-closed candidate promotion. This advances the remaining checkpoint
delivery work; it does not claim publication or enable a default alias.

Type of Change

  • New feature
  • Test addition/improvement
  • Documentation update

Changes Made

  • Pin the ClinicalE5-Small encoder and non-fiction public three-way NLI data.
    Attribute the author-published biomedical corpus; retain binary negatives
    as partial labels, not invented contradictions. Reject overlapping groups
    and remove whole groups with conflicting duplicate labels before splits.
  • Train on the owner's local Apple hardware with seeded, configuration-backed
    AdamW; select checkpoints on validation only. No paid job, restricted data,
    hosted inference, model upload or visibility-changing operation.
  • Export the same BERT sequence pooler/classifier to PyTorch, dynamic-axis
    ONNX int8 and MLX float32. Add the MLX offline runtime and inference-only
    torch context without introducing mandatory core dependencies.
  • Evaluate the serving int8 export on final heldouts and the separate negation
    challenge. Preserve failed accuracy/calibration/coverage/parity gates in
    aggregate-only reports. Never accept all-abstention as successful coverage
    or fabricate a Hub revision from a candidate weight hash.

Testing

  • Canonical make format, make lint, make format-check passed.
  • Final full suite at ee5104a: 23,170 passed, 179 skipped, 20 existing warnings.
  • Final focused training/NLI gate/backend tests: 59 passed.
  • Native Apple torch/MLX padded segment-pair parity: 6 passed offline.
  • Actual trained snapshot loaded/inferred through torch, ONNX int8 and
    MLX with sockets denied; actual dynamic ONNX/int8/MLX export preflight passed.
  • Historical manually closed parent acceptance audit: 178 passed, 3 optional skips.
  • Strict MkDocs build and wheel/sdist build passed; no package publishing.

Real candidate training/evaluation receipts will be added after the run finishes.
Synthetic unit tests are not model-quality evidence. Final holdouts are not
used for checkpoint selection.

Documentation

  • Updated the clinical NLI model guide, data/license protocol, reproducible
    commands, optional training tools and quality/export boundaries.
  • Added public helper docstrings.

Code Quality

  • Scoped self-review and git diff --check.
  • No unrelated formatting changes, downloaded text, weights, or PLANS files.

Dependencies

  • No new mandatory package dependencies. Training and export use optional
    tools in an isolated environment; imports stay lazy at runtime boundaries.

Related Issues

Related to #3236 and #3233. Keep #3236 open until genuine candidate quality,
independent immutable model publication, offline resolution and registry
activation satisfy its checkpoint acceptance criteria.

@maziyarpanahi maziyarpanahi added this to the v3.0 milestone Sep 29, 2026
@maziyarpanahi maziyarpanahi added feature New capability P0 Critical roadmap-v3 OpenMed V3 roadmap backlog labels Sep 29, 2026
@maziyarpanahi maziyarpanahi changed the title Add local clinical NLI training, exports, and quality gates Optional local clinical NLI training and export tooling Oct 2, 2026
@maziyarpanahi maziyarpanahi removed this from the v3.0 milestone Oct 2, 2026
@maziyarpanahi maziyarpanahi added P3 Strategic and removed P0 Critical roadmap-v3 OpenMed V3 roadmap backlog labels Oct 2, 2026
@maziyarpanahi maziyarpanahi added this to the Backlog milestone Oct 23, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New capability P3 Strategic

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant