Skip to content

Extend MLX backend coverage to ModernBERT and Longformer architectures - #2455

Merged
maziyarpanahi merged 8 commits into
masterfrom
feature/om-167-mlx-modernbert-longformer
Aug 21, 2026
Merged

maziyarpanahi merged 8 commits into
masterfrom
feature/om-167-mlx-modernbert-longformer

Conversation

@maziyarpanahi

@maziyarpanahi maziyarpanahi commented Aug 9, 2026 •

Copy link
Copy Markdown
Owner

Description

Adds native MLX token-classification backends for ModernBERT and Longformer, including architecture-specific normalization, converted-weight remapping, rotary/local attention, Longformer global attention, artifact loading, and INT8 shape preservation.

Type of Change

  • New feature
  • Test addition/improvement

Testing

Validated at exact head 17cd2075ad72d11cb0ac043552035ea949a3c11d against master 2c8c7b13ab757bd205a09899df0d33ca7aa2faa4 on Apple Metal:

  • Full repository suite: 13,058 passed, 130 skipped.
  • Complete MLX and backend unit suite: 188 passed, 1 skipped.
  • Export-matrix coherence suite: 3 passed.
  • ModernBERT, Longformer, model registry, and converter mypy checks passed.
  • Scoped pre-commit, Ruff, format, Bandit, secret, and diff checks passed.
  • Strict documentation artifact build passed.
  • Exact staged-artifact Chromium byte-budget gate passed.

Documentation

The MLX export support matrix and public family-resolution errors now include both backends and every supported alias.

Dependencies

No new dependency; tests use the existing optional MLX, PyTorch, and Transformers extras.

Related Issues

Closes #2340
Closes #2345

Screenshots/Examples

Not applicable; parity is covered by deterministic tiny reference models.

@maziyarpanahi maziyarpanahi added roadmap-v2 OpenMed V2 roadmap backlog feature New capability P2 Medium labels Aug 9, 2026
@maziyarpanahi

Copy link
Copy Markdown
Owner Author

Maintainer refresh on exact head 6393d30 against master 7c93a8e: the complete Metal-backed MLX suite passed with 166 tests and 1 expected skip; the 42 focused conversion, parity, and FP/INT8 tests passed with no skips; mypy and scoped hooks passed. This branch now consolidates #2340 and #2345; #2481 was closed as superseded. Hosted exact-head checks are starting. No merge performed.

@maziyarpanahi

Copy link
Copy Markdown
Owner Author

Hosted Apple Silicon integration exposed a documentation-coherence failure: the newly supported ModernBERT and Longformer aliases were missing from docs/export-matrix.md. Repaired on exact head 3917459. The focused export-matrix test passes (3 tests), scoped hooks pass, and the strict documentation build passes. Exact-head CI is restarting.

@maziyarpanahi
maziyarpanahi merged commit d3d98b1 into master Aug 21, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New capability P2 Medium roadmap-v2 OpenMed V2 roadmap backlog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extend MLX backend architecture coverage for student-model export Extend MLX backend coverage to ModernBERT and Longformer architectures

1 participant