Repository navigation
fix(verl): require full-dataset validation batches - #676
Merged
Zhiyuan He (hzy46) merged 2 commits intoNov 5, 2026
Merged
Zhiyuan He (hzy46) merged 2 commits into
Zhiyuan He (hzy46) merged 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are reported for the change enforcing full-dataset validation.
No open issues 🎉
What changed in this PR
This PR makes Agent Lightning’s full-dataset validation requirement explicit, following the maintainer recommendation in #663.
Changes:
- Add an early assertion requiring
data.val_batch_sizeto beNone, before importing the VERL runtime helper or accessing Ray. - Add six regression cases covering the accepted default and rejected non-null values.
- Document the requirement and correct YAML and CLI syntax.
| File | Description |
|---|---|
tests/verl/test_entrypoint.py |
Tests valid and invalid batch-size values at the Ray startup boundary. |
docs/20-trainer-configuration.md |
Explains full-dataset validation and null configuration syntax. |
agentlightning/verl/entrypoint.py |
Rejects non-null validation batch sizes with an actionable message. |
Review effort: Balanced
Contributor
|
Thanks. Please remove the tests. We do not need a specific test for an assert. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Require VERL's default
data.val_batch_size=nullat the start ofrun_ppo, with an actionable error before importing the VERL runtime helper or accessing Ray. Document the full-dataset validation requirement and correct YAML/CLI syntax.Implements the maintainer's suggestion in #663. The dedicated assertion tests have been removed as requested in review.
Validation on macOS, Python 3.12.13, VERL 0.8.0 and PyTorch 2.13.0, using the frozen CPU dependencies:
.venv/bin/pytest -v --durations=20 tests: 190 passed, 4 skipped (CUDA/vLLM-dependent cases)..venv/bin/pyright: 0 errors, 0 warnings..venv/bin/ruff check .,.venv/bin/ruff format --check .,.venv/bin/python scripts/check_headers.py,.venv/bin/pre-commit run --all-files --show-diff-on-failure, andgit diff --check: passed..venv/bin/mkdocs build --strict: passed.uv build --no-sources: source distribution and wheel built.No live GPU training was run.
AI assistance: prepared and validated with OpenAI Codex.