diff --git a/.github/PULL_REQUEST_TEMPLATE/data_submission.md b/.github/PULL_REQUEST_TEMPLATE/data_submission.md index 259f4c1e..a8e50638 100644 --- a/.github/PULL_REQUEST_TEMPLATE/data_submission.md +++ b/.github/PULL_REQUEST_TEMPLATE/data_submission.md @@ -1,14 +1,22 @@ # Pull Request Template: Benchmark Data Submission -Only one of the following check lists need be used. +Only one of the system check lists below need be used. -## New System Checklist +## Change level +- [ ] `major`: changes existing data +- [ ] `minor`: additive (new system, charge set) +- [ ] `bug`: fix, script, docs + +See [RELEASE.md](../../RELEASE.md) for more details. + +## System Checklist Please ensure the following criteria are met before submitting the PR: +- [ ] Labeled PR with version level `major`, `minor`, or `bug` +- [ ] [CHANGELOG.md](../../CHANGELOG.md) updated under `Unreleased` - [ ] **Data Format**: All files are in the expected format as outlined in `openfe_benchmarks/data/benchmark_systems/README.md`. - [ ] A `ligand.sdf` file is present -- [ ] The `charge_molecules.py` file was run using the `conda-lock_linux-64.yml` env on `ligands.sdf` - [ ] (If appropriate) A `protein.pdb` file is present for binding free energy calculations (BFE) - [ ] (If appropriate) The `charge_molecules.py` file was run using the `conda-lock_linux-64.yml` env on `cofactors.sdf` - [ ] (If appropriate) Produce ligand network for RFE calculations @@ -19,15 +27,4 @@ Please ensure the following criteria are met before submitting the PR: - [ ] Ensure this new system_group/system_name is represented in `benchmark_system_indexing.yml` - [ ] If there is a new partial charge method, it must be added to PARTIAL_CHARGE_TYPES in `_benchmark_systems.py` -## Update System Checklist - -Please ensure the following criteria are met before submitting the PR: - -- [ ] **Data Format**: All files are in the expected format as outlined in `openfe_benchmarks/data/benchmark_systems/README.md`. -- [ ] (If appropriate) The `charge_molecules.py` file was run using the `conda-lock_linux-64.yml` env on `ligands.sdf` -- [ ] (If appropriate) The `charge_molecules.py` file was run using the `conda-lock_linux-64.yml` env on `cofactors.sdf` -- [ ] (If appropriate) Produce ligand network for RFE calculations -- [ ] **Documentation**: A `PREPARATION_DETAILS.md` file includes a changelog of date and what was changed in the system -- [ ] If there is a new partial charge method, it must be added to PARTIAL_CHARGE_TYPES in `_benchmark_systems.py` - Thank you for contributing to the OpenFE Benchmarks! \ No newline at end of file diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 8a8e125d..8e63512c 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,36 +1,25 @@ -# Pull Request Template: Benchmark Results Submission +# Benchmark Results Submission ## Description -[Provide a brief description of this benchmark submission, including what systems were calculated and any notable features] + -## Required Files Checklist -Please ensure the following files are present in your submission directory: +## Change level: +See [RELEASE.md](../RELEASE.md). +- [ ] `major`: changes existing data +- [ ] `minor`: additive +- [ ] `bug`: fix, script, docs, new results -- [ ] **`submission.yaml`**: Complete metadata file with all required fields using `openfe_benchmarks/scripts/prepare_metadata_submission.py` -- [ ] **`computational_results.json.bz2`**: Compressed results file using `openfe_benchmarks/scripts/generate_results_archives.py` -- [ ] **Archive DOI**: Results uploaded to long-term archive (Zenodo, etc.) with DOI included in submission.yaml +## Submission +- [ ] Labeled PR with version level `major`, `minor`, or `bug` +- [ ] `submission.yaml` made with `openfe_benchmarks/scripts/prepare_metadata_submission.py` +- [ ] `computational_results.json.bz2` made with `openfe_benchmarks/scripts/generate_results_archives.py` +- [ ] Single experiment; settings differing from OpenFE defaults described in `summary` of this PR submission +- [ ] [CHANGELOG.md](../../CHANGELOG.md) updated under `Unreleased` -## Validation Checklist +## Checked by CI +- `submission.yaml` is valid and loads +- `submission_id` matches directory name and is unique +- Results file exists at the path in `results` -- [ ] **YAML validation**: `submission.yaml` is valid YAML and loads without errors -- [ ] **ID consistency**: `submission_id` in YAML matches directory name -- [ ] **Results file exists**: Compressed results file exists at path specified in `results` field -- [ ] **Archive accessible**: DOI resolves and archive is publicly accessible -- [ ] **Network keys valid**: AlchemicalNetwork keys in `benchmark_data` are valid -- [ ] **Calculation type valid**: Type matches actual calculations performed -- [ ] **No duplicate submission_id**: This submission_id is unique in the repository -- [ ] **Single experiment**: This submission contains results from a single experiment (not multiple experiments combined) and details of the experiment are clearly described in the metadata (see summary field) -- [ ] **Settings and parameters**: All noteable settings differences from default OpenFE settings are clearly described in the metadata (see summary field) also consider adding a tag in the tags field - -## Testing - -- [ ] CI validation checks pass - -## Additional Notes - -[Any additional context, special considerations, or notes about this submission] - ---- - -Thank you for contributing to the OpenFE Benchmarks! \ No newline at end of file +## Notes diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000..69106a14 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,21 @@ +# Changelog + +Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning follows [RELEASE.md](RELEASE.md). + +Each PR adds one line under `Unreleased`, in the matching section, ending with the PR number and its level ([major], [minor], or [bug]): +`- [minor] Description (#123)` + +Sections: **Added** (new data or features), **Changed** (changes to existing data or behavior; usually major), **Fixed** (bug fixes, new results, non-data changes), **Removed**. + +## [Unreleased] + +### Added + +- [major] Initial tagged release. No changelog was kept before this version; see `git log v1.0.0`. + +### Changed + +### Fixed + +### Removed + diff --git a/RELEASE.md b/RELEASE.md new file mode 100644 index 00000000..7bbdd730 --- /dev/null +++ b/RELEASE.md @@ -0,0 +1,28 @@ +# Release Policy of OpenFE-Benchmarks + +OpenFE-Benchmarks uses **Data-SemVer**: semantic versioning applied to published benchmark data, where "breaking" means breaking for someone consuming the data. + +- Version format is `X.Y.Z`, which is PEP 440 compliant. +- Releases are git tags `vX.Y.Z` on `main`. + +## Version levels + +| Level | Meaning | Examples | +|---|---|---| +| **X** (major) | Changes to existing data; not backward compatible | Editing or removing values in an existing benchmark data system; renaming or removing `submission.yaml` fields | +| **Y** (minor) | Independent additions; backward compatible | New datasets or systems; new charge sets; new results submission; new optional fields or script features | +| **Z** (bug) | Fixes and changes that do not alter published data | Script fixes, docs, CIs | + +Tie-breaker: if a change fits more than one level, use the highest. + +## Process + +1. The PR author picks the level and labels the PR `major`, `minor`, or `bug`. +2. The PR author adds an entry under `Unreleased` in [CHANGELOG.md](CHANGELOG.md). +3. At release time, maintainers cut a release by moving `Unreleased` into a dated `X.Y.Z` section and tagging `vX.Y.Z` on `main`. + +Note: Major changes may be submitted as PRs but are not merged without agreement between the Science Team Leads for both OpenFE and OpenFF. + +## Zenodo and citation + +Citation and DOI are obtained from Zenodo / GitHub integration for each `vX.Y.Z` tag.