[BUG] Read JSONL records without splitting Unicode inside strings #4205

Bug description

The JSONL/NDJSON loader calls str.splitlines(), which splits literal U+0085, U+2028 and U+2029 inside otherwise valid JSON strings. A valid physical record becomes a parsing failure.

Minimal reproduction

From a checkout at 0386c53a27951ac51f7a94b3876c69e8e7f84393, run this synthetic example with uv run --frozen --extra dev python (no model or external corpus):

import json
from pathlib import Path
from tempfile import TemporaryDirectory
from openmed.eval.datasets._dua import load_json_rows

with TemporaryDirectory() as directory:
    path = Path(directory) / "synthetic.jsonl"
    rows = [{"text": "synthetic\u2028note"}, {"text": "second note"}]
    path.write_text("\n".join(json.dumps(row, ensure_ascii=False) for row in rows), encoding="utf-8")
    assert load_json_rows(path, dataset="synthetic", authority="test") == rows

Expected behavior

Iterate physical UTF-8 file lines, retaining blank-line handling, physical error-line numbers and every json.loads object_pairs_hook argument.

Error output

ValueError: synthetic JSONL row 1 is invalid: Unterminated string starting at; the separator is inside a valid JSON string.

Native regression validation at the above upstream head produced 9 failed, 5 passed, 0 skipped, 0 errors, using the repository pytest settings and conftest. The failure is the behavior described above, rather than import or setup failure.

OpenMed version

3.0.0, upstream 0386c53a27951ac51f7a94b3876c69e8e7f84393.

Operating system

Ubuntu under WSL2, Linux filesystem.

Python version

3.11.15; development dependencies installed with uv sync --frozen --extra dev --python 3.11; uv.lock SHA-256 0539b1737218cf6c0c5a93c3765c8490314f20171390d3cc208496748cb1a446.

Model ID

N/A (synthetic inputs; no model inference).

PyTorch version / Transformers version

N/A.

Additional context and scope request

The returned collection is still materialized. This is not a constant-memory dataset API, a new JSON grammar or permission to load restricted corpora. Tests use synthetic text only.

Please confirm that this exact scope is accepted and available for a focused contribution. I would like to submit it as an individual PR from my fork after the repository validation gates clear. Final review and integration remain with the maintainers.

Privacy confirmation

• This report uses only synthetic data and contains no PHI, PII, secrets, or real patient data.