Skip to content

HEVCParser: don't loop forever on a NAL header with forbidden_zero_bit set - #1529

Merged
kmdewaal merged 1 commit into
MythTV:masterfrom
gsikorski:fix-hevcparser-forbidden-zero-bit-loop
Nov 5, 2026
Merged

kmdewaal merged 1 commit into
MythTV:masterfrom
gsikorski:fix-hevcparser-forbidden-zero-bit-loop

Conversation

@gsikorski

Copy link
Copy Markdown
Contributor

Resolves #1528

HEVCParser::addBytes() returns the number of bytes used, but its forbidden_zero_bit error path returned false, which is 0. DTVRecorder::FindHEVCKeyframes() then does i += (bytes_used - 1) and ++i, hands the parser the same bytes again, and the recorder thread loops on that TS packet forever. Each pass logs the error, and the log queue grows until the OOM killer kills mythbackend, along with every recording in progress.

This change skips the bad NAL header the same way the function's other early return does: resetRBSP(); return startP - bytes;. startP is already past the header byte, so the result is always >= 1.

Testing (on fixes/34, the Ubuntu 24.04 package rebuilt with this change; master's lines are identical):

  • A clean DVB-T2 HEVC recording with 00 00 01 80 01 planted in one video TS packet, played through an External Recorder input.
  • Without the fix: RSS 79 MB to 5.5 GB in 90 s, OOM kill, recording split in two.
  • With the fix: one log line, RSS flat at 77 MB, recording complete and the same size as the clean one.

The same code is in fixes/34, fixes/35 and fixes/36, so a backport would help those branches too.

Checklist

🤖 Generated with Claude Code

…t set

addBytes() returns the number of bytes used, but the forbidden_zero_bit
error path returned "false", i.e. 0. DTVRecorder::FindHEVCKeyframes()
then does "i += bytes_used - 1" and "++i", so it hands the parser the
same bytes again, the parser finds the same start code, logs the same
error and returns 0 again: the recorder thread never leaves that TS
packet. The log queue grows by ~85k messages/s until the kernel's OOM
killer kills mythbackend (20 GB RSS on a 32 GB machine), taking every
recording in progress with it.

A DVB-T2 reception error only has to produce 00 00 01 followed by a byte
>= 0x80 in the HEVC payload. Reproduced by planting 00 00 01 80 01 in
one video TS packet of a clean recording replayed through an External
Recorder: 79 MB -> 5.5 GB RSS in 90 s, recording split in two. With
this change: one log line, RSS flat, recording complete.

Skip the bad NAL header (reset the RBSP state and report the bytes used
up to and including it), as the function's other early return does.

The same code is in fixes/35, fixes/36 and master.

Resolves: MythTV#1528

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kmdewaal kmdewaal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@kmdewaal
kmdewaal merged commit 147b27e into MythTV:master Nov 5, 2026
7 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Backend OOM: HEVCParser loops forever on a NAL header with forbidden_zero_bit set

2 participants