Skip to content

Fix out-of-bounds read in is_valid_utf8 on truncated multi-byte sequences - #229

Merged
kgabis merged 2 commits into
kgabis:masterfrom
afonsojanu:fix/utf8-validation-oob-read-clean
Sep 15, 2026
Merged

kgabis merged 2 commits into
kgabis:masterfrom
afonsojanu:fix/utf8-validation-oob-read-clean

Conversation

@afonsojanu

Copy link
Copy Markdown
Contributor

While looking at the *_with_len string setters I noticed verify_utf8_sequence decides how many bytes a UTF-8 sequence needs purely from its lead byte, then unconditionally reads that many bytes to build the code point, without checking whether that many bytes are actually still inside the caller's buffer.

If a string's last byte is a 2/3/4-byte lead byte whose continuation bytes have been cut off, the read goes past the given length. This is directly reachable from public API such as json_value_init_string_with_len and json_object_set_string_with_len, whenever the buffer isn't followed by a null byte, which is exactly the case those _with_len entry points are meant to support (binary-safe / length-prefixed strings).

Repro under AddressSanitizer:

size_t n = 3;
char *buf = malloc(n);
buf[0] = 'a'; buf[1] = 'b'; buf[2] = (char)0xF0; /* 4-byte lead, no continuation bytes */
json_value_init_string_with_len(buf, n);
==ERROR: AddressSanitizer: heap-buffer-overflow ...
READ of size 1 ...
    #0 verify_utf8_sequence parson.c:356
    #1 is_valid_utf8 parson.c:396
    #2 json_value_init_string_with_len parson.c:1659

The fix threads the number of bytes remaining in the input down into verify_utf8_sequence and bails out before it looks past that, instead of only trusting num_bytes_in_utf8_sequence() to know how far it's safe to look.

Added cases to test_suite_5 in tests.c covering truncated 2/3/4-byte sequences sitting at the very end of an exact-size heap buffer (these crash under ASan without the fix), plus a full 4-byte sequence occupying the whole buffer to make sure valid input at the boundary still parses. Ran the existing suite under ASan+UBSan (all 356 checks pass) and also rebuilt with the project's own -std=c89 -pedantic-errors and the C++ build, both clean.

(Replaces #228, which had a stray commit trailer that shouldn't have been there — same fix and tests, clean history.)

verify_utf8_sequence decides how many bytes a UTF-8 sequence needs
from its lead byte, then reads that many bytes to compute the code
point, but it never checked whether that many bytes actually remain
before the end of the caller-supplied length. A string whose last
byte is a 2/3/4-byte lead byte with the trailing continuation bytes
cut off gets read past the buffer that was handed in.

This is reachable straight from public API, e.g.
json_value_init_string_with_len / json_object_set_string_with_len,
whenever the caller passes a length that isn't followed by a null
byte (which is exactly what those *_with_len entry points exist
for, e.g. binary-safe or length-prefixed strings). With ASan,
handing json_value_init_string_with_len a heap buffer whose last
byte is 0xF0 (and nothing else) aborts with a heap-buffer-overflow
in verify_utf8_sequence.

Pass the number of bytes left in the input down to
verify_utf8_sequence and bail out before it looks past that, instead
of only relying on num_bytes_in_utf8_sequence() to know how far it's
safe to look. Added test_suite_5 cases covering truncated 2/3/4-byte
sequences at the end of an exact-size heap buffer (which used to
crash) plus a full 4-byte sequence occupying the whole buffer, to
check nothing gets rejected that shouldn't be.
@kgabis
kgabis merged commit bdd339a into kgabis:master Sep 15, 2026
@kgabis

kgabis commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Thanks, I've modified it a bit so that it's more aligned with parson's code style. Also changed tests so that this bug can be caught without asan.

ewertons added a commit to Azure/azure-iot-sdk-c that referenced this pull request Oct 23, 2026
Includes the fix for an out-of-bounds read in is_valid_utf8 on truncated
multi-byte sequences (kgabis/parson#229). No API changes.
ewertons added a commit to Azure/azure-iot-sdk-c that referenced this pull request Oct 26, 2026
Includes the fix for an out-of-bounds read in is_valid_utf8 on truncated
multi-byte sequences (kgabis/parson#229). No API changes.
ewertons added a commit to Azure/azure-iot-sdk-c that referenced this pull request Oct 27, 2026
….4 (#2781)

* Update parson to 1.5.4 (ec53fb6) (#2779)

Includes the fix for an out-of-bounds read in is_valid_utf8 on truncated
multi-byte sequences (kgabis/parson#229). No API changes.

(cherry picked from commit cecc78d)

* IoTHub_Init initializes c-logging (backport of #2778)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants