Repository navigation
Fix out-of-bounds read in is_valid_utf8 on truncated multi-byte sequences - #229
Merged
kgabis merged 2 commits intoSep 15, 2026
Merged
Conversation
verify_utf8_sequence decides how many bytes a UTF-8 sequence needs from its lead byte, then reads that many bytes to compute the code point, but it never checked whether that many bytes actually remain before the end of the caller-supplied length. A string whose last byte is a 2/3/4-byte lead byte with the trailing continuation bytes cut off gets read past the buffer that was handed in. This is reachable straight from public API, e.g. json_value_init_string_with_len / json_object_set_string_with_len, whenever the caller passes a length that isn't followed by a null byte (which is exactly what those *_with_len entry points exist for, e.g. binary-safe or length-prefixed strings). With ASan, handing json_value_init_string_with_len a heap buffer whose last byte is 0xF0 (and nothing else) aborts with a heap-buffer-overflow in verify_utf8_sequence. Pass the number of bytes left in the input down to verify_utf8_sequence and bail out before it looks past that, instead of only relying on num_bytes_in_utf8_sequence() to know how far it's safe to look. Added test_suite_5 cases covering truncated 2/3/4-byte sequences at the end of an exact-size heap buffer (which used to crash) plus a full 4-byte sequence occupying the whole buffer, to check nothing gets rejected that shouldn't be.
Owner
|
Thanks, I've modified it a bit so that it's more aligned with parson's code style. Also changed tests so that this bug can be caught without asan. |
ewertons
added a commit
to Azure/azure-iot-sdk-c
that referenced
this pull request
Oct 23, 2026
Includes the fix for an out-of-bounds read in is_valid_utf8 on truncated multi-byte sequences (kgabis/parson#229). No API changes.
ewertons
added a commit
to Azure/azure-iot-sdk-c
that referenced
this pull request
Oct 26, 2026
Includes the fix for an out-of-bounds read in is_valid_utf8 on truncated multi-byte sequences (kgabis/parson#229). No API changes.
ewertons
added a commit
to Azure/azure-iot-sdk-c
that referenced
this pull request
Oct 27, 2026
….4 (#2781) * Update parson to 1.5.4 (ec53fb6) (#2779) Includes the fix for an out-of-bounds read in is_valid_utf8 on truncated multi-byte sequences (kgabis/parson#229). No API changes. (cherry picked from commit cecc78d) * IoTHub_Init initializes c-logging (backport of #2778)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While looking at the *_with_len string setters I noticed
verify_utf8_sequencedecides how many bytes a UTF-8 sequence needs purely from its lead byte, then unconditionally reads that many bytes to build the code point, without checking whether that many bytes are actually still inside the caller's buffer.If a string's last byte is a 2/3/4-byte lead byte whose continuation bytes have been cut off, the read goes past the given length. This is directly reachable from public API such as
json_value_init_string_with_lenandjson_object_set_string_with_len, whenever the buffer isn't followed by a null byte, which is exactly the case those_with_lenentry points are meant to support (binary-safe / length-prefixed strings).Repro under AddressSanitizer:
The fix threads the number of bytes remaining in the input down into
verify_utf8_sequenceand bails out before it looks past that, instead of only trustingnum_bytes_in_utf8_sequence()to know how far it's safe to look.Added cases to
test_suite_5in tests.c covering truncated 2/3/4-byte sequences sitting at the very end of an exact-size heap buffer (these crash under ASan without the fix), plus a full 4-byte sequence occupying the whole buffer to make sure valid input at the boundary still parses. Ran the existing suite under ASan+UBSan (all 356 checks pass) and also rebuilt with the project's own-std=c89 -pedantic-errorsand the C++ build, both clean.(Replaces #228, which had a stray commit trailer that shouldn't have been there — same fix and tests, clean history.)