Skip to content

Fix data loss when mixing line and block comments - #233

Open
jeff-grimes-pplx wants to merge 1 commit into
kgabis:masterfrom
jeff-grimes-pplx:fix/mixed-comment-styles
Open

jeff-grimes-pplx wants to merge 1 commit into
kgabis:masterfrom
jeff-grimes-pplx:fix/mixed-comment-styles

Conversation

@jeff-grimes-pplx

Copy link
Copy Markdown

Problem

json_parse_string_with_comments() (and therefore json_parse_file_with_comments()) silently drops data or fails to parse valid commented JSON when a line comment contains /* or a lone ".

const char *s =
    "{\n"
    "  \"a\": 1, // matches src/*.c\n"
    "  \"b\": 2, /* required */\n"
    "  \"c\": 3\n"
    "}";
JSON_Value *v = json_parse_string_with_comments(s);
/* expected {"a":1,"b":2,"c":3}, actual {"a":1,"c":3} -- "b" is silently lost */

A line comment with an unmatched quote (// 5" screen) followed by a block comment makes the whole document fail to parse (NULL).

Root Cause

Comments are stripped by two independent passes:

remove_comments(string_mutable_copy, "/*", "*/");
remove_comments(string_mutable_copy, "//", "\n");

Each pass only knows about string literals and its own comment style. During the /* */ pass, the /* inside // matches src/*.c is treated as the start of a block comment, which then runs until the */ of the next comment, blanking out the "b": 2, member in between. Likewise, a " inside a line comment flips the block pass's in_string state, so the following /* px */ is never removed and the parser sees garbage.

Fix

Replace the two passes with a single left-to-right pass that tracks string literals (including escapes), /* ... */ comments and // ... comments at the same time, so whichever construct opens first wins — exactly as a C/JS lexer would. Behaviour for every input that previously worked is unchanged: comment bodies are still overwritten with spaces, the newline that ends a line comment is preserved, and an unterminated block comment still blanks to the end of the input.

Minimal Reproducer

Compiling and running the snippet above under ASan/UBSan:

gcc -std=c89 -Wall -Wextra -fsanitize=address,undefined -g -I. repro.c parson.c -o repro && ./repro

Before: [FAIL] line comment containing '/*' must not swallow data (actual {"a":1,"c":3}) and [FAIL] line comment containing a lone '"' must not hide a block comment (actual NULL). After: all cases pass.

Verification

  • New test_suite_2_mixed_comments() in tests.c (4 cases) — fails on master (Tests failed: 4), passes with the fix.
  • make test, make testcpp, make test_hash_collisions: Tests failed: 0, Tests passed: 369 each.

json_parse_string_with_comments() stripped comments in two independent
passes: first every "/* ... */", then every "// ...". Each pass only
tracked string literals, not the other comment style, so text inside a
comment of one style was interpreted as code by the other pass:

  * A line comment containing "/*" (e.g. "// matches src/*.c") started a
    block comment in the first pass that ran up to the next "*/" later in
    the document, silently deleting the JSON in between, e.g.
    {"a":1, // src/*.c\n "b":2, /* x */\n "c":3} parsed as {"a":1,"c":3}.
  * A lone double quote in a line comment (e.g. "// 5\" screen") flipped
    the first pass's in-string state, so later block comments were left
    in place and valid input failed to parse.

Strip both comment styles in a single left-to-right pass so that whichever
construct (string, block comment or line comment) starts first owns the
text that follows it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant