Skip to content

fix: [client] Stop force-JSON-parsing non-JSON restSearch bodies - #24

Draft
elhoim wants to merge 1 commit into
MISP:mainfrom
elhoim:fix/nonjson-formats
Draft

elhoim wants to merge 1 commit into
MISP:mainfrom
elhoim:fix/nonjson-formats

Conversation

@elhoim

@elhoim elhoim commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

BLUF — Return non-JSON restSearch bodies as raw text instead of failing on a successful response.

  • Problem — check_response() in src/client.rs ran serde_json::from_str on every 2xx body, so search() with any of the nine non-JSON ReturnFormat values (Csv, Xml, Text, Stix, Suricata, Snort, Yara, Rpz, OpenIoc) failed with MispError::JsonError.
  • Fix — Split it into check_status plus check_response plus new check_response_raw/post_raw, added ReturnFormat::is_json() in src/search.rs, and routed those formats to a raw path returning Value::String(text).
  • Effect — Nine of eleven return formats now work, matching PyMISP, which decodes only json and yara-json.
  • CI — Red for repo-level reasons unrelated to this PR: the job ran on ubuntu-22.04 (no php8.3 mod_php) and test_event_crud_lifecycle raced the publish worker; both fixed in PR fix: [ci] Run integration tests on ubuntu-24.04 to match php8.3 #53.

Finding

check_response always JSON-parses 2xx bodies, breaking 9 of 11 ReturnFormats
Csv, Xml, Text, Stix, Suricata, Snort, Yara, Rpz, OpenIoc return non-JSON, so a successful response becomes MispError::JsonError. PyMISP returns raw text for everything except json/yara-json.

Change

check_response() in src/client.rs unconditionally ran serde_json::from_str on every 2xx body, so MispClient::search() with a non-JSON ReturnFormat (Csv, Xml, Text, Stix, Suricata, Snort, Yara, Rpz, OpenIoc) turned a successful response into MispError::JsonError. Split check_response into a shared check_status (status/error handling) plus check_response (JSON) and a new check_response_raw (raw text) / post_raw(), added ReturnFormat::is_json() in src/search.rs, and made search() route non-JSON formats through the raw path, returning Value::String(text) instead of parsing. client_blocking.rs just forwards to the async search(), so it needed no change.

Upstream evidence

pymisp/pymisp/api.py: search() at ~line 3082-3083 — elif return_format not in ['json', 'yara-json']: return self._check_response(response) — only json/yara-json get JSON-decoded, csv gets a similar raw-text branch at 3076-3081. _check_response at 4068-4108: tries loads(response.content) and, on failure, returns response.text (or response.text unconditionally for lenient_response_type non-JSON content-type) rather than raising. This confirms PyMISP returns raw text for non-json/yara-json return formats instead of forcing JSON parsing.

Verified against MISP core 843b6744 and PyMISP e7debb4f.

Tests

  • search_csv_returns_raw_text_not_json_error (client.rs) — mocks returnFormat=csv restSearch returning a raw CSV body; fails with MispError::JsonError before the fix, passes and asserts Value::String(csv_body) after
  • search_json_still_parses_json_body (client.rs) — pins that returnFormat=json still goes through the JSON path unchanged

Caveats

yara-json is a distinct PyMISP return_format value that decodes as JSON but is not modeled in the rustmisp ReturnFormat enum (only Yara exists, which is correctly treated as raw text) — that's a pre-existing enum-completeness gap, out of scope for this fix. client_blocking.rs's search() is a thin wrapper over the async client and required no change since it inherits the fix automatically. No cargo build/test/clippy was run per task constraints; cargo fmt was run and applied.


Verified locally: cargo fmt --check, cargo clippy --all-targets --all-features -- -D warnings, cargo test --all-features --lib, --test integration_tests --no-run, and cargo check --examples, each in an isolated target directory.

🤖 Generated with Claude Code

check_response() always ran serde_json::from_str on the 2xx body, so
search() with a non-json ReturnFormat (Csv, Xml, Text, Stix, Suricata,
Snort, Yara, Rpz, OpenIoc) turned a successful MISP response into
MispError::JsonError, since MISP returns those as plain text, not JSON.

PyMISP's search() only guarantees JSON decoding when return_format is
json or yara-json; every other format goes through _check_response,
which falls back to raw response.text() when the body doesn't parse as
JSON (pymisp/api.py search() ~line 3082 `elif return_format not in
['json', 'yara-json']: return self._check_response(response)`, and
_check_response ~line 4090-4108, whose except branch returns
response.text). stix2 is JSON in practice so it's treated as JSON here
too, matching PyMISP's effective behavior.

Split check_response into check_status (shared error handling) plus
check_response (JSON) and a new check_response_raw (raw text), added
post_raw(), and taught search() to route through the raw path for any
ReturnFormat where ReturnFormat::is_json() is false, returning
Value::String(text) instead of attempting to parse it.

Added search_csv_returns_raw_text_not_json_error, which fails against
the old check_response (JsonError on a CSV body) and passes with the
fix, plus search_json_still_parses_json_body to pin the unchanged json
path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JoSButbVaWEtGMXwTx73BZ
@elhoim
elhoim marked this pull request as draft September 3, 2026 05:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant