Repository navigation
Regression test harness - #129
Open
willmmiles wants to merge 24 commits into
Open
willmmiles wants to merge 24 commits into
willmmiles wants to merge 24 commits into
Conversation
A direct cast to uint32_t does not compile where pointers are 64 bits wide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Builds the library for the host against stand-ins for lwIP, FreeRTOS and the Arduino core, and runs Unity tests against it with `pio test -e native`. The lwIP mock plays the stack: it records every call, accounts for pcbs, pbufs and ports, and invokes the library's callbacks as lwIP would. The FreeRTOS mock runs the library's async task on the test's own thread, so every test is deterministic. Each test runs in a forked child. [env] becomes [target], so that the host environment does not inherit a board and a framework. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`pio test -e native-asan` runs the same suite under AddressSanitizer and UndefinedBehaviorSanitizer, with leak detection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unity runs only what RUN_TEST lists, so a test that is written but not listed, or a file whose run_*_tests() is never called from main, is silently dead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
willmmiles
force-pushed
the
harness-v6
branch
from
October 26, 2026 02:33
66d3cd1 to
3ae85ad
Compare
Member
you can commit the CI update : GitHub will run the new workflow from the branch with the current state and you will see the test failures (providing they are not ignored), and you can mark them ignored to sync with main state and we merge. This would add tests synced with main. then once merged you can open a new pr with the ignored removed and with the fixes to track the completion ? |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Needs no toolchain, so it reports in well under a minute. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mathieucarbou
approved these changes
Oct 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is a regression test harness built principally by @claude . It defines new
nativeandnative-asanenvironments for compiling for the build host, and a suite of LwIP mocks that can be pumped manually by test cases to simulate different event orderings.The test harness is configured within PlatformIO's framework; it can be run via
pio test -e native -e native-asan.Currently the test harness has 8 failing tests - 6 to do with out-of-memory handling, and 2 with life cycle corner cases that were discovered today. I have prepared a further commit that will add the test suite to the CI, but there's no point in adding it until the whole harness runs green. My preference is to track the failing tests as failing rather than disabling them or marking them as "expected to fail", but if you'd rather go the other way, it can be adjusted.