Skip to content

Lift the instructions before a libVEX panic instead of failing the block - #609

Open
NopeCharon wants to merge 2 commits into
angr:masterfrom
NopeCharon:fix/partial-lift-on-vex-panic
Open

NopeCharon wants to merge 2 commits into
angr:masterfrom
NopeCharon:fix/partial-lift-on-vex-panic

Conversation

@NopeCharon

Copy link
Copy Markdown
Contributor

Fixes #425.

libVEX reports most undecodable instructions by ending the block before them with Ijk_NoDecode. On some inputs it panics instead: an internal assertion fails, or a decoder reaches an "impossible" case, like the nonexistent segment register in the issue. pyvex catches the panic, but vex_lift then returns NULL for the whole block. So lift() returned an empty Ijk_NoDecode block even when every instruction before the failing one decodes fine. With the issue's input (mov word ptr [rip], 0x9090 followed by 8c f0), master gives a 0-byte block.

When the lift fails, LibVEXLifter now looks for the longest run of instructions at the start of the block that libVEX can lift. This is the re-lift with an instruction limit suggested in the issue. libVEX only reports the instruction count once a lift succeeds, so pyvex can't read N from _lift_r after a panic. It finds N with a binary search instead: if libVEX panics on the n-th instruction, every lift of n or more instructions fails and every lift of fewer succeeds. Most panics happen on the first instruction. That is also the case when lift() resumes at the failing instruction. So a single max_inst=1 probe runs before the search. The block comes back with an Ijk_NoDecode jumpkind pointing at the failing instruction, which is what libVEX produces for instructions it doesn't recognize. lift() (including the fallback lifters) and callers like angr then handle it the same way. The libVEX error is logged at debug level.

Nothing changes for lifts that succeed. Blocks whose first instruction makes libVEX panic still raise LiftingException / give an empty Ijk_NoDecode block, as before. No libVEX change is needed. The extra lifts happen only on the failure path: with 98 instructions before the failing one, the whole lift() call takes about 0.3 ms.

The issue's input now gives a 9-byte block with 1 instruction, ending in NEXT: PUT(rip) = 0x9; Ijk_NoDecode. Other architectures work the same way. For example, the MIPS32 putIReg assertion from #76 now keeps the instructions before the ld.

Test: test_lift_instructions_before_vex_panic in tests/test_lift.py covers:

  • the issue's input
  • 1 to 11, 50 and 98 instructions before the failing one, with and without skip_stmts
  • max_inst and blocks that end before the failing instruction
  • a failing first instruction

It fails on master (assert 0 == 9). The full test suite passes with the change.

libVEX reports most undecodable instructions by ending the block before
them with Ijk_NoDecode, but on some inputs it panics instead (an internal
assertion fails, or a decoder hits an "impossible" case such as a nonexistent
segment register on AMD64). pyvex catches the panic, but the whole lift then
returns NULL, so lift() produced an empty Ijk_NoDecode block even when every
instruction before the failing one decodes fine. This breaks, for example,
lifting self-modifying code that contains such an instruction.

When the lift fails, LibVEXLifter now finds the longest run of instructions
at the start of the block that libVEX can lift. A panic on the n-th
instruction makes every lift of n or more instructions fail and every lift
of fewer succeed, so this is a binary search over the instruction limit,
after a single probe that rules out the common case of the very first
instruction failing (which is also what happens when lift() resumes at the
failing instruction). The block is returned with an Ijk_NoDecode jumpkind
pointing at the failing instruction, the same shape libVEX produces for
instructions it does not recognize, so lift() and its callers treat it the
same way. The libVEX error message is logged at debug level.

Lifts that succeed are unaffected, and blocks whose first instruction makes
libVEX panic still raise LiftingException as before. libVEX itself does not
need to change: the retries only happen on the failure path and take a
handful of extra lifts.

Fixes angr#425.
Inside the nested vex_lift() function pyright no longer sees the
TYPE_CHECKING narrowing of self.data to CLiftSource, so compute the data
pointer once outside it. Log the number of lifted instructions from the
IRSB instead of the cffi lift result, whose fields pyright cannot see.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot partially lift basic blocks containing invalid instructions

1 participant