Mach-O: Register the LC_FUNCTION_STARTS entries as function hints

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

The Mach-O backend parses LC_FUNCTION_STARTS into lc_function_starts and then uses it for nothing, so a Mach-O object hands its consumer no function hints at all while an ELF one gets a hint for every FDE in .eh_frame. On a stripped image that table is the only record of where the functions begin.

Each entry now becomes a FunctionHint under a new FunctionHintSource.FUNCTION_STARTS. It is a source of its own rather than an export or an unwind record because ld64 records the address of every atom it placed in an executable section, and a producer's data atoms are atoms like any other, so a consumer has to be able to tell this table apart from one that names only functions. No size is recorded: the distance to the next entry is not a function length.

The test checks the hints on tests/aarch64/dyld_ios15.macho against the parsed table and against the bounds of __TEXT,__text.

Land this before angr/angr#6861, which decides which of these entries to use. Until that one lands angr takes every hint that is not an FDE as a function start, which on a Haskell image puts function heads on 129 info tables; the reverse order does not work at all, because angr#6861 reads a hint source this pull request adds.

Fixes #750. Validation: #754 (comment)

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head b987439b3f3fdd10e54790958e8b58d1895fc5fc against baseline 45c6509c753d07f740099035cd41f7f473dc6f31.

• Regression: pytest tests/test_macho.py -k function_starts — fails on baseline (FunctionHintSource has no FUNCTION_STARTS, and [h.addr for h in hints] is [] against the 36 entries the file records), passes on head
• Focused: pytest tests — 226 passed, 9 skipped
• Lint/type: pylint and pyright, merge-base relative, over the three changed files — no regressions (backend.py 9.97 -> 9.97, macho/macho.py 10.00 -> 10.00, tests/test_macho.py 9.93 -> 9.94; pyright badness unchanged or lower on all three)
• Pre-commit: all hooks over the changed files — pass

What the entries are, and why this is a hint source of its own rather than a symbol table. Measured on a GHC-produced x86-64 MH_DYLIB, sha256 1a0b6c5ea58c30af..., 8,968 recorded entries, scored with CFGFast(normalize=True, data_references=False, resolve_indirect_jumps=True), each side in its own process:

entries that are function starts	stock	this branch, entries fed in unfiltered
GHC closure entry code (*_info)		4,255 newly
GHC info table (_dsp_*_dsp, a data atom)		129 newly
blocks	17,024	23,777
functions	4,147	8,201
seconds	285	583

The 129 are the reason for the ordering note in the description: the table names data atoms as well as functions, and the labels above come from the producer's own naming rather than from anything angr or this branch computes.

Caveats: the address is recorded as the linked virtual address and rebased with the object, matching the ELF FDE hints; a 32-bit ARM Mach-O records the Thumb bit in the entry, which this branch passes through untouched.

Merge order, measured on the angr side against 503b1be0663e65de225194eeb8e2cee6fa20d8d3, one process per object. Landing this pull request before angr/angr#6861 leaves angr taking every hint that is not an FDE as a function start: CFGFast then recovers 21 functions instead of 19 on x86_64/fauxware.macho and 28 instead of 27 on multi_arch/fauxware_macho_multiarch, with no change on aarch64/dyld_ios15.macho, stable over three repetitions per side. Both of angr's Mach-O test files still pass in that pairing (pytest tests/analyses/cfg/test_cfgfast.py tests/analyses/cfg/test_macho_resolver.py — 48 passed, 2 skipped). The reverse order does not work at all: angr/angr#6861 reads FunctionHintSource.FUNCTION_STARTS, so without this pull request loading a Mach-O image raises AttributeError. So this one lands first.

[angr-bot]

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_754