foster/bitcoin-bitcoin-core
0
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-02-09 10:43:19 -05:00
Code Issues Activity

doc: Improve fuzzing docs for macOS users

Browse source
This commit is contained in:
Fabian Jahr 2020-01-13 13:16:10 +01:00
parent e7f8450357
commit b6c3e84e87
No known key found for this signature in database
GPG key ID: F13D1E9D890798CD
1 changed files with 58 additions and 17 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

75
doc/fuzzing.md
View file

@ -7,11 +7,8 @@ describe how to use it with AFL and libFuzzer.
## Preparing fuzzing ## Preparing fuzzing
AFL needs an input directory with examples, and an output directory where it The fuzzer needs some inputs to work on, but the inputs or seeds can be used
will place examples that it found. These can be anywhere in the file system, interchangeably between libFuzzer and AFL.
we'll define environment variables to make it easy to reference them.
libFuzzer will use the input directory as output directory.
Extract the example seeds (or other starting inputs) into the inputs Extract the example seeds (or other starting inputs) into the inputs
directory before starting fuzzing. directory before starting fuzzing.
@ -21,13 +18,19 @@ git clone https://github.com/bitcoin-core/qa-assets
export DIR_FUZZ_IN=$PWD/qa-assets/fuzz_seed_corpus export DIR_FUZZ_IN=$PWD/qa-assets/fuzz_seed_corpus
``` ```
Only for AFL: AFL needs an input directory with examples, and an output directory where it
will place examples that it found. These can be anywhere in the file system,
we'll define environment variables to make it easy to reference them.
So, only for AFL you need to configure the outputs path:
``` ```
mkdir outputs mkdir outputs
export AFLOUT=$PWD/outputs export AFLOUT=$PWD/outputs
``` ```
libFuzzer will use the input directory as output directory.
## AFL ## AFL
### Building AFL ### Building AFL
@ -41,6 +44,9 @@ make
export AFLPATH=$PWD export AFLPATH=$PWD
``` ```
For macOS you may need to ignore x86 compilation checks when running `make`:
`AFL_NO_X86=1 make`.
### Instrumentation ### Instrumentation
To build Bitcoin Core using AFL instrumentation (this assumes that the To build Bitcoin Core using AFL instrumentation (this assumes that the
@ -48,9 +54,15 @@ To build Bitcoin Core using AFL instrumentation (this assumes that the
``` ```
./configure --disable-ccache --disable-shared --enable-tests --enable-fuzz CC=${AFLPATH}/afl-gcc CXX=${AFLPATH}/afl-g++ ./configure --disable-ccache --disable-shared --enable-tests --enable-fuzz CC=${AFLPATH}/afl-gcc CXX=${AFLPATH}/afl-g++
export AFL_HARDEN=1 export AFL_HARDEN=1
cd src/
make make
``` ```
If you are using clang you will need to substitute `afl-gcc` with `afl-clang`
and `afl-g++` with `afl-clang++`, so the first line above becomes:
```
./configure --disable-ccache --disable-shared --enable-tests --enable-fuzz CC=${AFLPATH}/afl-clang CXX=${AFLPATH}/afl-clang++
```
We disable ccache because we don't want to pollute the ccache with instrumented We disable ccache because we don't want to pollute the ccache with instrumented
objects, and similarly don't want to use non-instrumented cached objects linked objects, and similarly don't want to use non-instrumented cached objects linked
in. in.
@ -60,25 +72,32 @@ The fuzzing can be sped up significantly (~200x) by using `afl-clang-fast` and
compiling using `afl-clang-fast`/`afl-clang-fast++` the resulting compiling using `afl-clang-fast`/`afl-clang-fast++` the resulting
binary will be instrumented in such a way that the AFL binary will be instrumented in such a way that the AFL
features "persistent mode" and "deferred forkserver" can be used. See features "persistent mode" and "deferred forkserver" can be used. See
https://github.com/mcarpenter/afl/tree/master/llvm_mode for details. https://github.com/google/AFL/tree/master/llvm_mode for details.
### Fuzzing ### Fuzzing
To start the actual fuzzing use: To start the actual fuzzing use:
``` ```
export FUZZ_TARGET=fuzz_target_foo # Pick a fuzz_target export FUZZ_TARGET=bech32 # Pick a fuzz_target
mkdir ${AFLOUT}/${FUZZ_TARGET} mkdir ${AFLOUT}/${FUZZ_TARGET}
$AFLPATH/afl-fuzz -i ${DIR_FUZZ_IN}/${FUZZ_TARGET} -o ${AFLOUT}/${FUZZ_TARGET} -m52 -- test/fuzz/${FUZZ_TARGET} $AFLPATH/afl-fuzz -i ${DIR_FUZZ_IN}/${FUZZ_TARGET} -o ${AFLOUT}/${FUZZ_TARGET} -m52 -- src/test/fuzz/${FUZZ_TARGET}
``` ```
You may have to change a few kernel parameters to test optimally - `afl-fuzz` You may have to change a few kernel parameters to test optimally - `afl-fuzz`
will print an error and suggestion if so. will print an error and suggestion if so.
On macOS you may need to set `AFL_NO_FORKSRV=1` to get the target to run.
```
export FUZZ_TARGET=bech32 # Pick a fuzz_target
mkdir ${AFLOUT}/${FUZZ_TARGET}
AFL_NO_FORKSRV=1 $AFLPATH/afl-fuzz -i ${DIR_FUZZ_IN}/${FUZZ_TARGET} -o ${AFLOUT}/${FUZZ_TARGET} -m52 -- src/test/fuzz/${FUZZ_TARGET}
```
## libFuzzer ## libFuzzer
A recent version of `clang`, the address/undefined sanitizers (ASan/UBSan) and libFuzzer is needed (all A recent version of `clang`, the address/undefined sanitizers (ASan/UBSan) and
found in the `compiler-rt` runtime libraries package). libFuzzer is needed (all found in the `compiler-rt` runtime libraries package).
To build all fuzz targets with libFuzzer, run To build all fuzz targets with libFuzzer, run
@ -87,11 +106,33 @@ To build all fuzz targets with libFuzzer, run
make make
``` ```
The fuzzer needs some inputs to work on, but the inputs or seeds can be used
interchangeably between libFuzzer and AFL.
See https://llvm.org/docs/LibFuzzer.html#running on how to run the libFuzzer See https://llvm.org/docs/LibFuzzer.html#running on how to run the libFuzzer
instrumented executable. instrumented executable.
Alternatively run the script in `./test/fuzz/test_runner.py` and provide it Alternatively, you can run the script through the fuzzing test harness (only
with the `${DIR_FUZZ_IN}` created earlier. libFuzzer supported so far). You need to pass it the inputs directory and
the specific test target you want to run.
```
./test/fuzz/test_runner.py ${DIR_FUZZ_IN} bech32
```
### macOS hints for libFuzzer
The default clang/llvm version supplied by Apple on macOS does not include
fuzzing libraries, so macOS users will need to install a full version, for
example using `brew install llvm`.
Should you run into problems with the address sanitizer, it is possible you
may need to run `./configure` with `--disable-asm` to avoid errors
with certain assembly code from Bitcoin Core's code. See [developer notes on sanitizers](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#sanitizers)
for more information.
You may also need to take care of giving the correct path for clang and
clang++, like `CC=/path/to/clang CXX=/path/to/clang++` if the non-systems
clang does not come first in your path.
Full configure that was tested on macOS Catalina with `brew` installed `llvm`:
```
./configure --disable-ccache --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ --disable-asm
```