diff mbox series

tests: integrate fuzz-dwfl-core into elfutils

Message ID 20211212151658.60269-1-evvers@ya.ru
State Superseded
Headers show
Series tests: integrate fuzz-dwfl-core into elfutils | expand

Commit Message

Evgeny Vereshchagin Dec. 12, 2021, 3:16 p.m. UTC
The fuzz target was integrated into OSS-Fuzz in
https://github.com/google/oss-fuzz/pull/6944 and since then it
has been running there continously (uncovering various issues
along the way). It's all well and good but since OSS-Fuzz
is far from the elfutils repository it's unnecessarily hard
to build the fuzz target locally, verify patches and more generally
test new code to make sure that it doesn't introduce new issues (
or reintroduce regressions). This patch aims to address all those
issues by moving the fuzz target into the elfutils repository,
integrating it into the testsuite and also providing a script
that can be used to build full-fledged fuzzers utilizing libFuzzer.
With this patch applied `make check` can be used to make sure
that files kept in tests/fuzz-dwfl-core-corpus don't crash the
code on various architecures.
`--enable-sanitize-{address,undefined}` and/or `--enable-valgrind`
can additionally be used to uncover issues like
https://sourceware.org/bugzilla/show_bug.cgi?id=28685
that don't always manifest themselves in simple segfaults. On top
of all that now the fuzz target can be built and linked against
libFuzzer locally by just running `./tests/build-fuzzers.sh`.

The patch was tested in https://github.com/evverx/elfutils/pull/49:

* the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x
  and x86_64

* Fedora packages were built on those architectures;

* elfutils was built with both clang and gcc with and without sanitizers
  to make sure the tests pass there;

* `make distcheck` passed;

* coverage reports were built to make sure "static" builds are intact;

* the fuzz target was built and run with ClusterFuzzLite to make sure
  it's still compatible with OSS-Fuzz;

* the code was analyzed by various static analyzers to make sure new alerts
  aren't introduced.

Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
---
 tests/.gitignore                           |   1 +
 tests/ChangeLog                            |   5 ++
 tests/Makefile.am                          |  20 ++++-
 tests/build-fuzzers.sh                     |  95 +++++++++++++++++++++
 tests/fuzz-dwfl-core-corpus/empty          |   0
 tests/fuzz-dwfl-core-corpus/oss-fuzz-41566 | Bin 0 -> 1553 bytes
 tests/fuzz-dwfl-core-corpus/oss-fuzz-41570 | Bin 0 -> 1233 bytes
 tests/fuzz-dwfl-core.c                     |  50 +++++++++++
 tests/fuzz-main.c                          |  43 ++++++++++
 tests/fuzz.h                               |   9 ++
 tests/run-fuzz-dwfl-core.sh                |  11 +++
 11 files changed, 232 insertions(+), 2 deletions(-)
 create mode 100755 tests/build-fuzzers.sh
 create mode 100644 tests/fuzz-dwfl-core-corpus/empty
 create mode 100644 tests/fuzz-dwfl-core-corpus/oss-fuzz-41566
 create mode 100644 tests/fuzz-dwfl-core-corpus/oss-fuzz-41570
 create mode 100644 tests/fuzz-dwfl-core.c
 create mode 100644 tests/fuzz-main.c
 create mode 100644 tests/fuzz.h
 create mode 100755 tests/run-fuzz-dwfl-core.sh

Comments

Mark Wielaard Dec. 17, 2021, 10:46 a.m. UTC | #1
Hi Evgeny,

On Sun, Dec 12, 2021 at 03:16:58PM +0000, Evgeny Vereshchagin via Elfutils-devel wrote:
> The fuzz target was integrated into OSS-Fuzz in
> https://github.com/google/oss-fuzz/pull/6944 and since then it
> has been running there continously (uncovering various issues
> along the way). It's all well and good but since OSS-Fuzz
> is far from the elfutils repository it's unnecessarily hard
> to build the fuzz target locally, verify patches and more generally
> test new code to make sure that it doesn't introduce new issues (
> or reintroduce regressions). This patch aims to address all those
> issues by moving the fuzz target into the elfutils repository,
> integrating it into the testsuite and also providing a script
> that can be used to build full-fledged fuzzers utilizing libFuzzer.
> With this patch applied `make check` can be used to make sure
> that files kept in tests/fuzz-dwfl-core-corpus don't crash the
> code on various architecures.
> `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind`
> can additionally be used to uncover issues like
> https://sourceware.org/bugzilla/show_bug.cgi?id=28685
> that don't always manifest themselves in simple segfaults. On top
> of all that now the fuzz target can be built and linked against
> libFuzzer locally by just running `./tests/build-fuzzers.sh`.

I like the general idea of this. I have been using src/stack as fuzz
target locally, but that is not ideal since it does too much. Having
specific fuzz target binaries is much better. I also like the idea of
making those fuzz-targets into regular testsuite targets. I am still
trying to wrap my head around the LLVMFuzzerTestOneInput and libfuzzer
requirements. I have experience with afl and honggfuzz which don't
have any external library requirement.

Also the LLVMFuzzerTestOneInput seems backwards. Shouldn't there be a
more generic name for a function that is called by a fuzzer? Maybe it
seems upside down because you translate from data stream to byte array
and back and then read in the reconstructed stream again. Once you
have the bytes you can simply call Elf *elf_memory (char *__image,
size_t __size), There is no need to first write out the image to disk
and then use a file descriptor to read it back in.

One thing I struggle with is the initial seed (corpus). It needs to be
as small as possible, but also contain some valid ELF (core) files, so
that the fuzzer knows which valid paths there are to try out. How did
you construct the initial corpus? I normally try to create at least
four (little|big) endian and (32|64) bit minimal valid ELF files, but
that is not always easy.

Finally I wonder if we cannot integrate the logic in build-fuzzers.sh
in the normal auto* build and create a "make fuzz" target that simply
uses CC=afl-gcc or CC=hfuzz-gcc and runs afl-fuzz or honggfuzz for a
couple of minutes if installed/detected by configure. The
build-fuzzers.sh script seems very specific to a google setup which
most people won't have locally and which seems somewhat tricky to
replicate on other CI builders.

Cheers,

Mark
Evgeny Vereshchagin Dec. 17, 2021, 12:23 p.m. UTC | #2
Hi Mark,

>  Once you
> have the bytes you can simply call Elf *elf_memory (char *__image,
> size_t __size), There is no need to first write out the image to disk
> and then use a file descriptor to read it back in.

I think I should have mentioned in the commit message that the fuzz
target came from systemd where elfutils is hidden behind functions
receiving filenames and file descriptors and I wanted to cover that code.
If I had switched to elf_memory I couldn't have covered the code paths
used by systemd. I agree that in other fuzz targets I'm planning to add
elf_memory should be used instead but it would be great if it was
possible to keep this kind of systemd-specific target.

> One thing I struggle with is the initial seed (corpus). It needs to be
> as small as possible, but also contain some valid ELF (core) files, so
> that the fuzzer knows which valid paths there are to try out. How did
> you construct the initial corpus? I normally try to create at least
> four (little|big) endian and (32|64) bit minimal valid ELF files, but
> that is not always easy.

I think the name of that directory is a misnomer because it isn't a seed corpus.
I should have probably called it "fuzz-dwfl-core-crashes" or "fuzz-dwfl-core-regressions"
because I simply put files that have triggered various issues there.

I think initially I just added a valid core file used in the systemd testsuite and
let OSS-Fuzz deal with the rest. More generally, with OSS-Fuzz I
kind of use brute force to get corpora covering as much
code as possible with as few files as possible and with them it usually takes about
5-10 seconds to figure out whether new bugs are introduced or not.
Eventually those corpora can be downloaded, unpacked and used in test scripts.

elfutils hasn't been there long enough to get links to those corpora but for example one of systemd corpora
can be downloaded from
https://storage.googleapis.com/systemd-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/systemd_fuzz-varlink/public.zip

> Finally I wonder if we cannot integrate the logic in build-fuzzers.sh
> in the normal auto* build and create a "make fuzz" target that simply
> uses CC=afl-gcc or CC=hfuzz-gcc and runs afl-fuzz or honggfuzz for a
> couple of minutes if installed/detected by configure.

I think it's a good idea but I'm not sure how to make that all
compatible with OSS-Fuzz. I'll try to figure out how to do that.

> The
> build-fuzzers.sh script seems very specific to a google setup which
> most people won't have locally and which seems somewhat tricky to
> replicate on other CI builders.


I tried to decouple it from OSS-Fuzz as much as I could so in its current
form to build the fuzzer with LibFuzzer it should be enough to install
clang and run the script.
diff mbox series

Patch

diff --git a/tests/.gitignore b/tests/.gitignore
index 99d04819..c5429d0a 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -66,6 +66,7 @@ 
 /find-prologues
 /funcretval
 /funcscopes
+/fuzz-dwfl-core
 /get-aranges
 /get-files
 /get-lines
diff --git a/tests/ChangeLog b/tests/ChangeLog
index 82061c6e..511c12cf 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,8 @@ 
+2021-12-13  Evgeny Vereshchagin  <evvers@ya.ru>
+
+	* Makefile.am: Integrate fuzz-dwfl-core into the testsuite and add a
+	script linking it against libFuzzer.
+
 2021-12-09  Frank Ch. Eigler  <fche@redhat.com>
 
 	* debuginfod-subr.sh (xfail): New proc.
diff --git a/tests/Makefile.am b/tests/Makefile.am
index b2da2c83..6e1dd699 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -62,6 +62,7 @@  check_PROGRAMS = arextract arsymtest newfile saridx scnnames sectiondump \
 		  getphdrnum leb128 read_unaligned \
 		  msg_tst system-elf-libelf-test \
 		  nvidia_extended_linemap_libdw \
+		  fuzz-dwfl-core \
 		  $(asm_TESTS)
 
 asm_TESTS = asm-tst1 asm-tst2 asm-tst3 asm-tst4 asm-tst5 \
@@ -197,7 +198,8 @@  TESTS = run-arextract.sh run-arsymtest.sh run-ar.sh newfile test-nlist \
 	msg_tst system-elf-libelf-test \
 	$(asm_TESTS) run-disasm-bpf.sh run-low_high_pc-dw-form-indirect.sh \
 	run-nvidia-extended-linemap-libdw.sh run-nvidia-extended-linemap-readelf.sh \
-	run-readelf-dw-form-indirect.sh run-strip-largealign.sh
+	run-readelf-dw-form-indirect.sh run-strip-largealign.sh \
+	run-fuzz-dwfl-core.sh
 
 if !BIARCH
 export ELFUTILS_DISABLE_BIARCH = 1
@@ -580,7 +582,11 @@  EXTRA_DIST = run-arextract.sh run-arsymtest.sh run-ar.sh \
 	     run-readelf-dw-form-indirect.sh testfile-dw-form-indirect.bz2 \
 	     run-nvidia-extended-linemap-libdw.sh run-nvidia-extended-linemap-readelf.sh \
 	     testfile_nvidia_linemap.bz2 \
-	     testfile-largealign.o.bz2 run-strip-largealign.sh
+	     testfile-largealign.o.bz2 run-strip-largealign.sh \
+	     run-fuzz-dwfl-core.sh \
+	     fuzz-dwfl-core-corpus/empty \
+	     fuzz-dwfl-core-corpus/oss-fuzz-41566 \
+	     fuzz-dwfl-core-corpus/oss-fuzz-41570
 
 
 if USE_VALGRIND
@@ -755,6 +761,16 @@  leb128_LDADD = $(libelf) $(libdw)
 read_unaligned_LDADD = $(libelf) $(libdw)
 nvidia_extended_linemap_libdw_LDADD = $(libelf) $(libdw)
 
+# Fuzz targets are split into two files so that they can be
+# compatible with the test suite and OSS-Fuzz. OSS-Fuzz takes
+# files containing LLVMFuzzerTestOneInput and links them against
+# libFuzzer, AFL++ and honggfuzz. The testsuite links them against
+# fuzz-main.c (which is a local driver reading files into buffers
+# and passing those buffers to LLVMFuzzerTestOneInput).
+noinst_HEADERS=fuzz.h
+fuzz_dwfl_core_SOURCES = fuzz-main.c fuzz-dwfl-core.c
+fuzz_dwfl_core_LDADD = $(libelf) $(libdw)
+
 # We want to test the libelf header against the system elf.h header.
 # Don't include any -I CPPFLAGS. Except when we install our own elf.h.
 if !INSTALL_ELFH
diff --git a/tests/build-fuzzers.sh b/tests/build-fuzzers.sh
new file mode 100755
index 00000000..1fbab1f7
--- /dev/null
+++ b/tests/build-fuzzers.sh
@@ -0,0 +1,95 @@ 
+#!/bin/bash -eu
+
+# This script is supposed to be compatible with OSS-Fuzz, i.e. it has to use
+# environment variables like $CC, $CFLAGS and $OUT, link the fuzz targets with CXX
+# (even though the project is written in C) and so on:
+# https://google.github.io/oss-fuzz/getting-started/new-project-guide/#buildsh
+
+# The fuzz targets it builds can't make any assumptions about
+# their runtime environment apart from /tmp being writable:
+# https://google.github.io/oss-fuzz/further-reading/fuzzer-environment/ .
+# Even though it says there that it's possible to link fuzz targets against
+# their dependencies dynamically by moving them to $OUT and changing
+# rpath, it tends to break coverage reports from time to time https://github.com/google/oss-fuzz/issues/6524
+# so all the dependencies are linked statically here.
+
+# This script is configured via https://github.com/google/oss-fuzz/blob/master/projects/elfutils/project.yaml
+# and used to build the elfutils project on OSS-Fuzz with three fuzzing engines
+# (libFuzzer, AFL++ and honggfuzz) on two architectures (x86_64 and i386)
+# with three sanitizers (ASan, UBSan and MSan) with coverage reports on top of
+# all that: https://oss-fuzz.com/coverage-report/job/libfuzzer_asan_elfutils/latest
+# so before changing anything ideally it should be tested with the OSS-Fuzz toolchain
+# described at https://google.github.io/oss-fuzz/advanced-topics/reproducing/#building-using-docker
+# by running something like:
+#
+# ./infra/helper.py pull_images
+# ./infra/helper.py build_image --no-pull elfutils
+# for sanitizer in address undefined memory; do
+#   for engine in libfuzzer afl honggfuzz; do
+#     ./infra/helper.py build_fuzzers --clean --sanitizer=$sanitizer --engine=$engine elfutils PATH/TO/ELFUTILS
+#     ./infra/helper.py check_build --sanitizer=$sanitizer --engine=$engine -e ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 elfutils
+#   done
+# done
+#
+# ./infra/helper.py build_fuzzers --clean --architecture=i386 elfutils PATH/TO/ELFUTILS
+# ./infra/helper.py check_build --architecture=i386 -e ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 elfutils
+#
+# ./infra/helper.py build_fuzzers --clean --sanitizer=coverage elfutils PATH/TO/ELFUTILS
+# ./infra/helper.py coverage --no-corpus-download --fuzz-target=fuzz-dwfl-core --corpus-dir=PATH/TO/ELFUTILS/tests/fuzz-dwfl-core-corpus/ elfutils
+#
+# It should be possible to eventually automate that with ClusterFuzzLite https://google.github.io/clusterfuzzlite/
+# but it doesn't seem to be compatible with buildbot currently.
+
+# The script can also be used to build and run the fuzz target locally without Docker.
+# After installing clang and the build dependencies of libelf by running something
+# like `dnf build-dep elfutils-devel` on Fedora or `apt-get build-dep libelf-dev`
+# on Debian/Ubuntu, the following commands should be run:
+#
+#  $ ./tests/oss-fuzz.sh
+#  $ ./out/fuzz-dwfl-core tests/fuzz-dwfl-core-corpus/
+
+set -eux
+
+cd "$(dirname -- "$0")/.."
+
+SANITIZER=${SANITIZER:-address}
+flags="-O1 -fno-omit-frame-pointer -g -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=$SANITIZER -fsanitize=fuzzer-no-link"
+
+export CC=${CC:-clang}
+export CFLAGS=${CFLAGS:-$flags}
+
+export CXX=${CXX:-clang++}
+export CXXFLAGS=${CXXFLAGS:-$flags}
+
+export OUT=${OUT:-"$(pwd)/out"}
+mkdir -p "$OUT"
+
+export LIB_FUZZING_ENGINE=${LIB_FUZZING_ENGINE:--fsanitize=fuzzer}
+
+make clean || true
+
+# ASan isn't compatible with -Wl,--no-undefined: https://github.com/google/sanitizers/issues/380
+find -name Makefile.am | xargs sed -i 's/,--no-undefined//'
+
+# ASan isn't compatible with -Wl,-z,defs either:
+# https://clang.llvm.org/docs/AddressSanitizer.html#usage
+sed -i 's/^\(ZDEFS_LDFLAGS=\).*/\1/' configure.ac
+
+autoreconf -i -f
+if ! ./configure --enable-maintainer-mode --disable-debuginfod --disable-libdebuginfod \
+            --without-bzlib --without-lzma --without-zstd \
+	    CC="$CC" CFLAGS="-Wno-error $CFLAGS" CXX="-Wno-error $CXX" CXXFLAGS="$CXXFLAGS" LDFLAGS="$CFLAGS"; then
+    cat config.log
+    exit 1
+fi
+
+ASAN_OPTIONS=detect_leaks=0 make -j$(nproc) V=1
+
+$CC $CFLAGS \
+	-D_GNU_SOURCE -DHAVE_CONFIG_H \
+	-I. -I./lib -I./libelf -I./libebl -I./libdw -I./libdwelf -I./libdwfl -I./libasm \
+	-c tests/fuzz-dwfl-core.c -o fuzz-dwfl-core.o
+$CXX $CXXFLAGS $LIB_FUZZING_ENGINE fuzz-dwfl-core.o \
+	./libdw/libdw.a ./libelf/libelf.a -l:libz.a \
+	-o "$OUT/fuzz-dwfl-core"
+zip -r -j "$OUT/fuzz-dwfl-core_seed_corpus.zip" tests/fuzz-dwfl-core-corpus
diff --git a/tests/fuzz-dwfl-core-corpus/empty b/tests/fuzz-dwfl-core-corpus/empty
new file mode 100644
index 00000000..e69de29b
diff --git a/tests/fuzz-dwfl-core-corpus/oss-fuzz-41566 b/tests/fuzz-dwfl-core-corpus/oss-fuzz-41566
new file mode 100644
index 0000000000000000000000000000000000000000..a4181e6572fea9d568e787b0a4b57bf5defe4563
GIT binary patch
literal 1553
zcmb<-^>JfjWK@8F|4<r1IWRx~LIlYA|Np-L14M)cC=Fr>fEZ9VL<T}KGT{)1sfFv4
zUU)AQVivOgk-<g|cVzZp@;xxpL4YbwB4i4>JUX8$ju<rrO9()6D<mHh!)664a$sa&
zU<1<5{z0w`Xl5eE95NeC848yfYVm&sNSOdFA{Z>7;vhRg07Vl897w_@)^+$TB4i4>
TJctkTAH-MG^A~=X;*bRZ0NBqd

literal 0
HcmV?d00001

diff --git a/tests/fuzz-dwfl-core-corpus/oss-fuzz-41570 b/tests/fuzz-dwfl-core-corpus/oss-fuzz-41570
new file mode 100644
index 0000000000000000000000000000000000000000..4052572f5a484963b7bf03bb350368877671e7b1
GIT binary patch
literal 1233
zcmb<-^>JfjWK_Tf7#Sb{Ro;R@j6vZ)5TFXvi#KpK)60jW_Kb!A{ty6VOppp_{$x;q
U@IXR1zy$|x8Hg$z3WkL+07HkLIsgCw

literal 0
HcmV?d00001

diff --git a/tests/fuzz-dwfl-core.c b/tests/fuzz-dwfl-core.c
new file mode 100644
index 00000000..8fea6f51
--- /dev/null
+++ b/tests/fuzz-dwfl-core.c
@@ -0,0 +1,50 @@ 
+#include <assert.h>
+#include <config.h>
+#include <stdlib.h>
+#include ELFUTILS_HEADER(dwfl)
+#include "fuzz.h"
+#include "system.h"
+
+static const Dwfl_Callbacks core_callbacks =
+  {
+    .find_elf = dwfl_build_id_find_elf,
+    .find_debuginfo = dwfl_standard_find_debuginfo,
+  };
+
+int
+LLVMFuzzerTestOneInput (const uint8_t *data, size_t size)
+{
+  char name[] = "/tmp/fuzz-dwfl-core.XXXXXX";
+  int fd = -1;
+  off_t offset;
+  ssize_t n;
+  Elf *core = NULL;
+  Dwfl *dwfl = NULL;
+
+  fd = mkstemp (name);
+  assert (fd >= 0);
+
+  n = write_retry (fd, data, size);
+  assert (n >= 0);
+
+  offset = lseek (fd, 0, SEEK_SET);
+  assert (offset == 0);
+
+  elf_version (EV_CURRENT);
+  core = elf_begin (fd, ELF_C_READ_MMAP, NULL);
+  if (core == NULL)
+    goto cleanup;
+  dwfl = dwfl_begin (&core_callbacks);
+  assert(dwfl != NULL);
+  if (dwfl_core_file_report (dwfl, core, NULL) < 0)
+    goto cleanup;
+  if (dwfl_report_end (dwfl, NULL, NULL) != 0)
+    goto cleanup;
+
+cleanup:
+  dwfl_end (dwfl);
+  elf_end (core);
+  close (fd);
+  unlink (name);
+  return 0;
+}
diff --git a/tests/fuzz-main.c b/tests/fuzz-main.c
new file mode 100644
index 00000000..35573792
--- /dev/null
+++ b/tests/fuzz-main.c
@@ -0,0 +1,43 @@ 
+#include <assert.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include "fuzz.h"
+
+int
+main (int argc, char **argv)
+{
+  for (int i = 1; i < argc; i++)
+    {
+      fprintf (stderr, "Running: %s\n", argv[i]);
+
+      FILE *f = fopen (argv[i], "r");
+      assert (f);
+
+      int p = fseek (f, 0, SEEK_END);
+      assert (p >= 0);
+
+      long len = ftell (f);
+      assert (len >= 0);
+
+      p = fseek (f, 0, SEEK_SET);
+      assert (p >= 0);
+
+      void *buf = malloc (len);
+      assert (buf != NULL || len == 0);
+
+      size_t n_read = fread (buf, 1, len, f);
+      assert (n_read == (size_t) len);
+
+      (void) fclose (f);
+
+      int r = LLVMFuzzerTestOneInput (buf, len);
+
+      /* Non-zero return values are reserved by LibFuzzer for future use
+         https://llvm.org/docs/LibFuzzer.html#fuzz-target  */
+      assert (r == 0);
+
+      free (buf);
+
+      fprintf (stderr, "Done:    %s: (%zd bytes)\n", argv[i], n_read);
+    }
+}
diff --git a/tests/fuzz.h b/tests/fuzz.h
new file mode 100644
index 00000000..c8fe7a3a
--- /dev/null
+++ b/tests/fuzz.h
@@ -0,0 +1,9 @@ 
+#ifndef _FUZZ_H
+#define _FUZZ_H	1
+
+#include <stddef.h>
+#include <stdint.h>
+
+int LLVMFuzzerTestOneInput (const uint8_t *data, size_t size);
+
+#endif /* fuzz.h */
diff --git a/tests/run-fuzz-dwfl-core.sh b/tests/run-fuzz-dwfl-core.sh
new file mode 100755
index 00000000..d7c0cea5
--- /dev/null
+++ b/tests/run-fuzz-dwfl-core.sh
@@ -0,0 +1,11 @@ 
+#!/bin/sh
+
+. $srcdir/test-subr.sh
+
+exit_status=0
+for file in ${abs_srcdir}/fuzz-dwfl-core-corpus/*; do
+    testrun ${abs_builddir}/fuzz-dwfl-core $file ||
+        { echo "*** failure in $file"; exit_status=1; }
+done
+
+exit $exit_status