From: Dodji Seketeli <dodji@redhat.com>
If the 'big-tests' sub-directory (or a symbolic link to it) is present
then this patch configures it.
By default, the patch defines a new git sub-module into the
'big-tests' sub-directory. That git sub-module is the repository
git://sourceware.org/git/libabigail-tests. Users can now add the
--recurse-submodules option to git clone when they clone
git://sourceware.org/git/libabigail, so that it clones git big-tests
submodule as well.
The patch adds a new 'big-tests' target (to the top-most Makefile.am)
which runs "make check" in the big-tests sub-directory.
The patch also adds a new 'full-check' target that runs targets check,
check-self-compare and big-tests.
* Makefile.am: Add big-tests,clean-big-tests and full-check
targets.
* configure.ac: Add an --enable-big-tests option. For now, this
option is mandatory to handle the optional 'big-tests' support.
If the --enable-big-tests option is provided and if the
'big-tests' sub-directory is present, configure the big-tests/
sub-package.
* .gitmodules: New sub-module configuration to get
git://sourceware.org/git/libabigail-tests and stick into ...
* big-tests: ... this directory as a sub-module.
* CONTRIBUTING: Update this to explain how to get the Big Tests
going.
* tests/Makefile.am: Update the test summary for the
check-self-compare target.
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
---
.gitmodules | 3 +
CONTRIBUTING | 159 ++++++++++++++++++++++++++++------------------
Makefile.am | 9 ++-
big-tests | 1 +
configure.ac | 32 ++++++++++
tests/Makefile.am | 2 +-
6 files changed, 142 insertions(+), 64 deletions(-)
create mode 100644 .gitmodules
create mode 160000 big-tests
new file mode 100644
@@ -0,0 +1,3 @@
+[submodule "big-tests"]
+ path = big-tests
+ url = git://sourceware.org/git/libabigail-tests.git
@@ -125,54 +125,100 @@ the libabigail *library* directly (as opposed to forking libabigail
command line tools) will be verified.
How tests are organized
------------------------
-
-There are two kinds of regression tests. Those that use the
-libabigail *library* directly, and those that spawn one of the
-libabigail command line tools.
-
-Generally, both are usually made of a loop that churns through a set of input
-binaries to compare. Once the comparison is done, the resulting
-report is compared against a reference report that is provided.
-
-Test executables have names that starts with 'runtest*'. For instance,
-under <build-directory>/tests/ you can find tests named
-runtestdiffdwarf, runtestabidiff, etc...
-
-If a test executable is named
-<build-directory>/tests/runtestdiffdwarf, then its source code is
-tests/test-diff-dwarf.cc. Similarly, the source code of the test
-<build-directory>/tests/runtestabidiff is tests/test-abidiff.cc.
-
-The data provided for each test (for instance the input binaries to
-compare and the reference report that should result from the
-comparison) is to be found under tests/data. So data for the test
-runtestdiffdwarf is to be found under tests/data/test-diff-dwarf.
-Data for the test runtestabidiff is to be found under
-tests/data/test-abidiff.cc.
-
-So adding your own tests usually just amounts to adding the right input
-into the right sub-directory of tests/data/. To do so,
-look at several tests/test-*.cc to see which one you'd like to add
-some input binaries to be compared in.
-
-Then once you know which tests/test-*.cc you'd like to extend, and if
-you added your input binaries and reference reports (maybe other
-things too) to the right sub-director of tests/data/, you just need to
-extend the array of input binaries/reference reports that the test
-walks to perform the comparisons. It's generally a global variable
-before the main() function of the test. In test-diff-dwarf.cc, for
-instance, the variable name is "in_out_specs". You just have to add a
-new entry to that array; that new entry contains the paths to your new
-input binaries and reference reports. Just read the code in there and
-use your brains. It should be straight forward.
-
-Ah, also, if you added new files for the tests, then the build system
-needs to be told that those files have to be added to the distribution
-tarball when we do "make dist" (or make distcheck). To do so, please
-make sure to add your new test input files to the
-tests/data/Makefile.am file, in the EXTRA_DIST variable. Look at how
-things are organized in that file, and please do things similarly.
+=======================
+
+There are two kinds of tests in libabigail:
+
+ * Regression tests.
+
+ These are in the tests/ sub-directory of the package, hosted
+ in the libabigail Git repository on sourceware. They are
+ meant to run quickly and not be too big. They are also meant
+ to end up in the released libabigail tarball.
+
+ There are two kinds of regression tests. Those that use the
+ libabigail *library* directly, and those that spawn one of the
+ libabigail command line tools.
+
+ Generally, both are usually made of a loop that churns through a set of input
+ binaries to compare. Once the comparison is done, the resulting
+ report is compared against a reference report that is provided.
+
+ Test executables have names that starts with 'runtest*'. For instance,
+ under <build-directory>/tests/ you can find tests named
+ runtestdiffdwarf, runtestabidiff, etc...
+
+ If a test executable is named
+ <build-directory>/tests/runtestdiffdwarf, then its source code is
+ tests/test-diff-dwarf.cc. Similarly, the source code of the test
+ <build-directory>/tests/runtestabidiff is tests/test-abidiff.cc.
+
+ The data provided for each test (for instance the input binaries to
+ compare and the reference report that should result from the
+ comparison) is to be found under tests/data. So data for the test
+ runtestdiffdwarf is to be found under tests/data/test-diff-dwarf.
+ Data for the test runtestabidiff is to be found under
+ tests/data/test-abidiff.cc.
+
+ So adding your own tests usually just amounts to adding the right input
+ into the right sub-directory of tests/data/. To do so,
+ look at several tests/test-*.cc to see which one you'd like to add
+ some input binaries to be compared in.
+
+ Then once you know which tests/test-*.cc you'd like to extend, and if
+ you added your input binaries and reference reports (maybe other
+ things too) to the right sub-director of tests/data/, you just need to
+ extend the array of input binaries/reference reports that the test
+ walks to perform the comparisons. It's generally a global variable
+ before the main() function of the test. In test-diff-dwarf.cc, for
+ instance, the variable name is "in_out_specs". You just have to add a
+ new entry to that array; that new entry contains the paths to your new
+ input binaries and reference reports. Just read the code in there and
+ use your brains. It should be straight forward.
+
+ Ah, also, if you added new files for the tests, then the build system
+ needs to be told that those files have to be added to the distribution
+ tarball when we do "make dist" (or make distcheck). To do so, please
+ make sure to add your new test input files to the
+ tests/data/Makefile.am file, in the EXTRA_DIST variable. Look at how
+ things are organized in that file, and please do things similarly.
+
+ * Integration tests AKA Big Tests.
+
+ These are bigger and slower tests that would be too cumbersome
+ to be shipped in the libabigail tarball. They are in a
+ separate Git repository which is at
+ git://sourceware.org/git/libabigail-tests.
+
+ That repository is meant to be checked out as a sub-module in
+ a libabigail tree. We affectionately call these tests "Big
+ Tests".
+
+ Suppose you've checked out a libabigail tree by doing:
+
+ $ git clone git://sourceware.org/git/libabigail
+
+ To get the integration tests do this:
+
+ $ cd libabigail/big-tests
+ $ git submodule init
+ $ git submodule update
+
+ The Git repository at
+ git://sourceware.org/git/libabigail-tests is now checked-out
+ as a sub-module in the "big-tests" sub-directory of the
+ libabigail tree.
+
+ To configure libabigail with the support of the Big Tests, do
+ this:
+
+ $ cd <build-directory>
+ $ /path/to/libabigail/configure --enable-big-tests
+
+ To launch the Big Tests, do this in the <build-directory>:
+
+ $ make big-tests
+
fedabipkgdiff testing
---------------------
@@ -291,20 +337,9 @@ the commit log message for you.
Modifying the website
=====================
-The source code of the website of libabigail is stored in CVS (sigh,
-yeah, that is so old school). You can check out that web source code
-by doing:
-
- CVS_RSH=ssh cvs -z9 -d :ext:user@sourceware.org:/cvs/libabigail/ co htdocs
-
-where 'user' is your username on the sourceware system.
-Alternatively, you can check out the the web source code anonymously,
-if you don't have any user account on the sourceware system by doing:
-
- export CVSROOT=:pserver:anoncvs@cygwin.com:/cvs/libabigail
- cvs login
- (the CVS anonymous password to use is "anoncvs")
- cvs checkout htdocs
+The source code of the website of libabigail is stored in Git. You
+can check out that web source code by doing:
+ git co git://sourceware.org/git/libabigail-htdocs.git
Happy Hacking!
@@ -30,7 +30,7 @@ libabigail.pc.in
# Don't make make repeat.
AM_MAKEFLAGS = --no-print-directory
-.PHONY: doc
+.PHONY: doc big-tests
doc: html-doc man info
@@ -94,3 +94,10 @@ distcheck-fast:
check-self-compare:
$(MAKE) -C tests check-self-compare
+big-tests:
+ test -d big-tests && $(MAKE) -C big-tests check
+
+clean-big-tests:
+ test -d big-tests && $(MAKE) -C big-tests clean
+
+full-check: check check-self-compare big-tests
new file mode 160000
@@ -0,0 +1 @@
+Subproject commit 8447ddfd40ee4d3dbb7b3ba8e5fae809a89334a5
@@ -231,6 +231,14 @@ AC_ARG_ENABLE(btf,
[disable support of btf files)]),
ENABLE_BTF=$enableval,
ENABLE_BTF=auto)
+
+ENABLE_BIG_TESTS=no
+AC_ARG_ENABLE(big-tests,
+ AS_HELP_STRING([--enable-big-tests=yes|no],
+ [enable the support of libabigail big tests (in the big-tests subdir)]),
+ ENABLE_BIG_TESTS=$enableval,
+ ENABLE_BIG_TESTS=no)
+
dnl *************************************************
dnl check for dependencies
dnl *************************************************
@@ -457,6 +465,29 @@ dnl Test if various functions and structs are present.
fi
fi
+if test x$ENABLE_BIG_TESTS = xyes; then
+ AC_MSG_NOTICE([looking for $srcdir/big-tests ...]);
+ if test -d $srcdir/big-tests -o -L $srcdir/big-tests; then
+ AC_CONFIG_COMMANDS([create-big-tests-build-dir-and-configure],
+ [AC_MSG_NOTICE([Found $srcdir/big-tests!])
+ cd ac_abs_top_srcdir/big-tests
+ autoreconf -i
+ cd $ac_abs_top_builddir
+ test -d big-tests || mkdir big-tests
+ cd big-tests
+ CTF_OPTS=
+ BTF_OPTS=
+ if test x$CTF_ENABLED = xyes; then CTF_OPTS="--enable-ctf"; fi;
+ if test x$BTF_ENABLED = xyes; then BTF_OPTS="--enable-btf"; fi;
+ AC_MSG_NOTICE([configuring $srcdir/big-tests --with-libabigail-tools-dir=$ac_abs_top_builddir/tools --with-libabigail-headers-dir=$ac_abs_top_srcdir/include $CTF_OPTS $BTF_OPTS])
+ $ac_abs_top_srcdir/big-tests/configure --with-libabigail-tools-dir=$ac_abs_top_builddir/tools --with-libabigail-headers-dir=$ac_abs_top_srcdir/include $CTF_OPTS $BTF_OPTS],
+ [CTF_ENABLED=$ENABLE_CTF
+ BTF_ENABLED=$ENABLE_BTF])
+ else
+ AC_MSG_ERROR([No sub-directory big-tests found. Please checkout libabigail-big-tests under a sub-directory named 'big-tests'])
+ fi
+fi
+
dnl Check for dependency: libxml
LIBXML2_VERSION=2.6.22
PKG_CHECK_MODULES(XML, libxml-2.0 >= $LIBXML2_VERSION)
@@ -1267,6 +1298,7 @@ AC_MSG_NOTICE([
Enable python 3 : ${ENABLE_PYTHON3}
Enable CTF front-end : ${ENABLE_CTF}
Enable BTF front-end : ${ENABLE_BTF}
+ Enable Big Libabigail tests : ${ENABLE_BIG_TESTS}
Enable running tests under Valgrind : ${enable_valgrind}
Enable build with -fsanitize=address : ${ENABLE_ASAN}
Enable build with -fsanitize=memory : ${ENABLE_MSAN}
@@ -274,5 +274,5 @@ check-valgrind-helgrind-recursive:
$(MAKE) check-valgrind-helgrind VALGRIND_FLAGS="${RECURSIVE_VALGRIND_FLAGS}"
check-self-compare:
- $(MAKE) check TESTS=runtestslowselfcompare.sh ENABLE_SLOW_TEST=yes
+ $(MAKE) check TESTS=runtestslowselfcompare.sh ENABLE_SLOW_TEST=yes AM_TESTSUITE_SUMMARY_HEADER=' for runtestslowselfcompare.sh'