[v2,3/8] Add self-test framework to gdb
Commit Message
I wanted to unit test the Rust lexer, so I added a simple unit testing
command to gdb.
The intent is that self tests will only be compiled into gdb in
development mode. In release mode they simply won't exist. So, this
exposes $development to C code as GDB_SELF_TEST.
In development mode, test functions are registered with the self test
module. A test function is just a function that does some checks, and
aborts on failure. I chose this rather than something fancier because
I think any such failure will require debugging anyhow.
Then this adds a new "maint selftest" command which invokes the test
functions, and a new dejagnu test case that invokes it.
2016-04-27 Tom Tromey <tom@tromey.com>
* NEWS: Add "maint selftest" entry.
* selftest.h: New file.
* selftest.c: New file.
* maint.c: Include selftest.h.
(maintenance_selftest): New function.
(_initialize_maint_cmds): Add "maint selftest" command.
* configure.ac (GDB_SELF_TEST): Maybe define.
* config.in, configure: Rebuild.
* Makefile.in (SFILES): Add selftest.c.
(COMMON_OBS): Add selftest.o.
2016-04-26 Tom Tromey <tom@tromey.com>
* gdb.texinfo (Maintenance Commands): Document "maint selftest".
2016-04-26 Tom Tromey <tom@tromey.com>
* gdb.gdb/unittest.exp: New file.
---
gdb/ChangeLog | 13 ++++++++
gdb/Makefile.in | 6 ++--
gdb/NEWS | 3 ++
gdb/config.in | 3 ++
gdb/configure | 6 ++++
gdb/configure.ac | 5 +++
gdb/doc/ChangeLog | 4 +++
gdb/doc/gdb.texinfo | 6 ++++
gdb/maint.c | 18 +++++++++++
gdb/selftest.c | 66 ++++++++++++++++++++++++++++++++++++++
gdb/selftest.h | 35 ++++++++++++++++++++
gdb/testsuite/ChangeLog | 4 +++
gdb/testsuite/gdb.gdb/unittest.exp | 17 ++++++++++
13 files changed, 184 insertions(+), 2 deletions(-)
create mode 100644 gdb/selftest.c
create mode 100644 gdb/selftest.h
create mode 100644 gdb/testsuite/gdb.gdb/unittest.exp
Comments
> From: Tom Tromey <tom@tromey.com>
> Cc: Tom Tromey <tom@tromey.com>
> Date: Wed, 27 Apr 2016 14:34:34 -0600
>
> I wanted to unit test the Rust lexer, so I added a simple unit testing
> command to gdb.
>
> The intent is that self tests will only be compiled into gdb in
> development mode. In release mode they simply won't exist. So, this
> exposes $development to C code as GDB_SELF_TEST.
>
> In development mode, test functions are registered with the self test
> module. A test function is just a function that does some checks, and
> aborts on failure. I chose this rather than something fancier because
> I think any such failure will require debugging anyhow.
>
> Then this adds a new "maint selftest" command which invokes the test
> functions, and a new dejagnu test case that invokes it.
>
> 2016-04-27 Tom Tromey <tom@tromey.com>
>
> * NEWS: Add "maint selftest" entry.
> * selftest.h: New file.
> * selftest.c: New file.
> * maint.c: Include selftest.h.
> (maintenance_selftest): New function.
> (_initialize_maint_cmds): Add "maint selftest" command.
> * configure.ac (GDB_SELF_TEST): Maybe define.
> * config.in, configure: Rebuild.
> * Makefile.in (SFILES): Add selftest.c.
> (COMMON_OBS): Add selftest.o.
>
> 2016-04-26 Tom Tromey <tom@tromey.com>
>
> * gdb.texinfo (Maintenance Commands): Document "maint selftest".
>
> 2016-04-26 Tom Tromey <tom@tromey.com>
>
> * gdb.gdb/unittest.exp: New file.
OK for the documentation parts.
Thanks.
Tom Tromey <tom@tromey.com> writes:
Hi Tom,
this framework is sufficient in current needs, but I'd like to think
more about it because it is a framework, and more and more tests may use
it in the future.
> In development mode, test functions are registered with the self test
> module. A test function is just a function that does some checks, and
> aborts on failure. I chose this rather than something fancier because
> I think any such failure will require debugging anyhow.
In this framework, all unit tests are running in one GDB instance, so:
- if one test fails, the following tests won't be run, even they are
from different modules,
- tests may change the GDB state, and the following tests may be affected,
I'd like the framework to only run tests from one module in one gdb
instance, restart gdb after they are finished, and continue running
tests from other modules. For example, we have registered three self
tests in different modules,
register_self_test (rust_lex_tests);
register_self_test (breakpoint_tests);
register_self_test (linespec_tests);
In gdb.gdb/unittest.exp, we can get the list of self tests from
different modules via command "maint info selftest", like
$ (gdb) maint info selftest
rust_lex_tests
breakpoint_tests
linespec_tests
and the output can be saved in $list_of_tests, start a fresh gdb every
time, and execute these tests one by one, like
foreach test $list_of_tests {
gdb_start
gdb_test "maint selftest $test" "Ran $decimal unit tests"
gdb_exit
}
Yao> I'd like the framework to only run tests from one module in one gdb
Yao> instance, restart gdb after they are finished, and continue running
Yao> tests from other modules. For example, we have registered three self
Yao> tests in different modules,
Sounds reasonable.
It would also be possible to use exceptions rather than asserts.
Then the current single command could be kept, since gdb wouldn't abort
on failure.
Let me know which you'd prefer.
Tom
Tom Tromey <tom@tromey.com> writes:
> It would also be possible to use exceptions rather than asserts.
> Then the current single command could be kept, since gdb wouldn't abort
> on failure.
>
> Let me know which you'd prefer.
Using exceptions is fine to me.
On Fri, Apr 29, 2016 at 8:29 AM, Yao Qi <qiyaoltc@gmail.com> wrote:
> Tom Tromey <tom@tromey.com> writes:
>
>> It would also be possible to use exceptions rather than asserts.
>> Then the current single command could be kept, since gdb wouldn't abort
>> on failure.
>>
>> Let me know which you'd prefer.
>
> Using exceptions is fine to me.
On a separate note, where do we, in the long term, want tests to live?
I'm wondering about the scalability of having tests in the same source file
reducing the readability, and whether we want conventions to avoid this.
I'm not saying 100% of tests have to, say, live in separate files
(e.g. foo.c tests -> foo-test.c or some such).
I'm just paranoid about having a repeat of the massive readability reduction
from the adding of record support to *-tdep.c files.
[Imagine how the sources will look 5 years from now.]
@@ -1,3 +1,16 @@
+2016-04-27 Tom Tromey <tom@tromey.com>
+
+ * NEWS: Add "maint selftest" entry.
+ * selftest.h: New file.
+ * selftest.c: New file.
+ * maint.c: Include selftest.h.
+ (maintenance_selftest): New function.
+ (_initialize_maint_cmds): Add "maint selftest" command.
+ * configure.ac (GDB_SELF_TEST): Maybe define.
+ * config.in, configure: Rebuild.
+ * Makefile.in (SFILES): Add selftest.c.
+ (COMMON_OBS): Add selftest.o.
+
2016-04-26 Tom Tromey <tom@tromey.com>
* expprint.c: Include f-lang.h.
@@ -869,8 +869,10 @@ SFILES = ada-exp.y ada-lang.c ada-typeprint.c ada-valprint.c ada-tasks.c \
proc-service.list progspace.c \
prologue-value.c psymtab.c \
regcache.c reggroups.c remote.c remote-fileio.c remote-notif.c reverse.c \
- sentinel-frame.c \
+ selftest.c sentinel-frame.c \
serial.c ser-base.c ser-unix.c ser-event.c skip.c \
+ selftest.c sentinel-frame.c \
+ serial.c ser-base.c ser-unix.c skip.c \
solib.c solib-target.c source.c \
stabsread.c stack.c probe.c stap-probe.c std-regs.c \
symfile.c symfile-debug.c symfile-mem.c symmisc.c symtab.c \
@@ -1060,7 +1062,7 @@ COMMON_OBS = $(DEPFILES) $(CONFIG_OBS) $(YYOBJ) \
go-lang.o go-valprint.o go-typeprint.o \
jv-lang.o jv-valprint.o jv-typeprint.o jv-varobj.o \
m2-lang.o opencl-lang.o p-lang.o p-typeprint.o p-valprint.o \
- sentinel-frame.o \
+ selftest.o sentinel-frame.o \
complaints.o typeprint.o \
ada-typeprint.o c-typeprint.o f-typeprint.o m2-typeprint.o \
ada-valprint.o c-valprint.o cp-valprint.o d-valprint.o f-valprint.o \
@@ -40,6 +40,9 @@ skip -rfunction regular-expression
maint info line-table REGEXP
Display the contents of GDB's internal line table data struture.
+maint selftest
+ Run any GDB unit tests that were compiled in.
+
* Support for tracepoints and fast tracepoints on s390-linux and s390x-linux
was added in GDBserver, including JIT compiling fast tracepoint's
conditional expression bytecode into native code.
@@ -65,6 +65,9 @@
/* Define to the default OS ABI for this configuration. */
#undef GDB_OSABI_DEFAULT
+/* Define if self-testing features should be enabled */
+#undef GDB_SELF_TEST
+
/* Define to 1 if you have `alloca', as a function or macro. */
#undef HAVE_ALLOCA
@@ -16482,6 +16482,12 @@ ac_config_links="$ac_config_links $ac_config_links_1"
$as_echo "#define GDB_DEFAULT_HOST_CHARSET \"UTF-8\"" >>confdefs.h
+if $development; then
+
+$as_echo "#define GDB_SELF_TEST 1" >>confdefs.h
+
+fi
+
gdb_ac_transform=`echo "$program_transform_name" | sed -e 's/\\$\\$/\\$/g'`
GDB_TRANSFORM_NAME=`echo gdb | sed -e "$gdb_ac_transform"`
@@ -2330,6 +2330,11 @@ dnl At the moment, we just assume it's UTF-8.
AC_DEFINE(GDB_DEFAULT_HOST_CHARSET, "UTF-8",
[Define to be a string naming the default host character set.])
+if $development; then
+ AC_DEFINE(GDB_SELF_TEST, 1,
+ [Define if self-testing features should be enabled])
+fi
+
GDB_AC_TRANSFORM([gdb], [GDB_TRANSFORM_NAME])
GDB_AC_TRANSFORM([gcore], [GCORE_TRANSFORM_NAME])
AC_CONFIG_FILES([gcore], [chmod +x gcore])
@@ -1,3 +1,7 @@
+2016-04-26 Tom Tromey <tom@tromey.com>
+
+ * gdb.texinfo (Maintenance Commands): Document "maint selftest".
+
2016-04-13 Antoine Tremblay <antoine.tremblay@ericsson.com>
* agentexpr.texi (byte): Fix zero_ext description.
@@ -34450,6 +34450,12 @@ that symbol is described. The type chain produced by this command is
a recursive definition of the data type as stored in @value{GDBN}'s
data structures, including its flags and contained types.
+@kindex maint selftest
+@cindex self tests
+Run any self tests that were compiled in to @value{GDBN}. If all the
+self tests pass, this will print a message. If a self test fails,
+@value{GDBN} will crash.
+
@kindex maint set dwarf always-disassemble
@kindex maint show dwarf always-disassemble
@item maint set dwarf always-disassemble
@@ -41,6 +41,7 @@
#include "top.h"
#include "timeval-utils.h"
#include "maint.h"
+#include "selftest.h"
#include "cli/cli-decode.h"
#include "cli/cli-utils.h"
@@ -982,6 +983,16 @@ show_per_command_cmd (char *args, int from_tty)
cmd_show_list (per_command_showlist, from_tty, "");
}
+
+/* The "maintenance selftest" command. */
+
+static void
+maintenance_selftest (char *args, int from_tty)
+{
+ run_self_tests ();
+}
+
+
void
_initialize_maint_cmds (void)
{
@@ -1154,6 +1165,13 @@ testsuite can check the command deprecator. You probably shouldn't use this,\n\
If you decide you want to use it: maintenance undeprecate 'commandname'"),
&maintenancelist);
+ add_cmd ("selftest", class_maintenance, maintenance_selftest, _("\
+Run gdb's unit tests.\n\
+Usage: maintenance selftest\n\
+This will run any unit tests that were built in to gdb.\n\
+gdb will abort if any test fails."),
+ &maintenancelist);
+
add_setshow_zinteger_cmd ("watchdog", class_maintenance, &watchdog, _("\
Set watchdog timer."), _("\
Show watchdog timer."), _("\
new file mode 100644
@@ -0,0 +1,66 @@
+/* GDB self-testing.
+ Copyright (C) 2016 Free Software Foundation, Inc.
+
+ This file is part of GDB.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include "defs.h"
+#include "selftest.h"
+#include "vec.h"
+
+typedef self_test_function *self_test_function_ptr;
+
+DEF_VEC_P (self_test_function_ptr);
+
+/* All the tests that have been registered. */
+
+static VEC (self_test_function_ptr) *tests;
+
+/* See selftest.h. */
+
+void
+register_self_test (self_test_function *function)
+{
+ VEC_safe_push (self_test_function_ptr, tests, function);
+}
+
+/* See selftest.h. */
+
+void
+run_self_tests (void)
+{
+ int i;
+ self_test_function_ptr func;
+
+ for (i = 0; VEC_iterate (self_test_function_ptr, tests, i, func); ++i)
+ {
+ QUIT;
+
+ TRY
+ {
+ (*func) ();
+ }
+ CATCH (ex, RETURN_MASK_ERROR)
+ {
+ internal_error (__FILE__, __LINE__,
+ _("self test threw exception: %s"),
+ ex.message);
+ }
+ END_CATCH
+ }
+
+ printf_filtered (_("Ran %u unit tests\n"),
+ VEC_length (self_test_function_ptr, tests));
+}
new file mode 100644
@@ -0,0 +1,35 @@
+/* GDB self-testing.
+ Copyright (C) 2016 Free Software Foundation, Inc.
+
+ This file is part of GDB.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef SELFTEST_H
+#define SELFTEST_H
+
+/* A test is just a function that does some checks and asserts if
+ something has gone wrong. */
+
+typedef void self_test_function (void);
+
+/* Register a new self-test. */
+
+extern void register_self_test (self_test_function *function);
+
+/* Run all the self tests. This will crash gdb if a test fails. */
+
+extern void run_self_tests (void);
+
+#endif /* SELFTEST_H */
@@ -1,3 +1,7 @@
+2016-04-26 Tom Tromey <tom@tromey.com>
+
+ * gdb.gdb/unittest.exp: New file.
+
2016-04-26 Bernhard Heckel <bernhard.heckel@intel.com>
* vla-type.exp: Print structure from toplevel.
new file mode 100644
@@ -0,0 +1,17 @@
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+gdb_start
+gdb_test "maintenance selftest" "Ran $decimal unit tests"