diff mbox

[v2,3/8] Add self-test framework to gdb

Message ID 1461789279-15996-4-git-send-email-tom@tromey.com
State New
Headers show

Commit Message

Tom Tromey April 27, 2016, 8:34 p.m. UTC
I wanted to unit test the Rust lexer, so I added a simple unit testing
command to gdb.

The intent is that self tests will only be compiled into gdb in
development mode.  In release mode they simply won't exist.  So, this
exposes $development to C code as GDB_SELF_TEST.

In development mode, test functions are registered with the self test
module.  A test function is just a function that does some checks, and
aborts on failure.  I chose this rather than something fancier because
I think any such failure will require debugging anyhow.

Then this adds a new "maint selftest" command which invokes the test
functions, and a new dejagnu test case that invokes it.

2016-04-27  Tom Tromey  <tom@tromey.com>

	* NEWS: Add "maint selftest" entry.
	* selftest.h: New file.
	* selftest.c: New file.
	* maint.c: Include selftest.h.
	(maintenance_selftest): New function.
	(_initialize_maint_cmds): Add "maint selftest" command.
	* configure.ac (GDB_SELF_TEST): Maybe define.
	* config.in, configure: Rebuild.
	* Makefile.in (SFILES): Add selftest.c.
	(COMMON_OBS): Add selftest.o.

2016-04-26  Tom Tromey  <tom@tromey.com>

	* gdb.texinfo (Maintenance Commands): Document "maint selftest".

2016-04-26  Tom Tromey  <tom@tromey.com>

	* gdb.gdb/unittest.exp: New file.
---
 gdb/ChangeLog                      | 13 ++++++++
 gdb/Makefile.in                    |  6 ++--
 gdb/NEWS                           |  3 ++
 gdb/config.in                      |  3 ++
 gdb/configure                      |  6 ++++
 gdb/configure.ac                   |  5 +++
 gdb/doc/ChangeLog                  |  4 +++
 gdb/doc/gdb.texinfo                |  6 ++++
 gdb/maint.c                        | 18 +++++++++++
 gdb/selftest.c                     | 66 ++++++++++++++++++++++++++++++++++++++
 gdb/selftest.h                     | 35 ++++++++++++++++++++
 gdb/testsuite/ChangeLog            |  4 +++
 gdb/testsuite/gdb.gdb/unittest.exp | 17 ++++++++++
 13 files changed, 184 insertions(+), 2 deletions(-)
 create mode 100644 gdb/selftest.c
 create mode 100644 gdb/selftest.h
 create mode 100644 gdb/testsuite/gdb.gdb/unittest.exp

Comments

Eli Zaretskii April 28, 2016, 5 a.m. UTC | #1
> From: Tom Tromey <tom@tromey.com>
> Cc: Tom Tromey <tom@tromey.com>
> Date: Wed, 27 Apr 2016 14:34:34 -0600
> 
> I wanted to unit test the Rust lexer, so I added a simple unit testing
> command to gdb.
> 
> The intent is that self tests will only be compiled into gdb in
> development mode.  In release mode they simply won't exist.  So, this
> exposes $development to C code as GDB_SELF_TEST.
> 
> In development mode, test functions are registered with the self test
> module.  A test function is just a function that does some checks, and
> aborts on failure.  I chose this rather than something fancier because
> I think any such failure will require debugging anyhow.
> 
> Then this adds a new "maint selftest" command which invokes the test
> functions, and a new dejagnu test case that invokes it.
> 
> 2016-04-27  Tom Tromey  <tom@tromey.com>
> 
> 	* NEWS: Add "maint selftest" entry.
> 	* selftest.h: New file.
> 	* selftest.c: New file.
> 	* maint.c: Include selftest.h.
> 	(maintenance_selftest): New function.
> 	(_initialize_maint_cmds): Add "maint selftest" command.
> 	* configure.ac (GDB_SELF_TEST): Maybe define.
> 	* config.in, configure: Rebuild.
> 	* Makefile.in (SFILES): Add selftest.c.
> 	(COMMON_OBS): Add selftest.o.
> 
> 2016-04-26  Tom Tromey  <tom@tromey.com>
> 
> 	* gdb.texinfo (Maintenance Commands): Document "maint selftest".
> 
> 2016-04-26  Tom Tromey  <tom@tromey.com>
> 
> 	* gdb.gdb/unittest.exp: New file.

OK for the documentation parts.

Thanks.
Yao Qi April 28, 2016, 10:28 a.m. UTC | #2
Tom Tromey <tom@tromey.com> writes:

Hi Tom,
this framework is sufficient in current needs, but I'd like to think
more about it because it is a framework, and more and more tests may use
it in the future.

> In development mode, test functions are registered with the self test
> module.  A test function is just a function that does some checks, and
> aborts on failure.  I chose this rather than something fancier because
> I think any such failure will require debugging anyhow.

In this framework, all unit tests are running in one GDB instance, so:

 - if one test fails, the following tests won't be run, even they are
   from different modules,
 - tests may change the GDB state, and the following tests may be affected,

I'd like the framework to only run tests from one module in one gdb
instance, restart gdb after they are finished, and continue running
tests from other modules.  For example, we have registered three self
tests in different modules,

register_self_test (rust_lex_tests);
register_self_test (breakpoint_tests);
register_self_test (linespec_tests);

In gdb.gdb/unittest.exp, we can get the list of self tests from
different modules via command "maint info selftest", like

$ (gdb) maint info selftest
rust_lex_tests
breakpoint_tests
linespec_tests

and the output can be saved in $list_of_tests, start a fresh gdb every
time, and execute these tests one by one, like

foreach test $list_of_tests {
  gdb_start
  gdb_test "maint selftest $test" "Ran $decimal unit tests"
  gdb_exit
}
Tom Tromey April 29, 2016, 3:13 p.m. UTC | #3
Yao> I'd like the framework to only run tests from one module in one gdb
Yao> instance, restart gdb after they are finished, and continue running
Yao> tests from other modules.  For example, we have registered three self
Yao> tests in different modules,

Sounds reasonable.

It would also be possible to use exceptions rather than asserts.
Then the current single command could be kept, since gdb wouldn't abort
on failure.

Let me know which you'd prefer.

Tom
Yao Qi April 29, 2016, 3:29 p.m. UTC | #4
Tom Tromey <tom@tromey.com> writes:

> It would also be possible to use exceptions rather than asserts.
> Then the current single command could be kept, since gdb wouldn't abort
> on failure.
>
> Let me know which you'd prefer.

Using exceptions is fine to me.
Doug Evans April 29, 2016, 5:28 p.m. UTC | #5
On Fri, Apr 29, 2016 at 8:29 AM, Yao Qi <qiyaoltc@gmail.com> wrote:
> Tom Tromey <tom@tromey.com> writes:
>
>> It would also be possible to use exceptions rather than asserts.
>> Then the current single command could be kept, since gdb wouldn't abort
>> on failure.
>>
>> Let me know which you'd prefer.
>
> Using exceptions is fine to me.

On a separate note, where do we, in the long term, want tests to live?

I'm wondering about the scalability of having tests in the same source file
reducing the readability, and whether we want conventions to avoid this.
I'm not saying 100% of tests have to, say, live in separate files
(e.g. foo.c tests -> foo-test.c or some such).
I'm just paranoid about having a repeat of the massive readability reduction
from the adding of record support to *-tdep.c files.

[Imagine how the sources will look 5 years from now.]
diff mbox

Patch

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index ea02ec9..d03a044 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,3 +1,16 @@ 
+2016-04-27  Tom Tromey  <tom@tromey.com>
+
+	* NEWS: Add "maint selftest" entry.
+	* selftest.h: New file.
+	* selftest.c: New file.
+	* maint.c: Include selftest.h.
+	(maintenance_selftest): New function.
+	(_initialize_maint_cmds): Add "maint selftest" command.
+	* configure.ac (GDB_SELF_TEST): Maybe define.
+	* config.in, configure: Rebuild.
+	* Makefile.in (SFILES): Add selftest.c.
+	(COMMON_OBS): Add selftest.o.
+
 2016-04-26  Tom Tromey  <tom@tromey.com>
 
 	* expprint.c: Include f-lang.h.
diff --git a/gdb/Makefile.in b/gdb/Makefile.in
index 400d2b0..c99bca6 100644
--- a/gdb/Makefile.in
+++ b/gdb/Makefile.in
@@ -869,8 +869,10 @@  SFILES = ada-exp.y ada-lang.c ada-typeprint.c ada-valprint.c ada-tasks.c \
 	proc-service.list progspace.c \
 	prologue-value.c psymtab.c \
 	regcache.c reggroups.c remote.c remote-fileio.c remote-notif.c reverse.c \
-	sentinel-frame.c \
+	selftest.c sentinel-frame.c \
 	serial.c ser-base.c ser-unix.c ser-event.c skip.c \
+	selftest.c sentinel-frame.c \
+	serial.c ser-base.c ser-unix.c skip.c \
 	solib.c solib-target.c source.c \
 	stabsread.c stack.c probe.c stap-probe.c std-regs.c \
 	symfile.c symfile-debug.c symfile-mem.c symmisc.c symtab.c \
@@ -1060,7 +1062,7 @@  COMMON_OBS = $(DEPFILES) $(CONFIG_OBS) $(YYOBJ) \
 	go-lang.o go-valprint.o go-typeprint.o \
 	jv-lang.o jv-valprint.o jv-typeprint.o jv-varobj.o \
 	m2-lang.o opencl-lang.o p-lang.o p-typeprint.o p-valprint.o \
-	sentinel-frame.o \
+	selftest.o sentinel-frame.o \
 	complaints.o typeprint.o \
 	ada-typeprint.o c-typeprint.o f-typeprint.o m2-typeprint.o \
 	ada-valprint.o c-valprint.o cp-valprint.o d-valprint.o f-valprint.o \
diff --git a/gdb/NEWS b/gdb/NEWS
index 7bf1e1a..77dfc0c 100644
--- a/gdb/NEWS
+++ b/gdb/NEWS
@@ -40,6 +40,9 @@  skip -rfunction regular-expression
 maint info line-table REGEXP
   Display the contents of GDB's internal line table data struture.
 
+maint selftest
+  Run any GDB unit tests that were compiled in.
+
 * Support for tracepoints and fast tracepoints on s390-linux and s390x-linux
   was added in GDBserver, including JIT compiling fast tracepoint's
   conditional expression bytecode into native code.
diff --git a/gdb/config.in b/gdb/config.in
index dc9da0a..905caf0 100644
--- a/gdb/config.in
+++ b/gdb/config.in
@@ -65,6 +65,9 @@ 
 /* Define to the default OS ABI for this configuration. */
 #undef GDB_OSABI_DEFAULT
 
+/* Define if self-testing features should be enabled */
+#undef GDB_SELF_TEST
+
 /* Define to 1 if you have `alloca', as a function or macro. */
 #undef HAVE_ALLOCA
 
diff --git a/gdb/configure b/gdb/configure
index 3cf95e7..9caf23f 100755
--- a/gdb/configure
+++ b/gdb/configure
@@ -16482,6 +16482,12 @@  ac_config_links="$ac_config_links $ac_config_links_1"
 $as_echo "#define GDB_DEFAULT_HOST_CHARSET \"UTF-8\"" >>confdefs.h
 
 
+if $development; then
+
+$as_echo "#define GDB_SELF_TEST 1" >>confdefs.h
+
+fi
+
 
   gdb_ac_transform=`echo "$program_transform_name" | sed -e 's/\\$\\$/\\$/g'`
   GDB_TRANSFORM_NAME=`echo gdb | sed -e "$gdb_ac_transform"`
diff --git a/gdb/configure.ac b/gdb/configure.ac
index 70452d3..b6a4aac 100644
--- a/gdb/configure.ac
+++ b/gdb/configure.ac
@@ -2330,6 +2330,11 @@  dnl  At the moment, we just assume it's UTF-8.
 AC_DEFINE(GDB_DEFAULT_HOST_CHARSET, "UTF-8",
           [Define to be a string naming the default host character set.])
 
+if $development; then
+  AC_DEFINE(GDB_SELF_TEST, 1,
+            [Define if self-testing features should be enabled])
+fi
+
 GDB_AC_TRANSFORM([gdb], [GDB_TRANSFORM_NAME])
 GDB_AC_TRANSFORM([gcore], [GCORE_TRANSFORM_NAME])
 AC_CONFIG_FILES([gcore], [chmod +x gcore])
diff --git a/gdb/doc/ChangeLog b/gdb/doc/ChangeLog
index 59956ef..748a8e2 100644
--- a/gdb/doc/ChangeLog
+++ b/gdb/doc/ChangeLog
@@ -1,3 +1,7 @@ 
+2016-04-26  Tom Tromey  <tom@tromey.com>
+
+	* gdb.texinfo (Maintenance Commands): Document "maint selftest".
+
 2016-04-13  Antoine Tremblay  <antoine.tremblay@ericsson.com>
 
 	* agentexpr.texi (byte): Fix zero_ext description.
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index 7abd55e..f42c5f0 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -34450,6 +34450,12 @@  that symbol is described.  The type chain produced by this command is
 a recursive definition of the data type as stored in @value{GDBN}'s
 data structures, including its flags and contained types.
 
+@kindex maint selftest
+@cindex self tests
+Run any self tests that were compiled in to @value{GDBN}.  If all the
+self tests pass, this will print a message.  If a self test fails,
+@value{GDBN} will crash.
+
 @kindex maint set dwarf always-disassemble
 @kindex maint show dwarf always-disassemble
 @item maint set dwarf always-disassemble
diff --git a/gdb/maint.c b/gdb/maint.c
index 358809d..60dc1ac 100644
--- a/gdb/maint.c
+++ b/gdb/maint.c
@@ -41,6 +41,7 @@ 
 #include "top.h"
 #include "timeval-utils.h"
 #include "maint.h"
+#include "selftest.h"
 
 #include "cli/cli-decode.h"
 #include "cli/cli-utils.h"
@@ -982,6 +983,16 @@  show_per_command_cmd (char *args, int from_tty)
   cmd_show_list (per_command_showlist, from_tty, "");
 }
 
+
+/* The "maintenance selftest" command.  */
+
+static void
+maintenance_selftest (char *args, int from_tty)
+{
+  run_self_tests ();
+}
+
+
 void
 _initialize_maint_cmds (void)
 {
@@ -1154,6 +1165,13 @@  testsuite can check the command deprecator. You probably shouldn't use this,\n\
 If you decide you want to use it: maintenance undeprecate 'commandname'"),
 	   &maintenancelist);
 
+  add_cmd ("selftest", class_maintenance, maintenance_selftest, _("\
+Run gdb's unit tests.\n\
+Usage: maintenance selftest\n\
+This will run any unit tests that were built in to gdb.\n\
+gdb will abort if any test fails."),
+	   &maintenancelist);
+
   add_setshow_zinteger_cmd ("watchdog", class_maintenance, &watchdog, _("\
 Set watchdog timer."), _("\
 Show watchdog timer."), _("\
diff --git a/gdb/selftest.c b/gdb/selftest.c
new file mode 100644
index 0000000..fecc527
--- /dev/null
+++ b/gdb/selftest.c
@@ -0,0 +1,66 @@ 
+/* GDB self-testing.
+   Copyright (C) 2016 Free Software Foundation, Inc.
+
+   This file is part of GDB.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include "defs.h"
+#include "selftest.h"
+#include "vec.h"
+
+typedef self_test_function *self_test_function_ptr;
+
+DEF_VEC_P (self_test_function_ptr);
+
+/* All the tests that have been registered.  */
+
+static VEC (self_test_function_ptr) *tests;
+
+/* See selftest.h.  */
+
+void
+register_self_test (self_test_function *function)
+{
+  VEC_safe_push (self_test_function_ptr, tests, function);
+}
+
+/* See selftest.h.  */
+
+void
+run_self_tests (void)
+{
+  int i;
+  self_test_function_ptr func;
+
+  for (i = 0; VEC_iterate (self_test_function_ptr, tests, i, func); ++i)
+    {
+      QUIT;
+
+      TRY
+	{
+	  (*func) ();
+	}
+      CATCH (ex, RETURN_MASK_ERROR)
+	{
+	  internal_error (__FILE__, __LINE__,
+			  _("self test threw exception: %s"),
+			  ex.message);
+	}
+      END_CATCH
+    }
+
+  printf_filtered (_("Ran %u unit tests\n"),
+		   VEC_length (self_test_function_ptr, tests));
+}
diff --git a/gdb/selftest.h b/gdb/selftest.h
new file mode 100644
index 0000000..b191e8f
--- /dev/null
+++ b/gdb/selftest.h
@@ -0,0 +1,35 @@ 
+/* GDB self-testing.
+   Copyright (C) 2016 Free Software Foundation, Inc.
+
+   This file is part of GDB.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#ifndef SELFTEST_H
+#define SELFTEST_H
+
+/* A test is just a function that does some checks and asserts if
+   something has gone wrong.  */
+
+typedef void self_test_function (void);
+
+/* Register a new self-test.  */
+
+extern void register_self_test (self_test_function *function);
+
+/* Run all the self tests.  This will crash gdb if a test fails.  */
+
+extern void run_self_tests (void);
+
+#endif /* SELFTEST_H */
diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
index 92eb9ff..c3b6104 100644
--- a/gdb/testsuite/ChangeLog
+++ b/gdb/testsuite/ChangeLog
@@ -1,3 +1,7 @@ 
+2016-04-26  Tom Tromey  <tom@tromey.com>
+
+	* gdb.gdb/unittest.exp: New file.
+
 2016-04-26  Bernhard Heckel  <bernhard.heckel@intel.com>
 
 	* vla-type.exp: Print structure from toplevel.
diff --git a/gdb/testsuite/gdb.gdb/unittest.exp b/gdb/testsuite/gdb.gdb/unittest.exp
new file mode 100644
index 0000000..8239349
--- /dev/null
+++ b/gdb/testsuite/gdb.gdb/unittest.exp
@@ -0,0 +1,17 @@ 
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+gdb_start
+gdb_test "maintenance selftest" "Ran $decimal unit tests"