diff mbox

testsuite: Add --status to runtest invocation

Message ID 1453157560-7080-1-git-send-email-simon.marchi@ericsson.com
State New, archived
Headers

Commit Message

Simon Marchi Jan. 18, 2016, 10:52 p.m. UTC 
  As discussed in this thread:

  https://sourceware.org/ml/gdb-patches/2016-01/msg00243.html

By default, if a test driver (a test .exp) ends with an uncaught
error/exception, the runtest command will still have a return code of 0
(success).  I think that if a test (or the environment) is broken and
the test ends up with an exception, it should be considered as failed so
that we can notice it and fix it.

Passing the --status flag to runtest will make it return an error if one
of the test it runs ends up with an uncaught error.

gdb/testsuite/ChangeLog:

	* Makefile.in (check-single): Pass --status to runtest.
	(check/%.exp): Likewise.
---
 gdb/testsuite/Makefile.in | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comments

Pedro Alves Jan. 19, 2016, 11:24 a.m. UTC | #1 
On 01/18/2016 10:52 PM, Simon Marchi wrote:
> As discussed in this thread:
> 
>   https://sourceware.org/ml/gdb-patches/2016-01/msg00243.html
> 
> By default, if a test driver (a test .exp) ends with an uncaught
> error/exception, the runtest command will still have a return code of 0
> (success).  I think that if a test (or the environment) is broken and
> the test ends up with an exception, it should be considered as failed so
> that we can notice it and fix it.
> 
> Passing the --status flag to runtest will make it return an error if one
> of the test it runs ends up with an uncaught error.
> 
> gdb/testsuite/ChangeLog:
> 
> 	* Makefile.in (check-single): Pass --status to runtest.
> 	(check/%.exp): Likewise.

Hmm, the perf bits of the runtest invocation already pass --status:

@GMAKE_TRUE@	$(DO_RUNTEST) --status --outdir=gdb.perf/outputs/$* lib/build-piece.exp WORKER=$* GDB_PARALLEL=gdb.perf $(RUNTESTFLAGS) GDB_PERFTEST_MODE=compile GDB_PERFTEST_SUBMODE=build-pieces
...
@GMAKE_TRUE@	$(DO_RUNTEST) --status --directory=gdb.perf --outdir gdb.perf/workers GDB_PARALLEL=gdb.perf $(RUNTESTFLAGS) GDB_PERFTEST_MODE=compile GDB_PERFTEST_SUBMODE=gen-workers
...
@GMAKE_TRUE@	$(DO_RUNTEST) --status --directory=gdb.perf --outdir gdb.perf GDB_PARALLEL=gdb.perf $(RUNTESTFLAGS) GDB_PERFTEST_MODE=compile GDB_PERFTEST_SUBMODE=final

But there's one that doesn't:

check-perf: all $(abs_builddir)/site.exp
	@if test ! -d gdb.perf; then mkdir gdb.perf; fi
	$(DO_RUNTEST) --directory=gdb.perf --outdir gdb.perf GDB_PERFTEST_MODE=$(GDB_PERFTEST_MODE) $(RUNTESTFLAGS)

Seems like an oversight?

How about adding --status to DO_RUNTEST directly instead, so
that all invocations are always covered?

Thanks,
Pedro Alves
diff mbox

Patch

diff --git a/gdb/testsuite/Makefile.in b/gdb/testsuite/Makefile.in
index fb0d8e5..ae7fa7b 100644
--- a/gdb/testsuite/Makefile.in
+++ b/gdb/testsuite/Makefile.in
@@ -193,7 +193,7 @@  DO_RUNTEST = \
 @GMAKE_TRUE@	$(MAKE) check TESTS="gdb.$*/*.exp"
 
 check-single:
-	$(DO_RUNTEST) $(RUNTESTFLAGS) $(expanded_tests_or_none)
+	$(DO_RUNTEST) --status $(RUNTESTFLAGS) $(expanded_tests_or_none)
 
 check-parallel:
 	-rm -rf cache outputs temp
@@ -227,7 +227,7 @@  do-check-parallel: $(TEST_TARGETS)
 
 @GMAKE_TRUE@check/%.exp:
 @GMAKE_TRUE@	-mkdir -p outputs/$*
-@GMAKE_TRUE@	@$(DO_RUNTEST) GDB_PARALLEL=yes --outdir=outputs/$* $*.exp $(RUNTESTFLAGS)
+@GMAKE_TRUE@	@$(DO_RUNTEST) GDB_PARALLEL=yes --outdir=outputs/$* $*.exp --status $(RUNTESTFLAGS)
 
 check/no-matching-tests-found:
 	@echo ""