diff mbox

[RFC] Run tests in sub dir of testsuite/

Message ID 1418266428-13166-1-git-send-email-yao@codesourcery.com
State New
Headers show

Commit Message

Yao Qi Dec. 11, 2014, 2:53 a.m. UTC
This patch is to run testsuite in a subdirectory of testsuite/, in
order to facilitate 'make clean'.  In sequential testing, directory
testsuite/sequential-tests is created, and in parallel testing,
directory testsuite/parallel-tests is created.  Testing artifacts
(such as directory gdb.base, gdb.ada, object files, shared libraries
and executables) are stored in it.

In testsuite/Makefile.in, we set OUTPUTDIR to either sequential-tests
or parallel-tests, determined by whether we run tests in parallel.
Makefile passes OUTPUTDIR to dejagnu, and lib/gdb.exp return the
expected file path according to OUTPUTDIR.

This patch changes the location of gdb.sum, so some build/test scripts
are affected.

gdb/testsuite:

2014-12-11  Yao Qi  <yao@codesourcery.com>

	* Makefile.in: Set OUTPUTDIR.
	(check-single): Create dir OUTPUTDIR and pass $(OUTPUTDIR) to
	runtest.
	(check-parallel): Use $(OUTPUTDIR).
	(check/%.exp): Pass $(OUTPUTDIR) to runtest.
	(check-perf): Likewise.
	(clean): Remove directory sequential-tests and parallel-tests.
	* lib/gdb.exp (standard_output_file): Return file path name
	according to new directory structure.
	(gdb_init): Use $OUTPUTDIR instead of outputs.
	(top-level): Create directory $OUTPUTDIR.
	* lib/trace-support.exp (get_in_proc_agent): Don't use $objdir
	in the path of inprocagent.
	* gdb.base/completion.exp: Use ${OUTPUTDIR}.
	* gdb.base/hashline1.exp: Use [pwd] instead of $objdir.
---
 gdb/testsuite/Makefile.in             | 25 +++++++++++++++----------
 gdb/testsuite/gdb.base/completion.exp |  2 +-
 gdb/testsuite/gdb.base/hashline1.exp  |  2 +-
 gdb/testsuite/lib/gdb.exp             | 21 ++++++++++++---------
 gdb/testsuite/lib/trace-support.exp   |  3 +--
 5 files changed, 30 insertions(+), 23 deletions(-)

Comments

Doug Evans Dec. 11, 2014, 3:05 a.m. UTC | #1
On Wed, Dec 10, 2014 at 6:53 PM, Yao Qi <yao@codesourcery.com> wrote:
> This patch is to run testsuite in a subdirectory of testsuite/, in
> order to facilitate 'make clean'.  In sequential testing, directory
> testsuite/sequential-tests is created, and in parallel testing,
> directory testsuite/parallel-tests is created.  Testing artifacts
> (such as directory gdb.base, gdb.ada, object files, shared libraries
> and executables) are stored in it.
>
> In testsuite/Makefile.in, we set OUTPUTDIR to either sequential-tests
> or parallel-tests, determined by whether we run tests in parallel.
> Makefile passes OUTPUTDIR to dejagnu, and lib/gdb.exp return the
> expected file path according to OUTPUTDIR.
>
> This patch changes the location of gdb.sum, so some build/test scripts
> are affected.
>
> gdb/testsuite:
>
> 2014-12-11  Yao Qi  <yao@codesourcery.com>
>
>         * Makefile.in: Set OUTPUTDIR.
>         (check-single): Create dir OUTPUTDIR and pass $(OUTPUTDIR) to
>         runtest.
>         (check-parallel): Use $(OUTPUTDIR).
>         (check/%.exp): Pass $(OUTPUTDIR) to runtest.
>         (check-perf): Likewise.
>         (clean): Remove directory sequential-tests and parallel-tests.
>         * lib/gdb.exp (standard_output_file): Return file path name
>         according to new directory structure.
>         (gdb_init): Use $OUTPUTDIR instead of outputs.
>         (top-level): Create directory $OUTPUTDIR.
>         * lib/trace-support.exp (get_in_proc_agent): Don't use $objdir
>         in the path of inprocagent.
>         * gdb.base/completion.exp: Use ${OUTPUTDIR}.
>         * gdb.base/hashline1.exp: Use [pwd] instead of $objdir.

Hi.

I can't review the patch right now, but I have a question.
Is there intended to be a followup patch to remove
all the makefiles in testsuite/ except testsuite/Makefile.in?

[I realize this patch is just RFC.
I just want to understand where this is headed.]

btw, while it might be cleaner to move gdb.log,gdb.sum into
the subdirs, it's not necessary.
I can update my build scripts, but if we're going
to be affecting a lot of build scripts then I'd
reconsider.
Yao Qi Dec. 11, 2014, 3:17 a.m. UTC | #2
On 12/11/2014 11:05 AM, Doug Evans wrote:
> Is there intended to be a followup patch to remove
> all the makefiles in testsuite/ except testsuite/Makefile.in?
> 
> [I realize this patch is just RFC.
> I just want to understand where this is headed.]

Yes, that is what I want to do in next step.

> 
> btw, while it might be cleaner to move gdb.log,gdb.sum into
> the subdirs, it's not necessary.
> I can update my build scripts, but if we're going
> to be affecting a lot of build scripts then I'd
> reconsider.

I am fine to leave gdb.log and gdb.sum in testsuite/.  The reason I move
them into testsuite/$OUTPUTSDIR/ is to make $OUTPUTSDIR self-contained.
 Either way is fine to me, and I don't have a strong
opinion on this.
Pedro Alves Dec. 12, 2014, 11:35 a.m. UTC | #3
On 12/11/2014 03:17 AM, Yao Qi wrote:
> On 12/11/2014 11:05 AM, Doug Evans wrote:
>> Is there intended to be a followup patch to remove
>> all the makefiles in testsuite/ except testsuite/Makefile.in?
>>
>> [I realize this patch is just RFC.
>> I just want to understand where this is headed.]
> 
> Yes, that is what I want to do in next step.
> 
>>
>> btw, while it might be cleaner to move gdb.log,gdb.sum into
>> the subdirs, it's not necessary.
>> I can update my build scripts, but if we're going
>> to be affecting a lot of build scripts then I'd
>> reconsider.
> 
> I am fine to leave gdb.log and gdb.sum in testsuite/.  The reason I move
> them into testsuite/$OUTPUTSDIR/ is to make $OUTPUTSDIR self-contained.
>  Either way is fine to me, and I don't have a strong
> opinion on this.

I tend to think it's better to leave them be if there's no
real good reason to move them.

Note that "make clean" doesn't delete gdb.sum/gdb.log,
"make distclean" does.  If "make clean" is just going to be "rm -f $subdirs",
then to preserve that it seems simplest to keep gdb.sum and gdb.log out
of the subdirs.

(nit, on "sequential-tests"/"parallel-tests" -- it's the run, not the
tests that are sequential or parallel.  I'd suggest dropping
"-tests" from the dir names or use "-run", to avoid potential confusion.)

Thanks,
Pedro Alves
Yao Qi Dec. 14, 2014, 6:11 a.m. UTC | #4
Pedro Alves <palves@redhat.com> writes:

> I tend to think it's better to leave them be if there's no
> real good reason to move them.
>
> Note that "make clean" doesn't delete gdb.sum/gdb.log,
> "make distclean" does.  If "make clean" is just going to be "rm -f $subdirs",
> then to preserve that it seems simplest to keep gdb.sum and gdb.log out
> of the subdirs.

OK, let us leave gdb.sum and gdb.log there.

>
> (nit, on "sequential-tests"/"parallel-tests" -- it's the run, not the
> tests that are sequential or parallel.  I'd suggest dropping
> "-tests" from the dir names or use "-run", to avoid potential confusion.)

"sequential-run" and "parallel-run" sounds good to me.
diff mbox

Patch

diff --git a/gdb/testsuite/Makefile.in b/gdb/testsuite/Makefile.in
index 1c923cd..99d5221 100644
--- a/gdb/testsuite/Makefile.in
+++ b/gdb/testsuite/Makefile.in
@@ -145,8 +145,12 @@  installcheck:
 # given.  If RUNTESTFLAGS is not empty, then by default the tests will
 # be serialized.  This can be overridden by setting FORCE_PARALLEL to
 # any non-empty value.  For a non-GNU make, do not parallelize.
+# If we run tests in parallel, set the OUTPUTDIR parallel-tests, otherwise
+# set it sequential-tests.
 @GMAKE_TRUE@CHECK_TARGET = $(if $(FORCE_PARALLEL),check-parallel,$(if $(RUNTESTFLAGS),check-single,$(if $(saw_dash_j),check-parallel,check-single)))
+@GMAKE_TRUE@OUTPUTDIR = $(if $(FORCE_PARALLEL),parallel-tests,$(if $(RUNTESTFLAGS),sequential-tests,$(if $(saw_dash_j),parallel-tests,sequential-tests)))
 @GMAKE_FALSE@CHECK_TARGET = check-single
+@GMAKE_FALSE@OUTPUTDIR = sequential-tests
 
 # Note that we must resort to a recursive make invocation here,
 # because GNU make 3.82 has a bug preventing MAKEFLAGS from being used
@@ -189,16 +193,17 @@  DO_RUNTEST = \
 @GMAKE_FALSE@expanded_tests_or_none = $(TESTS)
 
 check-single:
-	$(DO_RUNTEST) $(RUNTESTFLAGS) $(expanded_tests_or_none)
+	-mkdir -p $(OUTPUTDIR)
+	$(DO_RUNTEST) OUTPUTDIR=$(OUTPUTDIR) --outdir=$(OUTPUTDIR) $(RUNTESTFLAGS) $(expanded_tests_or_none)
 
 check-parallel:
-	-rm -rf cache outputs temp
+	-rm -rf cache $(OUTPUTDIR) temp
 	$(MAKE) -k do-check-parallel; \
 	$(SHELL) $(srcdir)/dg-extract-results.sh \
-	  `find outputs -name gdb.sum -print` > gdb.sum; \
+	  `find $(OUTPUTDIR) -name gdb.sum -print` > $(OUTPUTDIR)/gdb.sum; \
 	$(SHELL) $(srcdir)/dg-extract-results.sh -L \
-	  `find outputs -name gdb.log -print` > gdb.log
-	@sed -n '/=== gdb Summary ===/,$$ p' gdb.sum
+	  `find $(OUTPUTDIR) -name gdb.log -print` > $(OUTPUTDIR)/gdb.log
+	@sed -n '/=== gdb Summary ===/,$$ p' $(OUTPUTDIR)/gdb.sum
 
 # Turn a list of .exp files into "check/" targets.  Only examine .exp
 # files appearing in a gdb.* directory -- we don't want to pick up
@@ -222,8 +227,8 @@  do-check-parallel: $(TEST_TARGETS)
 	@:
 
 @GMAKE_TRUE@check/%.exp:
-@GMAKE_TRUE@	-mkdir -p outputs/$*
-@GMAKE_TRUE@	@$(DO_RUNTEST) GDB_PARALLEL=yes --outdir=outputs/$* $*.exp $(RUNTESTFLAGS)
+@GMAKE_TRUE@	-mkdir -p $(OUTPUTDIR)/$*
+@GMAKE_TRUE@	@$(DO_RUNTEST) GDB_PARALLEL=yes OUTPUTDIR=$(OUTPUTDIR) --outdir=$(OUTPUTDIR)/$* $*.exp $(RUNTESTFLAGS)
 
 check/no-matching-tests-found:
 	@echo ""
@@ -231,8 +236,8 @@  check/no-matching-tests-found:
 	@echo ""
 
 check-perf: all $(abs_builddir)/site.exp
-	@if test ! -d gdb.perf; then mkdir gdb.perf; fi
-	$(DO_RUNTEST) --directory=gdb.perf --outdir gdb.perf GDB_PERFTEST_MODE=both $(RUNTESTFLAGS)
+	@if test ! -d $(OUTPUTDIR)/gdb.perf; then mkdir $(OUTPUTDIR)/gdb.perf; fi
+	$(DO_RUNTEST) --directory=gdb.perf --outdir $(OUTPUTDIR)/gdb.perf GDB_PERFTEST_MODE=both OUTPUTDIR=$(OUTPUTDIR) $(RUNTESTFLAGS)
 
 force:;
 
@@ -240,7 +245,7 @@  clean mostlyclean:
 	-rm -f *~ core *.o a.out xgdb *.x *.grt bigcore.corefile .gdb_history
 	-rm -f core.* *.tf *.cl *.py tracecommandsscript copy1.txt zzz-gdbscript
 	-rm -f *.dwo *.dwp
-	-rm -rf outputs temp cache
+	-rm -rf sequential-tests parallel-tests temp cache
 	-rm -f read1.so expect-read1
 	if [ x"${ALL_SUBDIRS}" != x ] ; then \
 	    for dir in ${ALL_SUBDIRS}; \
diff --git a/gdb/testsuite/gdb.base/completion.exp b/gdb/testsuite/gdb.base/completion.exp
index c633a51..336600a 100644
--- a/gdb/testsuite/gdb.base/completion.exp
+++ b/gdb/testsuite/gdb.base/completion.exp
@@ -108,7 +108,7 @@  gdb_test_multiple "target ctf" "" {
 # is a parallel run or not.  Firstly check file exists, and then
 # do the test on file completion.
 
-foreach dir1 [ list "./gdb.base" "./outputs/gdb.base/completion" ] {
+foreach dir1 [ list "${OUTPUTDIR}/gdb.base" "${OUTPUTDIR}/gdb.base/completion" ] {
     if { [remote_file host exists ${dir1}/completion]
 	 && [remote_file host exists ${dir1}/completion0.o]
 	 && [remote_file host exists ${dir1}/completion1.o] } {
diff --git a/gdb/testsuite/gdb.base/hashline1.exp b/gdb/testsuite/gdb.base/hashline1.exp
index 1260b80..32290cd 100644
--- a/gdb/testsuite/gdb.base/hashline1.exp
+++ b/gdb/testsuite/gdb.base/hashline1.exp
@@ -31,7 +31,7 @@  close $fd
 
 # The choice of path name for the source file is important in order to trigger
 # the bug.  Using $new_srcfile here won't trigger the bug.
-set compile_name [relative_filename $objdir $new_srcfile]
+set compile_name [relative_filename [pwd] $new_srcfile]
 if { [gdb_compile $compile_name "${binfile}" executable {debug}] != "" } {
     untested hashline1.exp
     return -1
diff --git a/gdb/testsuite/lib/gdb.exp b/gdb/testsuite/lib/gdb.exp
index a29b661..38a3cbd 100644
--- a/gdb/testsuite/lib/gdb.exp
+++ b/gdb/testsuite/lib/gdb.exp
@@ -3672,15 +3672,16 @@  proc default_gdb_init { test_file_name } {
 # the directory is returned.
 
 proc standard_output_file {basename} {
-    global objdir subdir gdb_test_file_name GDB_PARALLEL
+    global objdir subdir gdb_test_file_name GDB_PARALLEL OUTPUTDIR
 
+    set dir [file join $objdir $OUTPUTDIR $subdir]
     if {[info exists GDB_PARALLEL]} {
-	set dir [file join $objdir outputs $subdir $gdb_test_file_name]
-	file mkdir $dir
-	return [file join $dir $basename]
-    } else {
-	return [file join $objdir $subdir $basename]
+	set dir [file join $dir $gdb_test_file_name]
     }
+
+    file mkdir $dir
+
+    return [file join $dir $basename]
 }
 
 # Return the name of a file in our standard temporary directory.
@@ -3809,9 +3810,9 @@  proc gdb_init { test_file_name } {
     # inotify-tools package to use this.
     global GDB_INOTIFY inotify_pid
     if {[info exists GDB_INOTIFY] && ![info exists inotify_pid]} {
-	global outdir tool inotify_log_file
+	global outdir tool inotify_log_file OUTPUTDIR
 
-	set exclusions {outputs temp gdb[.](log|sum) cache}
+	set exclusions {$OUTPUTDIR temp gdb[.](log|sum) cache}
 	set exclusion_re ([join $exclusions |])
 
 	set inotify_log_file [standard_temp_file inotify.out]
@@ -4805,10 +4806,12 @@  if {[info exists GDB_PARALLEL]} {
     if {[is_remote host]} {
 	unset GDB_PARALLEL
     } else {
-	file mkdir outputs temp cache
+	file mkdir temp cache
     }
 }
 
+file mkdir $OUTPUTDIR
+
 proc core_find {binfile {deletefiles {}} {arg ""}} {
     global objdir subdir
 
diff --git a/gdb/testsuite/lib/trace-support.exp b/gdb/testsuite/lib/trace-support.exp
index a076bac..f090bfa 100644
--- a/gdb/testsuite/lib/trace-support.exp
+++ b/gdb/testsuite/lib/trace-support.exp
@@ -319,12 +319,11 @@  proc gdb_find_recursion_test_baseline { filename } {
 # Return the location of the IPA library.
 
 proc get_in_proc_agent {} {
-    global objdir
 
     if [target_info exists in_proc_agent] {
 	return [target_info in_proc_agent]
     } else {
-	return $objdir/../gdbserver/libinproctrace.so
+	return ../gdbserver/libinproctrace.so
     }
 }