[RFC] Run tests in sub dir of testsuite/
Commit Message
This patch is to run testsuite in a subdirectory of testsuite/, in
order to facilitate 'make clean'. In sequential testing, directory
testsuite/sequential-tests is created, and in parallel testing,
directory testsuite/parallel-tests is created. Testing artifacts
(such as directory gdb.base, gdb.ada, object files, shared libraries
and executables) are stored in it.
In testsuite/Makefile.in, we set OUTPUTDIR to either sequential-tests
or parallel-tests, determined by whether we run tests in parallel.
Makefile passes OUTPUTDIR to dejagnu, and lib/gdb.exp return the
expected file path according to OUTPUTDIR.
This patch changes the location of gdb.sum, so some build/test scripts
are affected.
gdb/testsuite:
2014-12-11 Yao Qi <yao@codesourcery.com>
* Makefile.in: Set OUTPUTDIR.
(check-single): Create dir OUTPUTDIR and pass $(OUTPUTDIR) to
runtest.
(check-parallel): Use $(OUTPUTDIR).
(check/%.exp): Pass $(OUTPUTDIR) to runtest.
(check-perf): Likewise.
(clean): Remove directory sequential-tests and parallel-tests.
* lib/gdb.exp (standard_output_file): Return file path name
according to new directory structure.
(gdb_init): Use $OUTPUTDIR instead of outputs.
(top-level): Create directory $OUTPUTDIR.
* lib/trace-support.exp (get_in_proc_agent): Don't use $objdir
in the path of inprocagent.
* gdb.base/completion.exp: Use ${OUTPUTDIR}.
* gdb.base/hashline1.exp: Use [pwd] instead of $objdir.
---
gdb/testsuite/Makefile.in | 25 +++++++++++++++----------
gdb/testsuite/gdb.base/completion.exp | 2 +-
gdb/testsuite/gdb.base/hashline1.exp | 2 +-
gdb/testsuite/lib/gdb.exp | 21 ++++++++++++---------
gdb/testsuite/lib/trace-support.exp | 3 +--
5 files changed, 30 insertions(+), 23 deletions(-)
Comments
On Wed, Dec 10, 2014 at 6:53 PM, Yao Qi <yao@codesourcery.com> wrote:
> This patch is to run testsuite in a subdirectory of testsuite/, in
> order to facilitate 'make clean'. In sequential testing, directory
> testsuite/sequential-tests is created, and in parallel testing,
> directory testsuite/parallel-tests is created. Testing artifacts
> (such as directory gdb.base, gdb.ada, object files, shared libraries
> and executables) are stored in it.
>
> In testsuite/Makefile.in, we set OUTPUTDIR to either sequential-tests
> or parallel-tests, determined by whether we run tests in parallel.
> Makefile passes OUTPUTDIR to dejagnu, and lib/gdb.exp return the
> expected file path according to OUTPUTDIR.
>
> This patch changes the location of gdb.sum, so some build/test scripts
> are affected.
>
> gdb/testsuite:
>
> 2014-12-11 Yao Qi <yao@codesourcery.com>
>
> * Makefile.in: Set OUTPUTDIR.
> (check-single): Create dir OUTPUTDIR and pass $(OUTPUTDIR) to
> runtest.
> (check-parallel): Use $(OUTPUTDIR).
> (check/%.exp): Pass $(OUTPUTDIR) to runtest.
> (check-perf): Likewise.
> (clean): Remove directory sequential-tests and parallel-tests.
> * lib/gdb.exp (standard_output_file): Return file path name
> according to new directory structure.
> (gdb_init): Use $OUTPUTDIR instead of outputs.
> (top-level): Create directory $OUTPUTDIR.
> * lib/trace-support.exp (get_in_proc_agent): Don't use $objdir
> in the path of inprocagent.
> * gdb.base/completion.exp: Use ${OUTPUTDIR}.
> * gdb.base/hashline1.exp: Use [pwd] instead of $objdir.
Hi.
I can't review the patch right now, but I have a question.
Is there intended to be a followup patch to remove
all the makefiles in testsuite/ except testsuite/Makefile.in?
[I realize this patch is just RFC.
I just want to understand where this is headed.]
btw, while it might be cleaner to move gdb.log,gdb.sum into
the subdirs, it's not necessary.
I can update my build scripts, but if we're going
to be affecting a lot of build scripts then I'd
reconsider.
On 12/11/2014 11:05 AM, Doug Evans wrote:
> Is there intended to be a followup patch to remove
> all the makefiles in testsuite/ except testsuite/Makefile.in?
>
> [I realize this patch is just RFC.
> I just want to understand where this is headed.]
Yes, that is what I want to do in next step.
>
> btw, while it might be cleaner to move gdb.log,gdb.sum into
> the subdirs, it's not necessary.
> I can update my build scripts, but if we're going
> to be affecting a lot of build scripts then I'd
> reconsider.
I am fine to leave gdb.log and gdb.sum in testsuite/. The reason I move
them into testsuite/$OUTPUTSDIR/ is to make $OUTPUTSDIR self-contained.
Either way is fine to me, and I don't have a strong
opinion on this.
On 12/11/2014 03:17 AM, Yao Qi wrote:
> On 12/11/2014 11:05 AM, Doug Evans wrote:
>> Is there intended to be a followup patch to remove
>> all the makefiles in testsuite/ except testsuite/Makefile.in?
>>
>> [I realize this patch is just RFC.
>> I just want to understand where this is headed.]
>
> Yes, that is what I want to do in next step.
>
>>
>> btw, while it might be cleaner to move gdb.log,gdb.sum into
>> the subdirs, it's not necessary.
>> I can update my build scripts, but if we're going
>> to be affecting a lot of build scripts then I'd
>> reconsider.
>
> I am fine to leave gdb.log and gdb.sum in testsuite/. The reason I move
> them into testsuite/$OUTPUTSDIR/ is to make $OUTPUTSDIR self-contained.
> Either way is fine to me, and I don't have a strong
> opinion on this.
I tend to think it's better to leave them be if there's no
real good reason to move them.
Note that "make clean" doesn't delete gdb.sum/gdb.log,
"make distclean" does. If "make clean" is just going to be "rm -f $subdirs",
then to preserve that it seems simplest to keep gdb.sum and gdb.log out
of the subdirs.
(nit, on "sequential-tests"/"parallel-tests" -- it's the run, not the
tests that are sequential or parallel. I'd suggest dropping
"-tests" from the dir names or use "-run", to avoid potential confusion.)
Thanks,
Pedro Alves
Pedro Alves <palves@redhat.com> writes:
> I tend to think it's better to leave them be if there's no
> real good reason to move them.
>
> Note that "make clean" doesn't delete gdb.sum/gdb.log,
> "make distclean" does. If "make clean" is just going to be "rm -f $subdirs",
> then to preserve that it seems simplest to keep gdb.sum and gdb.log out
> of the subdirs.
OK, let us leave gdb.sum and gdb.log there.
>
> (nit, on "sequential-tests"/"parallel-tests" -- it's the run, not the
> tests that are sequential or parallel. I'd suggest dropping
> "-tests" from the dir names or use "-run", to avoid potential confusion.)
"sequential-run" and "parallel-run" sounds good to me.
@@ -145,8 +145,12 @@ installcheck:
# given. If RUNTESTFLAGS is not empty, then by default the tests will
# be serialized. This can be overridden by setting FORCE_PARALLEL to
# any non-empty value. For a non-GNU make, do not parallelize.
+# If we run tests in parallel, set the OUTPUTDIR parallel-tests, otherwise
+# set it sequential-tests.
@GMAKE_TRUE@CHECK_TARGET = $(if $(FORCE_PARALLEL),check-parallel,$(if $(RUNTESTFLAGS),check-single,$(if $(saw_dash_j),check-parallel,check-single)))
+@GMAKE_TRUE@OUTPUTDIR = $(if $(FORCE_PARALLEL),parallel-tests,$(if $(RUNTESTFLAGS),sequential-tests,$(if $(saw_dash_j),parallel-tests,sequential-tests)))
@GMAKE_FALSE@CHECK_TARGET = check-single
+@GMAKE_FALSE@OUTPUTDIR = sequential-tests
# Note that we must resort to a recursive make invocation here,
# because GNU make 3.82 has a bug preventing MAKEFLAGS from being used
@@ -189,16 +193,17 @@ DO_RUNTEST = \
@GMAKE_FALSE@expanded_tests_or_none = $(TESTS)
check-single:
- $(DO_RUNTEST) $(RUNTESTFLAGS) $(expanded_tests_or_none)
+ -mkdir -p $(OUTPUTDIR)
+ $(DO_RUNTEST) OUTPUTDIR=$(OUTPUTDIR) --outdir=$(OUTPUTDIR) $(RUNTESTFLAGS) $(expanded_tests_or_none)
check-parallel:
- -rm -rf cache outputs temp
+ -rm -rf cache $(OUTPUTDIR) temp
$(MAKE) -k do-check-parallel; \
$(SHELL) $(srcdir)/dg-extract-results.sh \
- `find outputs -name gdb.sum -print` > gdb.sum; \
+ `find $(OUTPUTDIR) -name gdb.sum -print` > $(OUTPUTDIR)/gdb.sum; \
$(SHELL) $(srcdir)/dg-extract-results.sh -L \
- `find outputs -name gdb.log -print` > gdb.log
- @sed -n '/=== gdb Summary ===/,$$ p' gdb.sum
+ `find $(OUTPUTDIR) -name gdb.log -print` > $(OUTPUTDIR)/gdb.log
+ @sed -n '/=== gdb Summary ===/,$$ p' $(OUTPUTDIR)/gdb.sum
# Turn a list of .exp files into "check/" targets. Only examine .exp
# files appearing in a gdb.* directory -- we don't want to pick up
@@ -222,8 +227,8 @@ do-check-parallel: $(TEST_TARGETS)
@:
@GMAKE_TRUE@check/%.exp:
-@GMAKE_TRUE@ -mkdir -p outputs/$*
-@GMAKE_TRUE@ @$(DO_RUNTEST) GDB_PARALLEL=yes --outdir=outputs/$* $*.exp $(RUNTESTFLAGS)
+@GMAKE_TRUE@ -mkdir -p $(OUTPUTDIR)/$*
+@GMAKE_TRUE@ @$(DO_RUNTEST) GDB_PARALLEL=yes OUTPUTDIR=$(OUTPUTDIR) --outdir=$(OUTPUTDIR)/$* $*.exp $(RUNTESTFLAGS)
check/no-matching-tests-found:
@echo ""
@@ -231,8 +236,8 @@ check/no-matching-tests-found:
@echo ""
check-perf: all $(abs_builddir)/site.exp
- @if test ! -d gdb.perf; then mkdir gdb.perf; fi
- $(DO_RUNTEST) --directory=gdb.perf --outdir gdb.perf GDB_PERFTEST_MODE=both $(RUNTESTFLAGS)
+ @if test ! -d $(OUTPUTDIR)/gdb.perf; then mkdir $(OUTPUTDIR)/gdb.perf; fi
+ $(DO_RUNTEST) --directory=gdb.perf --outdir $(OUTPUTDIR)/gdb.perf GDB_PERFTEST_MODE=both OUTPUTDIR=$(OUTPUTDIR) $(RUNTESTFLAGS)
force:;
@@ -240,7 +245,7 @@ clean mostlyclean:
-rm -f *~ core *.o a.out xgdb *.x *.grt bigcore.corefile .gdb_history
-rm -f core.* *.tf *.cl *.py tracecommandsscript copy1.txt zzz-gdbscript
-rm -f *.dwo *.dwp
- -rm -rf outputs temp cache
+ -rm -rf sequential-tests parallel-tests temp cache
-rm -f read1.so expect-read1
if [ x"${ALL_SUBDIRS}" != x ] ; then \
for dir in ${ALL_SUBDIRS}; \
@@ -108,7 +108,7 @@ gdb_test_multiple "target ctf" "" {
# is a parallel run or not. Firstly check file exists, and then
# do the test on file completion.
-foreach dir1 [ list "./gdb.base" "./outputs/gdb.base/completion" ] {
+foreach dir1 [ list "${OUTPUTDIR}/gdb.base" "${OUTPUTDIR}/gdb.base/completion" ] {
if { [remote_file host exists ${dir1}/completion]
&& [remote_file host exists ${dir1}/completion0.o]
&& [remote_file host exists ${dir1}/completion1.o] } {
@@ -31,7 +31,7 @@ close $fd
# The choice of path name for the source file is important in order to trigger
# the bug. Using $new_srcfile here won't trigger the bug.
-set compile_name [relative_filename $objdir $new_srcfile]
+set compile_name [relative_filename [pwd] $new_srcfile]
if { [gdb_compile $compile_name "${binfile}" executable {debug}] != "" } {
untested hashline1.exp
return -1
@@ -3672,15 +3672,16 @@ proc default_gdb_init { test_file_name } {
# the directory is returned.
proc standard_output_file {basename} {
- global objdir subdir gdb_test_file_name GDB_PARALLEL
+ global objdir subdir gdb_test_file_name GDB_PARALLEL OUTPUTDIR
+ set dir [file join $objdir $OUTPUTDIR $subdir]
if {[info exists GDB_PARALLEL]} {
- set dir [file join $objdir outputs $subdir $gdb_test_file_name]
- file mkdir $dir
- return [file join $dir $basename]
- } else {
- return [file join $objdir $subdir $basename]
+ set dir [file join $dir $gdb_test_file_name]
}
+
+ file mkdir $dir
+
+ return [file join $dir $basename]
}
# Return the name of a file in our standard temporary directory.
@@ -3809,9 +3810,9 @@ proc gdb_init { test_file_name } {
# inotify-tools package to use this.
global GDB_INOTIFY inotify_pid
if {[info exists GDB_INOTIFY] && ![info exists inotify_pid]} {
- global outdir tool inotify_log_file
+ global outdir tool inotify_log_file OUTPUTDIR
- set exclusions {outputs temp gdb[.](log|sum) cache}
+ set exclusions {$OUTPUTDIR temp gdb[.](log|sum) cache}
set exclusion_re ([join $exclusions |])
set inotify_log_file [standard_temp_file inotify.out]
@@ -4805,10 +4806,12 @@ if {[info exists GDB_PARALLEL]} {
if {[is_remote host]} {
unset GDB_PARALLEL
} else {
- file mkdir outputs temp cache
+ file mkdir temp cache
}
}
+file mkdir $OUTPUTDIR
+
proc core_find {binfile {deletefiles {}} {arg ""}} {
global objdir subdir
@@ -319,12 +319,11 @@ proc gdb_find_recursion_test_baseline { filename } {
# Return the location of the IPA library.
proc get_in_proc_agent {} {
- global objdir
if [target_info exists in_proc_agent] {
return [target_info in_proc_agent]
} else {
- return $objdir/../gdbserver/libinproctrace.so
+ return ../gdbserver/libinproctrace.so
}
}