From patchwork Fri Jan 23 07:13:44 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Evans X-Patchwork-Id: 4773 Received: (qmail 6266 invoked by alias); 23 Jan 2015 07:16:45 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Delivered-To: mailing list gdb-patches@sourceware.org Received: (qmail 5958 invoked by uid 89); 23 Jan 2015 07:16:04 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-2.0 required=5.0 tests=AWL, BAYES_00, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, SPF_PASS autolearn=ham version=3.3.2 X-HELO: mail-pd0-f176.google.com Received: from mail-pd0-f176.google.com (HELO mail-pd0-f176.google.com) (209.85.192.176) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-GCM-SHA256 encrypted) ESMTPS; Fri, 23 Jan 2015 07:14:35 +0000 Received: by mail-pd0-f176.google.com with SMTP id y10so6894891pdj.7 for ; Thu, 22 Jan 2015 23:14:32 -0800 (PST) X-Received: by 10.66.250.166 with SMTP id zd6mr8746647pac.41.1421997272643; Thu, 22 Jan 2015 23:14:32 -0800 (PST) Received: from seba.sebabeach.org.gmail.com (173-13-178-53-sfba.hfc.comcastbusiness.net. [173.13.178.53]) by mx.google.com with ESMTPSA id bk8sm876091pad.28.2015.01.22.23.14.30 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Jan 2015 23:14:31 -0800 (PST) From: Doug Evans To: Yao Qi Cc: gdb-patches Subject: Re: [RFC] Monster testcase generator for performance testsuite References: <87mw5xuzdc.fsf@codesourcery.com> <871tn7udyt.fsf@codesourcery.com> <87k30yt4rp.fsf@codesourcery.com> Date: Thu, 22 Jan 2015 23:13:44 -0800 In-Reply-To: <87k30yt4rp.fsf@codesourcery.com> (Yao Qi's message of "Thu, 8 Jan 2015 09:55:38 +0800") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 X-IsSubscribed: yes Yao Qi writes: > Can we have a perf test case generator without using parallel build? and > we can add building perf test cases in parallel in next step. I'd like > to add new things gradually. > > If you think it isn't necessary to do things in these two steps, I am > OK too. I don't have a strong opinion on this now. I'll take a look at > your patch in details. Cool, thanks. With testcases this big, a parallel build is really a must. I've also added the beginnings of sha1sum tracking so that incremental builds are even faster. This is still a work in progress, but it's at a point where I'm getting good data from it. Still need to add, e.g., an output verifier (data isn't valid unless the correct answer was printed, and manual verification is a pain / error prone). To use, e.g., bash$ make -j5 build-perf RUNTESTFLAGS="gmonster1.exp gmonster2.exp" bash$ make check-perf RUNTESTFLAGS="gdb.perf/gm*-*.exp GDB=/path/to/gdb" diff --git a/gdb/testsuite/Makefile.in b/gdb/testsuite/Makefile.in index 53cb754..c350162 100644 --- a/gdb/testsuite/Makefile.in +++ b/gdb/testsuite/Makefile.in @@ -227,13 +227,31 @@ do-check-parallel: $(TEST_TARGETS) @GMAKE_TRUE@check/%.exp: @GMAKE_TRUE@ -mkdir -p outputs/$* -@GMAKE_TRUE@ @$(DO_RUNTEST) GDB_PARALLEL=yes --outdir=outputs/$* $*.exp $(RUNTESTFLAGS) +@GMAKE_TRUE@ @$(DO_RUNTEST) GDB_PARALLEL=. --outdir=outputs/$* $*.exp $(RUNTESTFLAGS) check/no-matching-tests-found: @echo "" @echo "No matching tests found." @echo "" +@GMAKE_TRUE@pieces/%.exp: +@GMAKE_TRUE@ mkdir -p gdb.perf/outputs/$* +@GMAKE_TRUE@ $(DO_RUNTEST) --status --outdir=gdb.perf/outputs/$* lib/build-piece.exp PIECE=gdb.perf/pieces/$*.exp WORKER=$* GDB_PARALLEL=gdb.perf $(RUNTESTFLAGS) GDB_PERFTEST_MODE=build-pieces + +# GDB_PERFTEST_MODE appears *after* RUNTESTFLAGS here because we don't want +# anything in RUNTESTFLAGS to override it. +# We don't delete previous directories here as these programs can take +# awhile to build, and perftest.exp has support for deciding whether to +# recompile them. If you want to remove these directories, make clean. +@GMAKE_TRUE@build-perf: $(abs_builddir)/site.exp +@GMAKE_TRUE@ mkdir -p gdb.perf/pieces +@GMAKE_TRUE@ @: Step 1: Generate the build .exp files. +@GMAKE_TRUE@ $(DO_RUNTEST) --status --directory=gdb.perf --outdir gdb.perf/pieces GDB_PARALLEL=gdb.perf $(RUNTESTFLAGS) GDB_PERFTEST_MODE=gen-build-exps +@GMAKE_TRUE@ @: Step 2: Compile the pieces. +@GMAKE_TRUE@ $(MAKE) $$(cd gdb.perf && echo pieces/*/*.exp) +@GMAKE_TRUE@ @: Step 3: Do the final link. +@GMAKE_TRUE@ $(DO_RUNTEST) --status --directory=gdb.perf --outdir gdb.perf GDB_PARALLEL=gdb.perf $(RUNTESTFLAGS) GDB_PERFTEST_MODE=compile + check-perf: all $(abs_builddir)/site.exp @if test ! -d gdb.perf; then mkdir gdb.perf; fi $(DO_RUNTEST) --directory=gdb.perf --outdir gdb.perf GDB_PERFTEST_MODE=both $(RUNTESTFLAGS) @@ -245,6 +263,7 @@ clean mostlyclean: -rm -f core.* *.tf *.cl tracecommandsscript copy1.txt zzz-gdbscript -rm -f *.dwo *.dwp -rm -rf outputs temp cache + -rm -rf gdb.perf/pieces gdb.perf/outputs gdb.perf/temp gdb.perf/cache -rm -f read1.so expect-read1 if [ x"${ALL_SUBDIRS}" != x ] ; then \ for dir in ${ALL_SUBDIRS}; \ diff --git a/gdb/testsuite/gdb.perf/gm-hello.cc b/gdb/testsuite/gdb.perf/gm-hello.cc new file mode 100644 index 0000000..05b36e8 --- /dev/null +++ b/gdb/testsuite/gdb.perf/gm-hello.cc @@ -0,0 +1,18 @@ +/* Copyright (C) 2015 Free Software Foundation, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#include + +std::string hello ("Hello."); diff --git a/gdb/testsuite/gdb.perf/gmonster-null-lookup.py b/gdb/testsuite/gdb.perf/gmonster-null-lookup.py new file mode 100644 index 0000000..9bb839e --- /dev/null +++ b/gdb/testsuite/gdb.perf/gmonster-null-lookup.py @@ -0,0 +1,44 @@ +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Test handling of lookup of a symbol that doesn't exist. + +from perftest import perftest +from perftest import measure +from perftest import utils + +class NullLookup(perftest.TestCaseWithBasicMeasurements): + def __init__(self, name, run_names, binfile): + # We want to measure time in this test. + super(NullLookup, self).__init__(name) + self.run_names = run_names + self.binfile = binfile + + def warm_up(self): + pass + + def execute_test(self): + for run in self.run_names: + this_run_binfile = "%s-%s" % (self.binfile, + utils.convert_spaces(run)) + utils.select_file(this_run_binfile) + utils.runto_main() + utils.safe_execute("mt expand-symtabs") + iteration = 5 + while iteration > 0: + utils.safe_execute("mt flush-symbol-cache") + func = lambda: utils.safe_execute("p symbol_not_found") + self.measure.measure(func, run) + iteration -= 1 diff --git a/gdb/testsuite/gdb.perf/gmonster-ptype-string.py b/gdb/testsuite/gdb.perf/gmonster-ptype-string.py new file mode 100644 index 0000000..d39f4ce --- /dev/null +++ b/gdb/testsuite/gdb.perf/gmonster-ptype-string.py @@ -0,0 +1,45 @@ +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Measure speed of ptype of a std::string object. + +from perftest import perftest +from perftest import measure +from perftest import utils + +class GmonsterPtypeString(perftest.TestCaseWithBasicMeasurements): + def __init__(self, name, run_names, binfile): + # We want to measure time in this test. + super(GmonsterPtypeString, self).__init__(name) + self.run_names = run_names + self.binfile = binfile + + def warm_up(self): + pass + + def execute_test(self): + for run in self.run_names: + this_run_binfile = "%s-%s" % (self.binfile, + utils.convert_spaces(run)) + utils.select_file(this_run_binfile) + utils.runto_main() + utils.safe_execute("mt expand-symtabs") + iteration = 5 + while iteration > 0: + utils.safe_execute("mt flush-symbol-cache") + func1 = lambda: utils.safe_execute("ptype hello") + func = lambda: utils.run_n_times(2, func1) + self.measure.measure(func, run) + iteration -= 1 diff --git a/gdb/testsuite/gdb.perf/gmonster1-null-lookup.exp b/gdb/testsuite/gdb.perf/gmonster1-null-lookup.exp new file mode 100644 index 0000000..4600b95 --- /dev/null +++ b/gdb/testsuite/gdb.perf/gmonster1-null-lookup.exp @@ -0,0 +1,44 @@ +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Measure speed of lookup of a symbol that doesn't exist. +# Test parameters are the standard GenPerfTest parameters. + +load_lib perftest.exp + +if [skip_perf_tests] { + return 0 +} + +set testprog "gmonster1" + +GenPerfTest::load_test_description ${testprog}.exp + +# This variable is required by perftest.exp. +# This isn't the name of the test program, it's the name of the .py test. +# The harness assumes they are the same, which is not the case here. +set testfile "gmonster-null-lookup" + +array set testcase [make_testcase_config] + +PerfTest::assemble { + # Compilation is handled by ${testprog}.exp. + return 0 +} { + clean_restart +} { + global testcase + gdb_test "python NullLookup('$testprog:$testfile', [tcl_string_list_to_python_list $testcase(run_names)], '$testcase(binfile)').run()" +} diff --git a/gdb/testsuite/gdb.perf/gmonster1-ptype-string.exp b/gdb/testsuite/gdb.perf/gmonster1-ptype-string.exp new file mode 100644 index 0000000..26327aa --- /dev/null +++ b/gdb/testsuite/gdb.perf/gmonster1-ptype-string.exp @@ -0,0 +1,44 @@ +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Measure speed of ptype on a simple class. +# Test parameters are the standard GenPerfTest parameters. + +load_lib perftest.exp + +if [skip_perf_tests] { + return 0 +} + +set testprog "gmonster1" + +GenPerfTest::load_test_description ${testprog}.exp + +# This variable is required by perftest.exp. +# This isn't the name of the test program, it's the name of the .py test. +# The harness assumes they are the same, which is not the case here. +set testfile "gmonster-ptype-string" + +array set testcase [make_testcase_config] + +PerfTest::assemble { + # Compilation is handled by ${testprog}.exp. + return 0 +} { + clean_restart +} { + global testcase + gdb_test "python GmonsterPtypeString('$testprog:$testfile', [tcl_string_list_to_python_list $testcase(run_names)], '$testcase(binfile)').run()" +} diff --git a/gdb/testsuite/gdb.perf/gmonster1.cc b/gdb/testsuite/gdb.perf/gmonster1.cc new file mode 100644 index 0000000..0627a09 --- /dev/null +++ b/gdb/testsuite/gdb.perf/gmonster1.cc @@ -0,0 +1,20 @@ +/* Copyright (C) 2015 Free Software Foundation, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +int +main () +{ + return 0; +} diff --git a/gdb/testsuite/gdb.perf/gmonster1.exp b/gdb/testsuite/gdb.perf/gmonster1.exp new file mode 100644 index 0000000..fdaa191 --- /dev/null +++ b/gdb/testsuite/gdb.perf/gmonster1.exp @@ -0,0 +1,79 @@ +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Perftest description file for building the "gmonster1" benchmark. +# Where does the name come from? The benchmark is derived from one of the +# monster programs at Google. +# +# Perftest descriptions are loaded thrice: +# 1) To generate the build .exp files +# GDB_PERFTEST_MODE=gen-build-exps +# This step allows for parallel builds of the majority of pieces of the +# test binary and shlibs. +# 2) To compile the "pieces" of the binary and shlibs. +# "Pieces" are the bulk of the machine-generated sources of the test. +# This step is driven by lib/build-piece.exp. +# GDB_PERFTEST_MODE=build-pieces +# 3) To perform the final link of the binary and shlibs. +# GDB_PERFTEST_MODE=compile + +load_lib perftest.exp + +if [skip_perf_tests] { + return 0 +} + +if ![info exists MONSTER] { + set MONSTER "n" +} + +proc make_testcase_config { } { + global MONSTER + + set program_name "gmonster1" + array set testcase [GenPerfTest::init_testcase $program_name] + + set testcase(language) c++ + + # binary_sources needs to be embedded in an outer list because remember + # each element of the outer list is for each run, and here we want to use + # the same value for all runs. + set testcase(binary_sources) { { gmonster1.cc gm-hello.cc } } + + if { $MONSTER == "y" } { + set testcase(run_names) { 10-cus 100-cus 1000-cus 10000-cus } + set testcase(nr_compunits) { 10 100 1000 10000 } + } else { + set testcase(run_names) { 1-cu 10-cus 100-cus } + set testcase(nr_compunits) { 1 10 100 } + } + set testcase(nr_shlibs) { 0 } + + set testcase(nr_extern_functions) 10 + set testcase(nr_static_functions) 10 + + # class_specs needs to be embedded in an outer list because remember + # each element of the outer list is for each run, and here we want to use + # the same value for all runs. + set testcase(class_specs) { { { 0 10 } { 1 10 } { 2 10 } } } + set testcase(nr_members) 10 + set testcase(nr_static_members) 10 + set testcase(nr_methods) 10 + set testcase(nr_static_methods) 10 + + return [array get testcase] +} + +GenPerfTest::standard_driver gmonster1.exp make_testcase_config diff --git a/gdb/testsuite/gdb.perf/gmonster2-null-lookup.exp b/gdb/testsuite/gdb.perf/gmonster2-null-lookup.exp new file mode 100644 index 0000000..c6d3d91 --- /dev/null +++ b/gdb/testsuite/gdb.perf/gmonster2-null-lookup.exp @@ -0,0 +1,44 @@ +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Measure speed of lookup of a symbol that doesn't exist. +# Test parameters are the standard GenPerfTest parameters. + +load_lib perftest.exp + +if [skip_perf_tests] { + return 0 +} + +set testprog "gmonster2" + +GenPerfTest::load_test_description ${testprog}.exp + +# This variable is required by perftest.exp. +# This isn't the name of the test program, it's the name of the .py test. +# The harness assumes they are the same, which is not the case here. +set testfile "gmonster-null-lookup" + +array set testcase [make_testcase_config] + +PerfTest::assemble { + # Compilation is handled by ${testprog}.exp. + return 0 +} { + clean_restart +} { + global testcase + gdb_test "python NullLookup('$testprog:$testfile', [tcl_string_list_to_python_list $testcase(run_names)], '$testcase(binfile)').run()" +} diff --git a/gdb/testsuite/gdb.perf/gmonster2-ptype-string.exp b/gdb/testsuite/gdb.perf/gmonster2-ptype-string.exp new file mode 100644 index 0000000..23fa38d --- /dev/null +++ b/gdb/testsuite/gdb.perf/gmonster2-ptype-string.exp @@ -0,0 +1,44 @@ +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Measure blah with lots of shared libraries +# Test parameters are the standard GenPerfTest parameters. + +load_lib perftest.exp + +if [skip_perf_tests] { + return 0 +} + +set testprog "gmonster2" + +GenPerfTest::load_test_description ${testprog}.exp + +# This variable is required by perftest.exp. +# This isn't the name of the test program, it's the name of the .py test. +# The harness assumes they are the same, which is not the case here. +set testfile "gmonster-ptype-string" + +array set testcase [make_testcase_config] + +PerfTest::assemble { + # Compilation is handled by ${testprog}.exp. + return 0 +} { + clean_restart +} { + global testcase + gdb_test "python GmonsterPtypeString('$testprog:$testfile', [tcl_string_list_to_python_list $testcase(run_names)], '$testcase(binfile)').run()" +} diff --git a/gdb/testsuite/gdb.perf/gmonster2.cc b/gdb/testsuite/gdb.perf/gmonster2.cc new file mode 100644 index 0000000..0627a09 --- /dev/null +++ b/gdb/testsuite/gdb.perf/gmonster2.cc @@ -0,0 +1,20 @@ +/* Copyright (C) 2015 Free Software Foundation, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +int +main () +{ + return 0; +} diff --git a/gdb/testsuite/gdb.perf/gmonster2.exp b/gdb/testsuite/gdb.perf/gmonster2.exp new file mode 100644 index 0000000..6d62876 --- /dev/null +++ b/gdb/testsuite/gdb.perf/gmonster2.exp @@ -0,0 +1,79 @@ +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Perftest description file for building the "gmonster2" benchmark. +# Where does the name come from? The benchmark is derived from one of the +# monster programs at Google. +# +# Perftest descriptions are loaded thrice: +# 1) To generate the build .exp files +# GDB_PERFTEST_MODE=gen-build-exps +# This step allows for parallel builds of the majority of pieces of the +# test binary and shlibs. +# 2) To compile the "pieces" of the binary and shlibs. +# "Pieces" are the bulk of the machine-generated sources of the test. +# This step is driven by lib/build-piece.exp. +# GDB_PERFTEST_MODE=build-pieces +# 3) To perform the final link of the binary and shlibs. +# GDB_PERFTEST_MODE=compile + +load_lib perftest.exp + +if [skip_perf_tests] { + return 0 +} + +if ![info exists MONSTER] { + set MONSTER "n" +} + +proc make_testcase_config { } { + global MONSTER + + set program_name "gmonster2" + array set testcase [GenPerfTest::init_testcase $program_name] + + set testcase(language) c++ + + # binary_sources needs to be embedded in an outer list because remember + # each element of the outer list is for each run, and here we want to use + # the same value for all runs. + set testcase(binary_sources) { { gmonster2.cc gm-hello.cc } } + + if { $MONSTER == "y" } { + set testcase(run_names) { 10-sos 100-sos 1000-sos } + set testcase(nr_shlibs) { 10 100 1000 } + } else { + set testcase(run_names) { 1-so 10-sos 100-sos } + set testcase(nr_shlibs) { 1 10 100 } + } + set testcase(nr_compunits) 10 + + set testcase(nr_extern_functions) 10 + set testcase(nr_static_functions) 10 + + # class_specs needs to be embedded in an outer list because remember + # each element of the outer list is for each run, and here we want to use + # the same value for all runs. + set testcase(class_specs) { { { 0 10 } { 1 10 } { 2 10 } } } + set testcase(nr_members) 10 + set testcase(nr_static_members) 10 + set testcase(nr_methods) 10 + set testcase(nr_static_methods) 10 + + return [array get testcase] +} + +GenPerfTest::standard_driver gmonster2.exp make_testcase_config diff --git a/gdb/testsuite/gdb.perf/lib/perftest/utils.py b/gdb/testsuite/gdb.perf/lib/perftest/utils.py new file mode 100644 index 0000000..ed44500 --- /dev/null +++ b/gdb/testsuite/gdb.perf/lib/perftest/utils.py @@ -0,0 +1,56 @@ +# Copyright (C) 2015 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import gdb + +def safe_execute(command): + """Execute command, ignoring any gdb errors.""" + result = None + try: + result = gdb.execute(command, to_string=True) + except gdb.error: + pass + return result + + +def convert_spaces(file_name): + """Return file_name with all spaces replaced with "-".""" + return file_name.replace(" ", "-") + + +def select_file(file_name): + """Select a file for debugging. + + N.B. This turns confirmation off. + """ + safe_execute("set confirm off") + gdb.execute("file %s" % (file_name)) + + +def runto_main(): + """Run the program to "main". + + N.B. This turns confirmation off. + """ + safe_execute("set confirm off") + gdb.execute("tbreak main") + gdb.execute("run") + + +def run_n_times(count, func): + """Execute func count times.""" + while count > 0: + func() + count -= 1 diff --git a/gdb/testsuite/lib/build-piece.exp b/gdb/testsuite/lib/build-piece.exp new file mode 100644 index 0000000..c48774c --- /dev/null +++ b/gdb/testsuite/lib/build-piece.exp @@ -0,0 +1,36 @@ +# Copyright (C) 2014 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Utility to bootstrap building a piece of a performance test in a +# parallel build. +# See testsuite/Makefile.in:pieces/%.exp. + +# Dejagnu presents a kind of API to .exp files, but using this file to +# bootstrap the parallel build process breaks that. Before invoking $PIECE +# set various globals to their expected values. The tests may not use these +# today, but if/when they do the error modes are confusing, so fix it now. + +# $subdir is set to "lib", because that is where this file lives, +# which is not what tests expect. The makefile sets WORKER for us. +# Its value is /-. +set subdir [file dirname $WORKER] + +# $gdb_test_file_name is set to this file, build-piece, which is not what +# tests expect. This assumes each piece's build .exp file lives in +# $objdir/gdb.perf/pieces/. +# See perftest.exp:GenPerfTest::gen_build_exp_files. +set gdb_test_file_name [file tail [file dirname $PIECE]] + +source $PIECE diff --git a/gdb/testsuite/lib/cache.exp b/gdb/testsuite/lib/cache.exp index 8df04b9..9565b39 100644 --- a/gdb/testsuite/lib/cache.exp +++ b/gdb/testsuite/lib/cache.exp @@ -35,7 +35,7 @@ proc gdb_do_cache {name} { } if {[info exists GDB_PARALLEL]} { - set cache_filename [file join $objdir cache $cache_name] + set cache_filename [file join $objdir $GDB_PARALLEL cache $cache_name] if {[file exists $cache_filename]} { set fd [open $cache_filename] set gdb_data_cache($cache_name) [read -nonewline $fd] diff --git a/gdb/testsuite/lib/gdb.exp b/gdb/testsuite/lib/gdb.exp index a6f200f..4415801 100644 --- a/gdb/testsuite/lib/gdb.exp +++ b/gdb/testsuite/lib/gdb.exp @@ -3777,7 +3777,7 @@ proc standard_output_file {basename} { global objdir subdir gdb_test_file_name GDB_PARALLEL if {[info exists GDB_PARALLEL]} { - set dir [file join $objdir outputs $subdir $gdb_test_file_name] + set dir [file join $objdir $GDB_PARALLEL outputs $subdir $gdb_test_file_name] file mkdir $dir return [file join $dir $basename] } else { @@ -3791,7 +3791,7 @@ proc standard_temp_file {basename} { global objdir GDB_PARALLEL if {[info exists GDB_PARALLEL]} { - return [file join $objdir temp $basename] + return [file join $objdir $GDB_PARALLEL temp $basename] } else { return $basename } @@ -4693,17 +4693,27 @@ proc build_executable { testname executable {sources ""} {options {debug}} } { return [eval build_executable_from_specs $arglist] } -# Starts fresh GDB binary and loads EXECUTABLE into GDB. EXECUTABLE is -# the basename of the binary. -proc clean_restart { executable } { +# Starts fresh GDB binary and loads an optional executable into GDB. +# Usage: clean_restart [executable] +# EXECUTABLE is the basename of the binary. + +proc clean_restart { args } { global srcdir global subdir - set binfile [standard_output_file ${executable}] + + if { [llength $args] > 1 } { + error "bad number of args: [llength $args]" + } gdb_exit gdb_start gdb_reinitialize_dir $srcdir/$subdir - gdb_load ${binfile} + + if { [llength $args] >= 1 } { + set executable [lindex $args 0] + set binfile [standard_output_file ${executable}] + gdb_load ${binfile} + } } # Prepares for testing by calling build_executable_full, then @@ -4907,7 +4917,10 @@ if {[info exists GDB_PARALLEL]} { if {[is_remote host]} { unset GDB_PARALLEL } else { - file mkdir outputs temp cache + file mkdir \ + [file join $GDB_PARALLEL outputs] \ + [file join $GDB_PARALLEL temp] \ + [file join $GDB_PARALLEL cache] } } diff --git a/gdb/testsuite/lib/perftest.exp b/gdb/testsuite/lib/perftest.exp index 7c334ac..1aca310 100644 --- a/gdb/testsuite/lib/perftest.exp +++ b/gdb/testsuite/lib/perftest.exp @@ -12,6 +12,10 @@ # # You should have received a copy of the GNU General Public License # along with this program. If not, see . +# +# Notes: +# 1) This follows a Python convention for marking internal vs public functions. +# Internal functions are prefixed with "_". namespace eval PerfTest { # The name of python file on build. @@ -42,14 +46,7 @@ namespace eval PerfTest { # actual compilation. Return zero if compilation is successful, # otherwise return non-zero. proc compile {body} { - global GDB_PERFTEST_MODE - - if { [info exists GDB_PERFTEST_MODE] - && [string compare $GDB_PERFTEST_MODE "run"] } { - return [uplevel 2 $body] - } - - return 0 + return [uplevel 2 $body] } # Start up GDB. @@ -82,14 +79,24 @@ namespace eval PerfTest { proc assemble {compile startup run} { global GDB_PERFTEST_MODE - if { [eval compile {$compile}] } { - untested "Could not compile source files." + if ![info exists GDB_PERFTEST_MODE] { + return + } + + if { "$GDB_PERFTEST_MODE" == "gen-build-exps" + || "$GDB_PERFTEST_MODE" == "build-pieces" } { return } + if { [string compare $GDB_PERFTEST_MODE "run"] } { + if { [eval compile {$compile}] } { + untested "Could not compile source files." + return + } + } + # Don't execute the run if GDB_PERFTEST_MODE=compile. - if { [info exists GDB_PERFTEST_MODE] - && [string compare $GDB_PERFTEST_MODE "compile"] == 0} { + if { [string compare $GDB_PERFTEST_MODE "compile"] == 0} { return } @@ -110,10 +117,11 @@ proc skip_perf_tests { } { if [info exists GDB_PERFTEST_MODE] { - if { "$GDB_PERFTEST_MODE" != "compile" + if { "$GDB_PERFTEST_MODE" != "gen-build-exps" + && "$GDB_PERFTEST_MODE" != "build-pieces" + && "$GDB_PERFTEST_MODE" != "compile" && "$GDB_PERFTEST_MODE" != "run" && "$GDB_PERFTEST_MODE" != "both" } { - # GDB_PERFTEST_MODE=compile|run|both is allowed. error "Unknown value of GDB_PERFTEST_MODE." return 1 } @@ -123,3 +131,958 @@ proc skip_perf_tests { } { return 1 } + +# Given a list of tcl strings, return the same list as the text form of a +# python list. + +proc tcl_string_list_to_python_list { l } { + proc quote { text } { + return "\"$text\"" + } + set quoted_list "" + foreach elm $l { + lappend quoted_list [quote $elm] + } + return "([join $quoted_list {, }])" +} + +# A simple testcase generator. +# +# Usage Notes: +# +# 1) The length of each parameter list must either be one, in which case the +# same value is used for each run, or the length must match all other +# parameters of length greater than one. +# +# 2) Values for parameters that vary across runs must appear in increasing +# order. E.g. nr_shlibs = { 0 1 10 } is good, { 1 0 10 } is bad. +# This rule simplifies the code a bit, without being onerous on the user: +# a) Report generation doesn't have to sort the output by run, it'll already +# be sorted. +# b) In the static object file case, the last run can be used used to generate +# all the source files. +# +# TODO: +# 1) Lots. E.g., having functions call each other within an objfile and across +# objfiles to measure things like backtrace times. +# 2) Lots. E.g., inline methods. +# +# Implementation Notes: +# +# 1) The implementation would be a bit simpler if we could assume Tcl 8.5. +# Then we could use a dictionary to record the testcase instead of an array. +# With the array we use here, there is only one copy of it and instead of +# passing its value we pass its name. Yay Tcl. +# +# 2) Array members cannot (apparently) be references in the conditional +# expression of a for loop (-> variable not found error). That is why they're +# all extracted before the for loop. + +if ![info exists CAT_PROGRAM] { + set CAT_PROGRAM "/bin/cat" +} + +if ![info exists SHA1SUM_PROGRAM] { + set SHA1SUM_PROGRAM "/usr/bin/sha1sum" +} + +namespace eval GenPerfTest { + + # The default level of compilation parallelism we support. + set DEFAULT_PERF_TEST_COMPILE_PARALLELISM 10 + + # The language of the test. + set DEFAULT_LANGUAGE "c" + + # Extra source files for the binary. + # This must at least include the file with main(), + # each test must supply its own. + set DEFAULT_BINARY_SOURCES {} + + # The number of shared libraries to create. + set DEFAULT_NR_SHLIBS 0 + + # The number of compunits in each objfile. + set DEFAULT_NR_COMPUNITS 1 + + # The number of public globals in each compunit. + set DEFAULT_NR_EXTERN_GLOBALS 1 + + # The number of static globals in each compunit. + set DEFAULT_NR_STATIC_GLOBALS 1 + + # The number of public functions in each compunit. + set DEFAULT_NR_EXTERN_FUNCTIONS 1 + + # The number of static functions in each compunit. + set DEFAULT_NR_STATIC_FUNCTIONS 1 + + # List of pairs of class depth and number of classes at that depth. + # By "depth" here we mean nesting within a namespace. + # E.g., + # class foo {}; + # namespace n { class foo {}; class bar {}; } + # would be represented as { { 0 1 } { 1 2 } }. + # This is only used if the selected language permits it. + set DEFAULT_CLASS_SPECS {} + + # Number of members in each class. + # This is only used if classes are enabled. + set DEFAULT_NR_MEMBERS 0 + + # Number of static members in each class. + # This is only used if classes are enabled. + set DEFAULT_NR_STATIC_MEMBERS 0 + + # Number of methods in each class. + # This is only used if classes are enabled. + set DEFAULT_NR_METHODS 0 + + # Number of static methods in each class. + # This is only used if classes are enabled. + set DEFAULT_NR_STATIC_METHODS 0 + + set suffixes(c) "c" + set suffixes(c++) "cc" + + # Helper function to generate .exp build files. + + proc _gen_build_exp_files { program_name nr_workers output_dir code } { + verbose -log "_gen_build_exp_files: $nr_workers workers" + for { set i 0 } { $i < $nr_workers } { incr i } { + set file_name "$output_dir/${program_name}-${i}.exp" + verbose -log "_gen_build_exp_files: Generating $file_name" + set f [open $file_name "w"] + puts $f "# DO NOT EDIT, machine generated file." + puts $f "# See perftest.exp:GenPerfTest::gen_build_exp_files." + puts $f "" + puts $f "set worker_nr $i" + puts $f "" + puts $f "# The body of the file is supplied by the test." + puts $f "" + puts $f $code + close $f + } + return 0 + } + + # Generate .exp files to build all the "pieces" of the testcase. + # This doesn't include "main" or any test-specific stuff. + # This mostly consists of the "bulk" (aka "crap" :-)) of the testcase to + # give gdb something meaty to chew on. + # The result is 0 for success, -1 for failure. + # + # Benchmarks generated by some of the tests are big. I mean really big. + # And it's a pain to build one piece at a time, we need a parallel build. + # To achieve this, given the framework we're working with, we generate + # several .exp files, and then let testsuite/Makefile.in's support for + # parallel runs of the testsuite to do its thing. + + proc gen_build_exp_files { test_description_exp make_config_thunk_name } { + global objdir PERF_TEST_COMPILE_PARALLELISM + + if { [file tail $test_description_exp] != $test_description_exp } { + error "test description file contains directory name" + } + + set program_name [file rootname $test_description_exp] + + set output_dir "$objdir/gdb.perf/pieces/$program_name" + file mkdir $output_dir + + # N.B. The generation code below cannot reference anything that exists + # here, the code isn't run until later, in another process. That is + # why we split up the assignment to $code. + # TODO(dje): Not the cleanest way, but simple enough for now. + set code { + # This code is put in each copy of the generated .exp file. + + load_lib perftest.exp + + GenPerfTest::load_test_description} + append code " $test_description_exp" + append code { + + array set testcase [} + append code "$make_config_thunk_name" + append code {] + + if { [GenPerfTest::compile_pieces testcase $worker_nr] < 0 } { + return -1 + } + + return 0 + } + + return [_gen_build_exp_files $program_name $PERF_TEST_COMPILE_PARALLELISM $output_dir $code] + } + + # Load a perftest description. + # Test descriptions are used to build the input files (binary + shlibs) + # of one or more performance tests. + + proc load_test_description { basename } { + global srcdir + + if { [file tail $basename] != $basename } { + error "test description file contains directory name" + } + + verbose -log "load_file $srcdir/gdb.perf/$basename" + if { [load_file $srcdir/gdb.perf/$basename] == 0 } { + error "Unable to load test description $basename" + } + } + + # Create a testcase object for test NAME. + # The caller must call this as: + # array set my_test [GenPerfTest::init_testcase $name] + + proc init_testcase { name } { + set testcase(name) $name + set testcase(language) $GenPerfTest::DEFAULT_LANGUAGE + set testcase(run_names) [list $name] + set testcase(binary_sources) $GenPerfTest::DEFAULT_BINARY_SOURCES + set testcase(nr_shlibs) $GenPerfTest::DEFAULT_NR_SHLIBS + set testcase(nr_compunits) $GenPerfTest::DEFAULT_NR_COMPUNITS + + set testcase(nr_extern_globals) $GenPerfTest::DEFAULT_NR_EXTERN_GLOBALS + set testcase(nr_static_globals) $GenPerfTest::DEFAULT_NR_STATIC_GLOBALS + set testcase(nr_extern_functions) $GenPerfTest::DEFAULT_NR_EXTERN_FUNCTIONS + set testcase(nr_static_functions) $GenPerfTest::DEFAULT_NR_STATIC_FUNCTIONS + + set testcase(class_specs) $GenPerfTest::DEFAULT_CLASS_SPECS + set testcase(nr_members) $GenPerfTest::DEFAULT_NR_MEMBERS + set testcase(nr_static_members) $GenPerfTest::DEFAULT_NR_STATIC_MEMBERS + set testcase(nr_methods) $GenPerfTest::DEFAULT_NR_METHODS + set testcase(nr_static_methods) $GenPerfTest::DEFAULT_NR_STATIC_METHODS + + # The location of this file drives the location of all other files. + # The choice is derived from standard_output_file. We don't use it + # because of the parallel build support, we want each worker's log/sum + # files to go in different directories, but we don't want their output + # to go in different directories. + # N.B. The value here must be kept in sync with Makefile.in. + global objdir + set name_no_spaces [_convert_spaces $name] + set testcase(binfile) "$objdir/gdb.perf/outputs/$name_no_spaces/$name_no_spaces" + + return [array get testcase] + } + + proc _verify_parameter_lengths { self_var } { + upvar 1 $self_var self + set params { + binary_sources + nr_shlibs nr_compunits + nr_extern_globals nr_static_globals + nr_extern_functions nr_static_functions + class_specs + nr_members nr_static_members + nr_methods nr_static_methods + } + set nr_runs [llength $self(run_names)] + foreach p $params { + set n [llength $self($p)] + if { $n > 1 } { + if { $n != $nr_runs } { + error "Bad number of values for parameter $p" + } + set values $self($p) + for { set i 0 } { $i < $n - 1 } { incr i } { + if { [lindex $values $i] > [lindex $values [expr $i + 1]] } { + error "Values of parameter $p are not increasing" + } + } + } + } + } + + # Verify the testcase is valid (as best we can, this isn't exhaustive). + + proc _verify_testcase { self_var } { + upvar 1 $self_var self + _verify_parameter_lengths self + + # Each test must supply its own main(). We don't check for main here, + # but we do verify the test supplied something. + if { [llength $self(binary_sources)] == 0 } { + error "Missing value for binary_sources" + } + } + + # Return the value of parameter PARAM for run RUN_NR. + + proc _get_param { param run_nr } { + if { [llength $param] == 1 } { + # Since PARAM may be a list of lists we need to use lindex. This + # also works for scalars (scalars are degenerate lists). + return [lindex $param 0] + } + return [lindex $param $run_nr] + } + + # Return non-zero if all files (binaries + shlibs) can be compiled from + # one set of object files. This is a simple optimization to speed up + # test build times. This happens if the only variation among runs is + # nr_shlibs or nr_compunits. + + proc _static_object_files_p { self_var } { + upvar 1 $self_var self + # These values are either scalars, or can vary across runs but don't + # affect whether we can share the generated object objects between + # runs. + set static_object_file_params { + name language run_names nr_shlibs nr_compunits binary_sources + } + foreach name [array names self] { + if { [lsearch $static_object_file_params $name] < 0 } { + # name is not in static_object_file_params. + if { [llength $self($name)] > 1 } { + # The user could provide a list that is all the same value, + # so check for that. + set first_value [lindex $self($name) 0] + foreach elm [lrange $self($name) 1 end] { + if { $elm != $first_value } { + return 0 + } + } + } + } + } + return 1 + } + + # Return non-zero if classes are enabled. + + proc _classes_enabled_p { self_var run_nr } { + upvar 1 $self_var self + set class_specs [_get_param $self(class_specs) $run_nr] + foreach elm $class_specs { + if { [llength $elm] != 2 } { + error "Bad class spec: $elm" + } + if { [lindex $elm 1] > 0 } { + return 1 + } + } + return 0 + } + + # Spaces in file names are a pain, remove them. + # They appear if the user puts spaces in the test name or run name. + + proc _convert_spaces { file_name } { + return [regsub -all " " $file_name "-"] + } + + # Return the compilation flags for the test. + + proc _compile_flags { self_var } { + upvar 1 $self_var self + set result {debug} + switch $self(language) { + c++ { + lappend result "c++" + } + } + return $result + } + + # Return the path to put source/object files in for run number RUN_NR. + + proc _make_object_dir_name { self_var static run_nr } { + upvar 1 $self_var self + # Note: The output directory already includes the name of the test + # description file. + set bindir [file dirname $self(binfile)] + # Put the pieces in a subdirectory, there are a lot of them. + if $static { + return "$bindir/pieces" + } else { + set run_name [_convert_spaces [lindex $self(run_names) $run_nr]] + return "$bindir/pieces/$run_name" + } + } + + # CU_NR is either the compilation unit number or "main". + # RUN_NR is ignored if STATIC is non-zero. + + proc _make_binary_source_name { self_var static run_nr cu_nr } { + upvar 1 $self_var self + set source_suffix $GenPerfTest::suffixes($self(language)) + if { !$static } { + set run_name [_get_param $self(run_names) $run_nr] + set source_name "${run_name}-${cu_nr}.$source_suffix" + } else { + set source_name "$self(name)-${cu_nr}.$source_suffix" + } + return "[_make_object_dir_name self $static $run_nr]/[_convert_spaces $source_name]" + } + + # Generated object files get put in the same directory as their source. + + proc _make_binary_object_name { self_var static run_nr cu_nr } { + upvar 1 $self_var self + set source_name [_make_binary_source_name self $static $run_nr $cu_nr] + return [file rootname $source_name].o + } + + proc _make_shlib_source_name { self_var static run_nr so_nr cu_nr } { + upvar 1 $self_var self + set source_suffix $GenPerfTest::suffixes($self(language)) + if { !$static } { + set run_name [_get_param $self(run_names) $run_nr] + set source_name "$self(name)-${run_name}-lib${so_nr}-${cu_nr}.$source_suffix" + } else { + set source_name "$self(name)-lib${so_nr}-${cu_nr}.$source_suffix" + } + return "[_make_object_dir_name self $static $run_nr]/[_convert_spaces $source_name]" + } + + # Return the list of source/object files for the binary. + # This is the source files specified in test param binary_sources as well + # as the names of all the object file "pieces". + # STATIC is the value of _static_object_files_p for the test. + + proc _make_binary_input_file_names { self_var static run_nr } { + upvar 1 $self_var self + global srcdir subdir + set nr_compunits [_get_param $self(nr_compunits) $run_nr] + set result {} + foreach source [_get_param $self(binary_sources) $run_nr] { + lappend result "$srcdir/$subdir/$source" + } + for { set cu_nr 0 } { $cu_nr < $nr_compunits } { incr cu_nr } { + lappend result [_make_binary_object_name self $static $run_nr $cu_nr] + } + return $result + } + + proc _make_binary_name { self_var run_nr } { + upvar 1 $self_var self + set run_name [_get_param $self(run_names) $run_nr] + set exe_name "$self(binfile)-[_convert_spaces ${run_name}]" + return $exe_name + } + + proc _make_shlib_name { self_var static run_nr so_nr } { + upvar 1 $self_var self + if { !$static } { + set run_name [_get_param $self(run_names) $run_nr] + set lib_name "$self(name)-${run_name}-lib${so_nr}" + } else { + set lib_name "$self(name)-lib${so_nr}" + } + set output_dir [file dirname $self(binfile)] + return "[_make_object_dir_name self $static $run_nr]/[_convert_spaces $lib_name]" + } + + proc _create_file { self_var path } { + upvar 1 $self_var self + verbose -log "Creating file: $path" + set f [open $path "w"] + return $f + } + + proc _write_header { self_var f } { + upvar 1 $self_var self + puts $f "// DO NOT EDIT, machine generated file." + puts $f "// See perftest.exp:GenPerfTest." + } + + proc _write_static_globals { self_var f run_nr } { + upvar 1 $self_var self + puts $f "" + set nr_static_globals [_get_param $self(nr_static_globals) $run_nr] + # Rather than parameterize the number of const/non-const globals, + # and their types, we keep it simple for now. Even the number of + # bss/non-bss globals may be useful; later, if warranted. + for { set i 0 } { $i < $nr_static_globals } { incr i } { + if { $i % 2 == 0 } { + set const "const " + } else { + set const "" + } + puts $f "static ${const}int static_global_$i = $i;" + } + } + + # ID is "" for the binary, and a unique symbol prefix for each SO. + + proc _write_extern_globals { self_var f run_nr id cu_nr } { + upvar 1 $self_var self + puts $f "" + set nr_extern_globals [_get_param $self(nr_extern_globals) $run_nr] + # Rather than parameterize the number of const/non-const globals, + # and their types, we keep it simple for now. Even the number of + # bss/non-bss globals may be useful; later, if warranted. + for { set i 0 } { $i < $nr_extern_globals } { incr i } { + if { $i % 2 == 0 } { + set const "const " + } else { + set const "" + } + puts $f "${const}int ${id}global_${cu_nr}_$i = $cu_nr * 1000 + $i;" + } + } + + proc _write_static_functions { self_var f run_nr } { + upvar 1 $self_var self + set nr_static_functions [_get_param $self(nr_static_functions) $run_nr] + for { set i 0 } { $i < $nr_static_functions } { incr i } { + puts $f "" + puts $f "static void" + puts $f "static_function_$i (void)" + puts $f "{" + puts $f "}" + } + } + + # ID is "" for the binary, and a unique symbol prefix for each SO. + + proc _write_extern_functions { self_var f run_nr id cu_nr } { + upvar 1 $self_var self + set nr_extern_functions [_get_param $self(nr_extern_functions) $run_nr] + for { set i 0 } { $i < $nr_extern_functions } { incr i } { + puts $f "" + puts $f "void" + puts $f "${id}function_${cu_nr}_$i (void)" + puts $f "{" + puts $f "}" + } + } + + proc _write_classes { self_var f run_nr cu_nr } { + upvar 1 $self_var self + set class_specs [_get_param $self(class_specs) $run_nr] + set nr_members [_get_param $self(nr_members) $run_nr] + set nr_static_members [_get_param $self(nr_static_members) $run_nr] + set nr_methods [_get_param $self(nr_methods) $run_nr] + set nr_static_methods [_get_param $self(nr_static_methods) $run_nr] + foreach spec $class_specs { + set depth [lindex $spec 0] + set nr_classes [lindex $spec 1] + puts $f "" + for { set i 0 } { $i < $depth } { incr i } { + puts $f "namespace ns_${i}" + puts $f "\{" + } + for { set c 0 } { $c < $nr_classes } { incr c } { + set class_name "class_${cu_nr}_${c}" + puts $f "class $class_name" + puts $f "\{" + puts $f " public:" + for { set i 0 } { $i < $nr_members } { incr i } { + puts $f " int member_$i;" + } + for { set i 0 } { $i < $nr_static_members } { incr i } { + # Rather than parameterize the number of const/non-const + # members, and their types, we keep it simple for now. + if { $i % 2 == 0 } { + puts $f " static const int static_member_$i = $i;" + } else { + puts $f " static int static_member_$i;" + } + } + for { set i 0 } { $i < $nr_methods } { incr i } { + puts $f " void method_$i (void);" + } + for { set i 0 } { $i < $nr_static_methods } { incr i } { + puts $f " static void static_method_$i (void);" + } + puts $f "\};" + _write_static_members self $f $run_nr $class_name + _write_methods self $f $run_nr $class_name + _write_static_methods self $f $run_nr $class_name + } + for { set i 0 } { $i < $depth } { incr i } { + puts $f "\}" + } + } + } + + proc _write_static_members { self_var f run_nr class_name } { + upvar 1 $self_var self + puts $f "" + set nr_static_members [_get_param $self(nr_static_members) $run_nr] + # Rather than parameterize the number of const/non-const + # members, and their types, we keep it simple for now. + for { set i 0 } { $i < $nr_static_members } { incr i } { + if { $i % 2 == 0 } { + # Static const members are initialized inline. + } else { + puts $f "int ${class_name}::static_member_$i = $i;" + } + } + } + + proc _write_methods { self_var f run_nr class_name } { + upvar 1 $self_var self + set nr_methods [_get_param $self(nr_methods) $run_nr] + for { set i 0 } { $i < $nr_methods } { incr i } { + puts $f "" + puts $f "void" + puts $f "${class_name}::method_$i (void)" + puts $f "{" + puts $f "}" + } + } + + proc _write_static_methods { self_var f run_nr class_name } { + upvar 1 $self_var self + set nr_static_methods [_get_param $self(nr_static_methods) $run_nr] + for { set i 0 } { $i < $nr_static_methods } { incr i } { + puts $f "" + puts $f "void" + puts $f "${class_name}::static_method_$i (void)" + puts $f "{" + puts $f "}" + } + } + + proc _gen_binary_compunit_source { self_var static run_nr cu_nr } { + upvar 1 $self_var self + set source_file [_make_binary_source_name self $static $run_nr $cu_nr] + set f [_create_file self $source_file] + _write_header self $f + _write_static_globals self $f $run_nr + _write_extern_globals self $f $run_nr "" $cu_nr + _write_static_functions self $f $run_nr + _write_extern_functions self $f $run_nr "" $cu_nr + if [_classes_enabled_p self $run_nr] { + _write_classes self $f $run_nr $cu_nr + } + close $f + return $source_file + } + + # Generate the sources for the pieces of the binary. + # The result is a list of source file names and accompanying object file + # names. The pieces are split across workers. + # E.g., with 10 workers the result for worker 0 is + # { { source0 object0 } { source10 object10 } ... } + + proc _gen_binary_source { self_var worker_nr static run_nr } { + upvar 1 $self_var self + verbose -log "GenPerfTest::_gen_binary_source worker $worker_nr run $run_nr, started [timestamp -format %c]" + set nr_compunits [_get_param $self(nr_compunits) $run_nr] + global PERF_TEST_COMPILE_PARALLELISM + set nr_workers $PERF_TEST_COMPILE_PARALLELISM + set result {} + for { set cu_nr $worker_nr } { $cu_nr < $nr_compunits } { incr cu_nr $nr_workers } { + set source_file [_gen_binary_compunit_source self $static $run_nr $cu_nr] + set object_file [_make_binary_object_name self $static $run_nr $cu_nr] + lappend result [list $source_file $object_file] + } + verbose -log "GenPerfTest::_gen_binary_source worker $worker_nr run $run_nr, done [timestamp -format %c]" + return $result + } + + proc _gen_shlib_compunit_source { self_var static run_nr so_nr cu_nr } { + upvar 1 $self_var self + set source_file [_make_shlib_source_name self $static $run_nr $so_nr $cu_nr] + set f [_create_file self $source_file] + _write_header self $f + _write_static_globals self $f $run_nr + _write_extern_globals self $f $run_nr "shlib${so_nr}_" $cu_nr + _write_static_functions self $f $run_nr + _write_extern_functions self $f $run_nr "shlib${so_nr}_" $cu_nr + if [_classes_enabled_p self $run_nr] { + _write_classes self $f $run_nr $cu_nr + } + close $f + return $source_file + } + + proc _gen_shlib_source { self_var static run_nr so_nr } { + upvar 1 $self_var self + verbose -log "GenPerfTest::_gen_shlib_source run $run_nr so $so_nr, started [timestamp -format %c]" + set result "" + set nr_compunits [_get_param $self(nr_compunits) $run_nr] + for { set cu_nr 0 } { $cu_nr < $nr_compunits } { incr cu_nr } { + lappend result [_gen_shlib_compunit_source self $static $run_nr $so_nr $cu_nr] + } + verbose -log "GenPerfTest::_gen_shlib_source run $run_nr so $so_nr, done [timestamp -format %c]" + return $result + } + + # Write all sidebad non-file inputs, as well as OPTIONS to INPUTS_FILE. + + proc _write_inputs_file { inputs_file options } { + global env + set f [open $inputs_file "w"] + puts $f "options: $options" + puts $f "PATH: [getenv PATH]" + close $f + } + + # Generate the sha1sum of all the inputs. + # The result is a list of { error_code text }. + # Upon success error_code is zero and text is the sha1sum. + # Otherwise, error_code is non_zero and text is the error message. + + proc _gen_sha1sum_for_inputs { source inputs } { + global CAT_PROGRAM SHA1SUM_PROGRAM + set catch_result [catch "exec $CAT_PROGRAM $source $inputs | $SHA1SUM_PROGRAM" output] + return [list $catch_result $output] + } + + # Return the contents of TEXT_FILE. + # It is assumed TEXT_FILE exists and is readable. + # This is used for reading files containing sha1sums, the + # last newline is removed. + + proc _read_file { text_file } { + set f [open $text_file "r"] + set result [read -nonewline $f] + close $f + return $result + } + + # Write TEXT to TEXT_FILE. + # It is assumed TEXT_FILE can be opened/created and written to. + + proc _write_file { text_file text } { + set f [open $text_file "w"] + puts $f $text + close $f + } + + # Wrapper on gdb_compile* that computes sha1sums of inputs to decide + # whether the compile is needed. + # The result is the result of gdb_compile*: "" == success, otherwise + # a compilation error occurred and the output is an error message. + # This doesn't take all inputs into account, just the useful ones. + # As an extension (or simplification) on gdb_compile*, if TYPE is + # shlib then call gdb_compile_shlib, otherwise call gdb_compile. + # Other possibilities *could* be handled this way, e.g., pthreads. TBD. + + proc _perftest_compile { source dest type options } { + verbose -log "_perftest_compile $source $dest $type $options" + # To keep things simple, we put all non-file inputs into a file and + # then cat all input files through sha1sum. + set sha1sum_file ${dest}.sha1sum + set sha1new_file ${dest}.sha1new + set inputs_file ${dest}.inputs + _write_inputs_file $inputs_file $options + set sha1sum [_gen_sha1sum_for_inputs $source $inputs_file] + if { [lindex $sha1sum 0] != 0 } { + return "sha1sum generation error: [lindex $sha1sum 1]" + } + set sha1sum [lindex $sha1sum 1] + if [file exists $sha1sum_file] { + set last_sha1sum [_read_file $sha1sum_file] + verbose -log "last: $last_sha1sum, new: $sha1sum" + if { $sha1sum == $last_sha1sum } { + verbose -log "using existing build for $dest" + return "" + } + } + # No such luck, we need to compile. + file delete $sha1sum_file + if { $type == "shlib" } { + set result [gdb_compile_shlib $source $dest $options] + } else { + set result [gdb_compile $source $dest $type $options] + } + if { $result == "" } { + verbose -log "wrote sha1sum: $sha1sum" + _write_file $sha1sum_file $sha1sum + } + return $result + } + + proc _compile_binary_pieces { self_var worker_nr static run_nr } { + upvar 1 $self_var self + set compile_flags [_compile_flags self] + set nr_compunits [_get_param $self(nr_compunits) $run_nr] + global PERF_TEST_COMPILE_PARALLELISM + set nr_workers $PERF_TEST_COMPILE_PARALLELISM + # Generate the source first so we can more easily measure how long that + # takes. [It doesn't take hardly any time at all, relative to the time + # it takes to compile it, but this will provide numbers to show that.] + set todo_list [_gen_binary_source self $worker_nr $static $run_nr] + verbose -log "GenPerfTest::_compile_binary_pieces worker $worker_nr run $run_nr, started [timestamp -format %c]" + foreach elm $todo_list { + set source_file [lindex $elm 0] + set object_file [lindex $elm 1] + if { [_perftest_compile $source_file $object_file object $compile_flags] != "" } { + verbose -log "GenPerfTest::_compile_binary_pieces worker $worker_nr run $run_nr, failed [timestamp -format %c]" + return -1 + } + } + verbose -log "GenPerfTest::_compile_binary_pieces worker $worker_nr run $run_nr, done [timestamp -format %c]" + return 0 + } + + # Helper function to compile the pieces of a shlib. + # Note: gdb_compile_shlib{,_pthreads} don't support first building object + # files and then building the shlib. Therefore our hands are tied, and we + # just build the shlib in one step. This is less of a parallelization + # problem if there are multiple shlibs: Each worker can build a different + # shlib. If this proves to be a problem in practice we can enhance + # gdb_compile_shlib* then. + + proc _compile_shlib { self_var static run_nr so_nr } { + upvar 1 $self_var self + set source_files [_gen_shlib_source self $static $run_nr $so_nr] + set shlib_file [_make_shlib_name self $static $run_nr $so_nr] + set compile_flags [_compile_flags self] + if { [_perftest_compile $source_files $shlib_file shlib $compile_flags] != "" } { + return -1 + } + return 0 + } + + # Compile the pieces of the binary and possible shlibs for the test. + # The result is 0 for success, -1 for failure. + + proc _compile_pieces { self_var worker_nr } { + upvar 1 $self_var self + global PERF_TEST_COMPILE_PARALLELISM + set nr_workers $PERF_TEST_COMPILE_PARALLELISM + set nr_runs [llength $self(run_names)] + set static [_static_object_files_p self] + verbose -log "_compile_pieces: static flag: $static" + file mkdir "[file dirname $self(binfile)]/pieces" + if $static { + # All the pieces look the same (run over run) so just build all the + # shlibs of the last run (which is the largest). + set last_run [expr $nr_runs - 1] + set nr_shlibs [_get_param $self(nr_shlibs) $last_run] + set object_dir [_make_object_dir_name self $static ignored] + file mkdir $object_dir + for { set so_nr $worker_nr } { $so_nr < $nr_shlibs } { incr so_nr $nr_workers } { + if { [_compile_shlib self $static $last_run $so_nr] < 0 } { + return -1 + } + } + if { [_compile_binary_pieces self $worker_nr $static $last_run] < 0 } { + return -1 + } + } else { + for { set run_nr 0 } { $run_nr < $nr_runs } { incr run_nr } { + set nr_shlibs [_get_param $self(nr_shlibs) $run_nr] + set object_dir [_make_object_dir_name self $static $run_nr] + file mkdir $object_dir + for { set so_nr $worker_nr } { $so_nr < $nr_shlibs } { incr so_nr $nr_workers } { + if { [_compile_shlib self $static $run_nr $so_nr] < 0 } { + return -1 + } + } + if { [_compile_binary_pieces self $worker_nr $static $run_nr] < 0 } { + return -1 + } + } + } + return 0 + } + + proc compile_pieces { self_var worker_nr } { + upvar 1 $self_var self + verbose -log "GenPerfTest::compile_pieces worker $worker_nr, started [timestamp -format %c]" + verbose -log "self: [array get self]" + _verify_testcase self + if { [_compile_pieces self $worker_nr] < 0 } { + verbose -log "GenPerfTest::compile_pieces worker $worker_nr, failed [timestamp -format %c]" + return -1 + } + verbose -log "GenPerfTest::compile_pieces worker $worker_nr, done [timestamp -format %c]" + return 0 + } + + proc _make_shlib_flags { self_var static run_nr } { + upvar 1 $self_var self + set nr_shlibs [_get_param $self(nr_shlibs) $run_nr] + set result "" + for { set i 0 } { $i < $nr_shlibs } { incr i } { + lappend result "shlib=[_make_shlib_name self $static $run_nr $i]" + } + return $result + } + + proc _compile_binary { self_var static run_nr } { + upvar 1 $self_var self + set input_files [_make_binary_input_file_names self $static $run_nr] + set binary_file [_make_binary_name self $run_nr] + set compile_flags [_compile_flags self] + set shlib_flags [_make_shlib_flags self $static $run_nr] + if { $shlib_flags != "" } { + set compile_flags "$compile_flags $shlib_flags" + } + if { [_perftest_compile $input_files $binary_file executable $compile_flags] != "" } { + return -1 + } + return 0 + } + + # Helper function for compile. + # The result is 0 for success, -1 for failure. + + proc _compile { self_var } { + upvar 1 $self_var self + set nr_runs [llength $self(run_names)] + set static [_static_object_files_p self] + verbose -log "_compile: static flag: $static" + for { set run_nr 0 } { $run_nr < $nr_runs } { incr run_nr } { + if { [_compile_binary self $static $run_nr] < 0 } { + return -1 + } + } + return 0 + } + + # Main entry point to compile the test program. + # It is assumed all the pieces of the binary (all the .o's, except those + # from test-supplied sources) have already been built with compile_pieces. + # There's no need to compile any shlibs here, as compile_pieces will have + # already built them too. + # The result is 0 for success, -1 for failure. + + proc compile { self_var } { + upvar 1 $self_var self + verbose -log "GenPerfTest::compile, started [timestamp -format %c]" + verbose -log "self: [array get self]" + _verify_testcase self + if { [_compile self] < 0 } { + verbose -log "GenPerfTest::compile, failed [timestamp -format %c]" + return -1 + } + verbose -log "GenPerfTest::compile, done [timestamp -format %c]" + return 0 + } + + proc standard_driver { exp_file_name make_config_thunk_name } { + global GDB_PERFTEST_MODE + switch $GDB_PERFTEST_MODE { + gen-build-exps { + if { [GenPerfTest::gen_build_exp_files $exp_file_name \ + $make_config_thunk_name] < 0 } { + return -1 + } + } + build-pieces { + ;# Nothing to do. + } + compile { + array set testcase [$make_config_thunk_name] + if { [GenPerfTest::compile testcase] < 0 } { + return -1 + } + } + run { + ;# Nothing to do. + } + both { + ;# Don't do anything here. Tests that use us must have + ;# explicitly separate compile/run steps. + } + } + return 0 + } +} + +if ![info exists PERF_TEST_COMPILE_PARALLELISM] { + set PERF_TEST_COMPILE_PARALLELISM $GenPerfTest::DEFAULT_PERF_TEST_COMPILE_PARALLELISM +}