diff mbox

Add test that exercises all bfd architecture, osabi, endian, etc. combinations

Message ID bb875ba8-2a1c-2446-da02-e4158bf048a4@redhat.com
State New, archived
Headers

Commit Message

Pedro Alves Dec. 9, 2016, 2:32 p.m. UTC 
  On 12/09/2016 01:56 PM, Yao Qi wrote:
> On Fri, Dec 9, 2016 at 1:11 PM, Pedro Alves <palves@redhat.com> wrote:
>> (long time passed...)
>>
>> This test would have caught the rl78 and rx problems that Yao
>> fixed today.  Clearly I should have pushed it in earlier so we would
>> have caught those regressions earlier.  :-/
>> The only reason I didn't, was that minute mentioned above.
>> I'm playing with splitting this test in 4 files, to bring that down
>> in a parallel run.  I'll repost with that.
>>

Below's what I had in mind.  Splitting in 4 brings the time down to
~30 seconds for me, while splitting in 8 brings it to ~25 seconds.
Looks like we hit diminishing returns, so I left it at 8.

I also added kfail/skips for rl78 and rx, otherwise the test
crashes GDB...  Those can be removed as soon as your patches
are in (I hope, I haven't tested whether the archs have
further problems that would be exposed by this patch).

WDYT?

> 
> Never mind.  It is still great to have it, and very appreciate you
> write a such great test case!

Thanks!

> 
> I'll extend all-architectures.exp to have a test "disassemble 0x0,+4"
> for PR 20939.  GDB now aborts due to the "foreign frame" again.

That seems more dependent on host architecture than
target architecture, I think?  I.e., cycling over
target architectures and disassembling won't really add
more coverage?

From a71b3ff786ea63df7ad82a02ca59e362a58d6fb6 Mon Sep 17 00:00:00 2001
From: Pedro Alves <palves@redhat.com>
Date: Fri, 9 Dec 2016 14:16:40 +0000
Subject: [PATCH] split

---
 gdb/testsuite/gdb.base/all-architectures-0.exp     | 17 +++++++++++
 gdb/testsuite/gdb.base/all-architectures-1.exp     | 17 +++++++++++
 gdb/testsuite/gdb.base/all-architectures-2.exp     | 17 +++++++++++
 gdb/testsuite/gdb.base/all-architectures-3.exp     | 17 +++++++++++
 gdb/testsuite/gdb.base/all-architectures-4.exp     | 17 +++++++++++
 gdb/testsuite/gdb.base/all-architectures-5.exp     | 17 +++++++++++
 gdb/testsuite/gdb.base/all-architectures-6.exp     | 17 +++++++++++
 gdb/testsuite/gdb.base/all-architectures-7.exp     | 17 +++++++++++
 ...-architectures.exp => all-architectures.exp.in} | 34 ++++++++++++++++++++++
 9 files changed, 170 insertions(+)
 create mode 100644 gdb/testsuite/gdb.base/all-architectures-0.exp
 create mode 100644 gdb/testsuite/gdb.base/all-architectures-1.exp
 create mode 100644 gdb/testsuite/gdb.base/all-architectures-2.exp
 create mode 100644 gdb/testsuite/gdb.base/all-architectures-3.exp
 create mode 100644 gdb/testsuite/gdb.base/all-architectures-4.exp
 create mode 100644 gdb/testsuite/gdb.base/all-architectures-5.exp
 create mode 100644 gdb/testsuite/gdb.base/all-architectures-6.exp
 create mode 100644 gdb/testsuite/gdb.base/all-architectures-7.exp
 rename gdb/testsuite/gdb.base/{all-architectures.exp => all-architectures.exp.in} (91%)

Comments

Yao Qi Dec. 9, 2016, 2:56 p.m. UTC | #1 
On Fri, Dec 9, 2016 at 2:32 PM, Pedro Alves <palves@redhat.com> wrote:
> On 12/09/2016 01:56 PM, Yao Qi wrote:
>> On Fri, Dec 9, 2016 at 1:11 PM, Pedro Alves <palves@redhat.com> wrote:
>>> (long time passed...)
>>>
>>> This test would have caught the rl78 and rx problems that Yao
>>> fixed today.  Clearly I should have pushed it in earlier so we would
>>> have caught those regressions earlier.  :-/
>>> The only reason I didn't, was that minute mentioned above.
>>> I'm playing with splitting this test in 4 files, to bring that down
>>> in a parallel run.  I'll repost with that.
>>>
>
> Below's what I had in mind.  Splitting in 4 brings the time down to
> ~30 seconds for me, while splitting in 8 brings it to ~25 seconds.
> Looks like we hit diminishing returns, so I left it at 8.
>
> I also added kfail/skips for rl78 and rx, otherwise the test
> crashes GDB...  Those can be removed as soon as your patches
> are in (I hope, I haven't tested whether the archs have
> further problems that would be exposed by this patch).
>
> WDYT?
>

That looks good to me.  After your test is pushed in, I'll push my
rl78 and rx patches in, and remove the kfail from this test.  My
rl78 and rx patches should go to 7.12 branch, but I am not sure
this test case should go to 7.12 or not.

>>
>> I'll extend all-architectures.exp to have a test "disassemble 0x0,+4"
>> for PR 20939.  GDB now aborts due to the "foreign frame" again.
>
> That seems more dependent on host architecture than
> target architecture, I think?  I.e., cycling over
> target architectures and disassembling won't really add
> more coverage?

Yes, PR 20939 is about host arch, but such test does find other issues,
like PR 20955.  Other issues are shown up after the rl78 and rx segment
fault is fixed.
Pedro Alves Dec. 9, 2016, 2:58 p.m. UTC | #2 
On 12/09/2016 02:56 PM, Yao Qi wrote:
> On Fri, Dec 9, 2016 at 2:32 PM, Pedro Alves <palves@redhat.com> wrote:
>> On 12/09/2016 01:56 PM, Yao Qi wrote:
>>> On Fri, Dec 9, 2016 at 1:11 PM, Pedro Alves <palves@redhat.com> wrote:
>>>> (long time passed...)
>>>>
>>>> This test would have caught the rl78 and rx problems that Yao
>>>> fixed today.  Clearly I should have pushed it in earlier so we would
>>>> have caught those regressions earlier.  :-/
>>>> The only reason I didn't, was that minute mentioned above.
>>>> I'm playing with splitting this test in 4 files, to bring that down
>>>> in a parallel run.  I'll repost with that.
>>>>
>>
>> Below's what I had in mind.  Splitting in 4 brings the time down to
>> ~30 seconds for me, while splitting in 8 brings it to ~25 seconds.
>> Looks like we hit diminishing returns, so I left it at 8.
>>
>> I also added kfail/skips for rl78 and rx, otherwise the test
>> crashes GDB...  Those can be removed as soon as your patches
>> are in (I hope, I haven't tested whether the archs have
>> further problems that would be exposed by this patch).
>>
>> WDYT?
>>
> 
> That looks good to me.

OK, I'll push it in then.

> After your test is pushed in, I'll push my
> rl78 and rx patches in, and remove the kfail from this test.  

Perfect.

> My
> rl78 and rx patches should go to 7.12 branch, but I am not sure
> this test case should go to 7.12 or not.

Yeah, should probably leave it on master only.

> 
>>>
>>> I'll extend all-architectures.exp to have a test "disassemble 0x0,+4"
>>> for PR 20939.  GDB now aborts due to the "foreign frame" again.
>>
>> That seems more dependent on host architecture than
>> target architecture, I think?  I.e., cycling over
>> target architectures and disassembling won't really add
>> more coverage?
> 
> Yes, PR 20939 is about host arch, but such test does find other issues,
> like PR 20955.  Other issues are shown up after the rl78 and rx segment
> fault is fixed.

OK, I see.

Thanks,
Pedro Alves
diff mbox

Patch

diff --git a/gdb/testsuite/gdb.base/all-architectures-0.exp b/gdb/testsuite/gdb.base/all-architectures-0.exp
new file mode 100644
index 0000000..cd28390
--- /dev/null
+++ b/gdb/testsuite/gdb.base/all-architectures-0.exp
@@ -0,0 +1,17 @@ 
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+set test_slice 0
+source $srcdir/$subdir/all-architectures.exp.in
diff --git a/gdb/testsuite/gdb.base/all-architectures-1.exp b/gdb/testsuite/gdb.base/all-architectures-1.exp
new file mode 100644
index 0000000..034fbc4
--- /dev/null
+++ b/gdb/testsuite/gdb.base/all-architectures-1.exp
@@ -0,0 +1,17 @@ 
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+set test_slice 1
+source $srcdir/$subdir/all-architectures.exp.in
diff --git a/gdb/testsuite/gdb.base/all-architectures-2.exp b/gdb/testsuite/gdb.base/all-architectures-2.exp
new file mode 100644
index 0000000..142bc74
--- /dev/null
+++ b/gdb/testsuite/gdb.base/all-architectures-2.exp
@@ -0,0 +1,17 @@ 
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+set test_slice 2
+source $srcdir/$subdir/all-architectures.exp.in
diff --git a/gdb/testsuite/gdb.base/all-architectures-3.exp b/gdb/testsuite/gdb.base/all-architectures-3.exp
new file mode 100644
index 0000000..02ebf14
--- /dev/null
+++ b/gdb/testsuite/gdb.base/all-architectures-3.exp
@@ -0,0 +1,17 @@ 
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+set test_slice 3
+source $srcdir/$subdir/all-architectures.exp.in
diff --git a/gdb/testsuite/gdb.base/all-architectures-4.exp b/gdb/testsuite/gdb.base/all-architectures-4.exp
new file mode 100644
index 0000000..1041869
--- /dev/null
+++ b/gdb/testsuite/gdb.base/all-architectures-4.exp
@@ -0,0 +1,17 @@ 
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+set test_slice 4
+source $srcdir/$subdir/all-architectures.exp.in
diff --git a/gdb/testsuite/gdb.base/all-architectures-5.exp b/gdb/testsuite/gdb.base/all-architectures-5.exp
new file mode 100644
index 0000000..9a524d4
--- /dev/null
+++ b/gdb/testsuite/gdb.base/all-architectures-5.exp
@@ -0,0 +1,17 @@ 
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+set test_slice 5
+source $srcdir/$subdir/all-architectures.exp.in
diff --git a/gdb/testsuite/gdb.base/all-architectures-6.exp b/gdb/testsuite/gdb.base/all-architectures-6.exp
new file mode 100644
index 0000000..35092e9
--- /dev/null
+++ b/gdb/testsuite/gdb.base/all-architectures-6.exp
@@ -0,0 +1,17 @@ 
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+set test_slice 6
+source $srcdir/$subdir/all-architectures.exp.in
diff --git a/gdb/testsuite/gdb.base/all-architectures-7.exp b/gdb/testsuite/gdb.base/all-architectures-7.exp
new file mode 100644
index 0000000..a1702bf
--- /dev/null
+++ b/gdb/testsuite/gdb.base/all-architectures-7.exp
@@ -0,0 +1,17 @@ 
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+set test_slice 7
+source $srcdir/$subdir/all-architectures.exp.in
diff --git a/gdb/testsuite/gdb.base/all-architectures.exp b/gdb/testsuite/gdb.base/all-architectures.exp.in
similarity index 91%
rename from gdb/testsuite/gdb.base/all-architectures.exp
rename to gdb/testsuite/gdb.base/all-architectures.exp.in
index 9748304..4a8099c 100644
--- a/gdb/testsuite/gdb.base/all-architectures.exp
+++ b/gdb/testsuite/gdb.base/all-architectures.exp.in
@@ -34,6 +34,11 @@ 
 # This test catches the issue when run against gdb linked with
 # libmcheck, or run under Valgrind.
 
+# Note: this test is actually split in several driver .exp files, in
+# order to be able to parallelize the work.  Each driver .exp file
+# exercises a different slice of the supported architectures.  See
+# all-architectures-*.exp and the TEST_SLICE variable.
+
 clean_restart
 
 # By default, preparation steps don't output a PASS message.  This is
@@ -181,13 +186,42 @@  proc gdb_test_no_output_osabi {cmd test} {
 # of writting, it'd jump from 20s to 4min on x86-64 GNU/Linux with
 # --enable-targets=all.
 
+set num_slices 8
+set num_archs [llength $supported_archs]
+set archs_per_slice [expr (($num_archs + $num_slices - 1) / $num_slices)]
+
 with_test_prefix "tests" {
     foreach_with_prefix osabi $supported_osabis {
 
 	gdb_test_no_output_osabi "set osabi $osabi" \
 	    "set osabi"
 
+	set arch_count 0
 	foreach_with_prefix arch $supported_archs {
+
+	    incr arch_count
+
+	    # Skip architectures outside our slice.
+	    if {$arch_count < [expr $test_slice * $archs_per_slice]} {
+		continue
+	    }
+	    if {$arch_count >= [expr ($test_slice + 1) * $archs_per_slice]} {
+		continue
+	    }
+
+	    if {$arch == "rl78"} {
+		if {$want_tests_messages} {
+		    kfail "set architecture rl78" "gdb/20953"
+		}
+		continue
+	    }
+	    if {$arch == "rx"} {
+		if {$want_tests_messages} {
+		    kfail "set architecture rx" "gdb/20954"
+		}
+		continue
+	    }
+
 	    if {$arch == "auto"} {
 		continue
 	    }