diff mbox

test case names in gdb testsuite

Message ID B09655FC-6477-4A70-954C-BA9038AA2344@comcast.net
State New, archived
Headers

Commit Message

Mike Stump Feb. 18, 2016, 11:16 p.m. UTC 
  So, the design of the tests suite was to have portable names that can survive prefix changes.  In the gdb test suite, I see:

PASS: gdb.base/break-fun-addr.exp: /home/mrs/work/binutils-machine/gdb/testsuite/outputs/gdb.base/break-fun-addr/break-fun-addr1: break *main
PASS: gdb.base/break-fun-addr.exp: /home/mrs/work/binutils-machine/gdb/testsuite/outputs/gdb.base/break-fun-addr/break-fun-addr1: run to breakpoint at *main
PASS: gdb.base/break-fun-addr.exp: /home/mrs/work/binutils-machine/gdb/testsuite/outputs/gdb.base/break-fun-addr/break-fun-addr1: p $pc == main
FAIL: gdb.base/break-fun-addr.exp: /home/mrs/work/binutils-machine/gdb/testsuite/outputs/gdb.base/break-fun-addr/break-fun-addr2: run to breakpoint at *main
PASS: gdb.base/break-fun-addr.exp: /home/mrs/work/binutils-machine/gdb/testsuite/outputs/gdb.base/break-fun-addr/break-fun-addr2: p $pc == main

and”

PASS: gdb.python/py-completion.exp: source /home/mrs/work/binutils-machine/gdb/testsuite/../../../binutils/gdb/testsuite/gdb.python/py-completion.py
PASS: gdb.python/py-completion.exp: source /home/mrs/work/binutils-machine/gdb/testsuite/../../../binutils/gdb/testsuite/gdb.python/py-completion.py

which are the only violations of the general rule I see. 

I’m doing tool validating, and noticed the account name that did the test in the PASS:/FAIL: lines.  This causes failures when comparing across different testing accounts.  gcc, the reset of binutils and newlib don’t have this problem, gdb is the last tool to do this.


So, for the first group, this:


produces:

PASS: gdb.python/py-completion.exp: source gdb.python/py-completion.py
PASS: gdb.python/py-completion.exp: completefileinit completion
PASS: gdb.python/py-completion.exp: discard #1
PASS: gdb.python/py-completion.exp: completefilemethod completion
PASS: gdb.python/py-completion.exp: discard #2
PASS: gdb.python/py-completion.exp: completefilecommandcond completion
PASS: gdb.python/py-completion.exp: source gdb.python/py-completion.py
PASS: gdb.python/py-completion.exp: list all completions of 'complete completel'
PASS: gdb.python/py-completion.exp: discard #3
PASS: gdb.python/py-completion.exp: list all completions of 'complete completelimit1 c'
PASS: gdb.python/py-completion.exp: discard #4
PASS: gdb.python/py-completion.exp: tab-complete 'completelimit1 c'
PASS: gdb.python/py-completion.exp: discard #5
PASS: gdb.python/py-completion.exp: list all completions of 'complete completelimit2 c’

Ok?


The inline text won’t make it though patch, so, here is the attached version that will.
diff --git a/gdb/testsuite/gdb.base/break-fun-addr.exp b/gdb/testsuite/gdb.base/break-fun-addr.exp
index e8bed3f..96043ef 100644
--- a/gdb/testsuite/gdb.base/break-fun-addr.exp
+++ b/gdb/testsuite/gdb.base/break-fun-addr.exp
@@ -46,7 +46,7 @@ if  { [gdb_compile "${srcdir}/${subdir}/${srcfile1}" "${binfile1}" executable {d
 
 clean_restart ${binfile1}
 
-with_test_prefix "${binfile1}" {
+with_test_prefix "${subdir}/${gdb_test_file_name}/${testfile1}" {
 
     gdb_test "break *main" \
         "Breakpoint.*at.* file .*$srcfile1, line .*" \
@@ -73,7 +73,7 @@ if  { [gdb_compile "${srcdir}/${subdir}/${srcfile2}" "${binfile2}" executable {d
 # start a new debugging session with a different executable.
 gdb_load ${binfile2}
 
-with_test_prefix "${binfile2}" {
+with_test_prefix "${subdir}/${gdb_test_file_name}/${testfile2}" {
 
     gdb_run_cmd
     gdb_test "" \
diff --git a/gdb/testsuite/gdb.python/py-completion.exp b/gdb/testsuite/gdb.python/py-completion.exp
index 900d6f5..d8376d4 100644
--- a/gdb/testsuite/gdb.python/py-completion.exp
+++ b/gdb/testsuite/gdb.python/py-completion.exp
@@ -23,7 +23,7 @@ gdb_start
 # Skip all tests if Python scripting is not enabled.
 if { [skip_python_tests] } { continue }
 
-gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py"
+gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py" "source ${subdir}/${testfile}.py"
 
 # Create a temporary directory
 set testdir "[standard_output_file "py-completion-testdir"]/"
@@ -73,7 +73,7 @@ gdb_test_multiple "" "completefilecommandcond completion" {
 # with the expected output of the below tests in a buggy gdb.
 gdb_exit
 gdb_start
-gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py"
+gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py" "source ${subdir}/${testfile}.py"
 
 gdb_test_sequence "complete completel" \
     "list all completions of 'complete completel'" {

Comments

Simon Marchi Feb. 19, 2016, 2:58 p.m. UTC | #1 
On 16-02-18 06:16 PM, Mike Stump wrote:
> So, the design of the tests suite was to have portable names that can survive prefix changes.  In the gdb test suite, I see:
> 
> PASS: gdb.base/break-fun-addr.exp: /home/mrs/work/binutils-machine/gdb/testsuite/outputs/gdb.base/break-fun-addr/break-fun-addr1: break *main
> PASS: gdb.base/break-fun-addr.exp: /home/mrs/work/binutils-machine/gdb/testsuite/outputs/gdb.base/break-fun-addr/break-fun-addr1: run to breakpoint at *main
> PASS: gdb.base/break-fun-addr.exp: /home/mrs/work/binutils-machine/gdb/testsuite/outputs/gdb.base/break-fun-addr/break-fun-addr1: p $pc == main
> FAIL: gdb.base/break-fun-addr.exp: /home/mrs/work/binutils-machine/gdb/testsuite/outputs/gdb.base/break-fun-addr/break-fun-addr2: run to breakpoint at *main
> PASS: gdb.base/break-fun-addr.exp: /home/mrs/work/binutils-machine/gdb/testsuite/outputs/gdb.base/break-fun-addr/break-fun-addr2: p $pc == main
> 
> and”
> 
> PASS: gdb.python/py-completion.exp: source /home/mrs/work/binutils-machine/gdb/testsuite/../../../binutils/gdb/testsuite/gdb.python/py-completion.py
> PASS: gdb.python/py-completion.exp: source /home/mrs/work/binutils-machine/gdb/testsuite/../../../binutils/gdb/testsuite/gdb.python/py-completion.py
> 
> which are the only violations of the general rule I see. 
> 
> I’m doing tool validating, and noticed the account name that did the test in the PASS:/FAIL: lines.  This causes failures when comparing across different testing accounts.  gcc, the reset of binutils and newlib don’t have this problem, gdb is the last tool to do this.

Good point.

> So, for the first group, this:
> 
> diff --git a/gdb/testsuite/gdb.base/break-fun-addr.exp b/gdb/testsuite/gdb.base/break-fun-addr.exp
> index e8bed3f..96043ef 100644
> --- a/gdb/testsuite/gdb.base/break-fun-addr.exp
> +++ b/gdb/testsuite/gdb.base/break-fun-addr.exp
> @@ -46,7 +46,7 @@ if  { [gdb_compile "${srcdir}/${subdir}/${srcfile1}" "${binfile1}" executable {d
>  
>  clean_restart ${binfile1}
>  
> -with_test_prefix "${binfile1}" {
> +with_test_prefix "${subdir}/${gdb_test_file_name}/${testfile1}" {

Looks good.  You could even just use ${testfile1} to be more succinct.  There is already
"gdb.base/break-fun-addr.exp" on the result line, which makes it a bit redundant.


>      gdb_test "break *main" \
>          "Breakpoint.*at.* file .*$srcfile1, line .*" \
> @@ -73,7 +73,7 @@ if  { [gdb_compile "${srcdir}/${subdir}/${srcfile2}" "${binfile2}" executable {d
>  # start a new debugging session with a different executable.
>  gdb_load ${binfile2}
>  
> -with_test_prefix "${binfile2}" {
> +with_test_prefix "${subdir}/${gdb_test_file_name}/${testfile2}" {

Same here.

>      gdb_run_cmd
>      gdb_test "” \
> 
> should fix it.  This avoids the parallel names and the prefix.  The test case names now look like:
> 
> PASS: gdb.base/break-fun-addr.exp: gdb.base/break-fun-addr/break-fun-addr1: break *main
> PASS: gdb.base/break-fun-addr.exp: gdb.base/break-fun-addr/break-fun-addr1: run to breakpoint at *main
> PASS: gdb.base/break-fun-addr.exp: gdb.base/break-fun-addr/break-fun-addr1: p $pc == main
> PASS: gdb.base/break-fun-addr.exp: gdb.base/break-fun-addr/break-fun-addr2: p $pc == main
> 
> which is preferred.
> 
> For the second one, this:
> 
> diff --git a/gdb/testsuite/gdb.python/py-completion.exp b/gdb/testsuite/gdb.python/py-completion.exp
> index 900d6f5..d8376d4 100644
> --- a/gdb/testsuite/gdb.python/py-completion.exp
> +++ b/gdb/testsuite/gdb.python/py-completion.exp
> @@ -23,7 +23,7 @@ gdb_start
>  # Skip all tests if Python scripting is not enabled.
>  if { [skip_python_tests] } { continue }
>  
> -gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py"
> +gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py" "source ${subdir}/${testfile}.py"
>  
>  # Create a temporary directory
>  set testdir "[standard_output_file "py-completion-testdir"]/"
> @@ -73,7 +73,7 @@ gdb_test_multiple "" "completefilecommandcond completion" {
>  # with the expected output of the below tests in a buggy gdb.
>  gdb_exit
>  gdb_start
> -gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py"
> +gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py" "source ${subdir}/${testfile}.py"
>  
>  gdb_test_sequence "complete completel" \
>      "list all completions of 'complete completel'” {
> 
> produces:
> 
> PASS: gdb.python/py-completion.exp: source gdb.python/py-completion.py
> PASS: gdb.python/py-completion.exp: completefileinit completion
> PASS: gdb.python/py-completion.exp: discard #1
> PASS: gdb.python/py-completion.exp: completefilemethod completion
> PASS: gdb.python/py-completion.exp: discard #2
> PASS: gdb.python/py-completion.exp: completefilecommandcond completion
> PASS: gdb.python/py-completion.exp: source gdb.python/py-completion.py
> PASS: gdb.python/py-completion.exp: list all completions of 'complete completel'
> PASS: gdb.python/py-completion.exp: discard #3
> PASS: gdb.python/py-completion.exp: list all completions of 'complete completelimit1 c'
> PASS: gdb.python/py-completion.exp: discard #4
> PASS: gdb.python/py-completion.exp: tab-complete 'completelimit1 c'
> PASS: gdb.python/py-completion.exp: discard #5
> PASS: gdb.python/py-completion.exp: list all completions of 'complete completelimit2 c’
> 
> Ok?

That looks good to me, but a maintainer must approve.

THanks, Simon
diff mbox

Patch

diff --git a/gdb/testsuite/gdb.base/break-fun-addr.exp b/gdb/testsuite/gdb.base/break-fun-addr.exp
index e8bed3f..96043ef 100644
--- a/gdb/testsuite/gdb.base/break-fun-addr.exp
+++ b/gdb/testsuite/gdb.base/break-fun-addr.exp
@@ -46,7 +46,7 @@  if  { [gdb_compile "${srcdir}/${subdir}/${srcfile1}" "${binfile1}" executable {d
 
 clean_restart ${binfile1}
 
-with_test_prefix "${binfile1}" {
+with_test_prefix "${subdir}/${gdb_test_file_name}/${testfile1}" {
 
     gdb_test "break *main" \
         "Breakpoint.*at.* file .*$srcfile1, line .*" \
@@ -73,7 +73,7 @@  if  { [gdb_compile "${srcdir}/${subdir}/${srcfile2}" "${binfile2}" executable {d
 # start a new debugging session with a different executable.
 gdb_load ${binfile2}
 
-with_test_prefix "${binfile2}" {
+with_test_prefix "${subdir}/${gdb_test_file_name}/${testfile2}" {
 
     gdb_run_cmd
     gdb_test "” \

should fix it.  This avoids the parallel names and the prefix.  The test case names now look like:

PASS: gdb.base/break-fun-addr.exp: gdb.base/break-fun-addr/break-fun-addr1: break *main
PASS: gdb.base/break-fun-addr.exp: gdb.base/break-fun-addr/break-fun-addr1: run to breakpoint at *main
PASS: gdb.base/break-fun-addr.exp: gdb.base/break-fun-addr/break-fun-addr1: p $pc == main
PASS: gdb.base/break-fun-addr.exp: gdb.base/break-fun-addr/break-fun-addr2: p $pc == main

which is preferred.

For the second one, this:

diff --git a/gdb/testsuite/gdb.python/py-completion.exp b/gdb/testsuite/gdb.python/py-completion.exp
index 900d6f5..d8376d4 100644
--- a/gdb/testsuite/gdb.python/py-completion.exp
+++ b/gdb/testsuite/gdb.python/py-completion.exp
@@ -23,7 +23,7 @@  gdb_start
 # Skip all tests if Python scripting is not enabled.
 if { [skip_python_tests] } { continue }
 
-gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py"
+gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py" "source ${subdir}/${testfile}.py"
 
 # Create a temporary directory
 set testdir "[standard_output_file "py-completion-testdir"]/"
@@ -73,7 +73,7 @@  gdb_test_multiple "" "completefilecommandcond completion" {
 # with the expected output of the below tests in a buggy gdb.
 gdb_exit
 gdb_start
-gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py"
+gdb_test_no_output "source ${srcdir}/${subdir}/${testfile}.py" "source ${subdir}/${testfile}.py"
 
 gdb_test_sequence "complete completel" \
     "list all completions of 'complete completel'” {