From patchwork Tue May 14 15:12:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Hayward X-Patchwork-Id: 32684 Received: (qmail 61362 invoked by alias); 14 May 2019 15:13:05 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Delivered-To: mailing list gdb-patches@sourceware.org Received: (qmail 61309 invoked by uid 89); 14 May 2019 15:13:04 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-23.9 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_SHORT, MIME_BASE64_BLANKS, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.1 spammy=Performance X-HELO: EUR01-DB5-obe.outbound.protection.outlook.com Received: from mail-eopbgr150077.outbound.protection.outlook.com (HELO EUR01-DB5-obe.outbound.protection.outlook.com) (40.107.15.77) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 14 May 2019 15:13:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U9+0VWORIWCP4KG6SOJ4DgDgcTqQNS0YkQY0Jxc/tV8=; b=DhSD9LsYi1pfTt9kmFPnG1C3X+m+d5MV9YapQo6hrq1ikSfcukXuzXIPpZyTNr9X5HXOi4+eXQlAIJ3bPbWzhDAMRXKMrZKuyaPbSlSq02IJYBgd9f0FCQI808Nd2DnEXHIAhIr1VMQkMOgLFpMilszpLbmneUkvjMK//RmC/mc= Received: from DB6PR0802MB2133.eurprd08.prod.outlook.com (10.172.227.22) by DB6PR0802MB2535.eurprd08.prod.outlook.com (10.172.251.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1878.24; Tue, 14 May 2019 15:12:48 +0000 Received: from DB6PR0802MB2133.eurprd08.prod.outlook.com ([fe80::8c26:bb4b:6c93:9d40]) by DB6PR0802MB2133.eurprd08.prod.outlook.com ([fe80::8c26:bb4b:6c93:9d40%2]) with mapi id 15.20.1878.024; Tue, 14 May 2019 15:12:48 +0000 From: Alan Hayward To: "gdb-patches@sourceware.org" CC: nd , Alan Hayward Subject: [PATCH 8/9] testsuite: Add script to quickly re-run tests Date: Tue, 14 May 2019 15:12:48 +0000 Message-ID: <20190514151238.8765-9-alan.hayward@arm.com> References: <20190514151238.8765-1-alan.hayward@arm.com> In-Reply-To: <20190514151238.8765-1-alan.hayward@arm.com> authentication-results: spf=none (sender IP is ) smtp.mailfrom=Alan.Hayward@arm.com; x-ms-exchange-purlcount: 1 x-ms-oob-tlc-oobclassifiers: OLM:7219; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 MIME-Version: 1.0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-IsSubscribed: yes Using the .cmd and .in files created by a test it is fairly simple to manually re-run a test. For gdbserver tests, the .replay log can also be used. Document the process required to do this. In addition create a script to automate the process. 2019-05-14 Alan Hayward * README (Re-running Tests Outside The Testsuite): New section. * lib/gdbserver-support.exp (gdbserver_run): Mark kill as optional. * replaytest: New test. --- gdb/testsuite/README | 23 ++++ gdb/testsuite/lib/gdbserver-support.exp | 2 +- gdb/testsuite/replaytest | 147 ++++++++++++++++++++++++ 3 files changed, 171 insertions(+), 1 deletion(-) create mode 100755 gdb/testsuite/replaytest -- 2.20.1 (Apple Git-117) diff --git a/gdb/testsuite/README b/gdb/testsuite/README index 98fc8d1b852..b5fbb456bee 100644 --- a/gdb/testsuite/README +++ b/gdb/testsuite/README @@ -95,6 +95,29 @@ example: The script will output its analysis report to the standard output. +Re-running Tests Outside The Testsuite +************************************** + +When running a test, the arguments used to run GDB are saved to gdb.cmd +and any commands sent to GDB are saved to gdb.in. As well as being a +reference of the commands run, they can be used to manually re-run a test +by giving the .in file as a batch file to a GDB launched with the arguments +in the .cmd file. For many tests you can use just the .in file: + $ gdb -x gdb.in + +Tests that run GDB multiple times will append .1, .2, .3 etc to the end +of each .cmd and .in file. + +For tests requiring gdbserver, this can by executing the contents of +gdbserver.cmd. Alternatively, the gdbserver.replay file (created when +running the test with GDBSERVER_DEBUG="replay") can be used with the +gdbreplay test. + +This process can be automated with the "replaytest" script. +For example: + $ replaytest gdb testsuite/outputs/gdb.base/store +For more details see the documentation in the script. + Running the Performance Tests ***************************** diff --git a/gdb/testsuite/lib/gdbserver-support.exp b/gdb/testsuite/lib/gdbserver-support.exp index 2ccc717ef60..ade99c0ea16 100644 --- a/gdb/testsuite/lib/gdbserver-support.exp +++ b/gdb/testsuite/lib/gdbserver-support.exp @@ -484,7 +484,7 @@ proc gdbserver_run { child_args } { # Kill anything running before we try to start gdbserver, in case # we are sharing a serial connection. global gdb_prompt - send_gdb "kill\n" + send_gdb "kill\n" optional gdb_expect 120 { -re "Kill the program being debugged. .y or n. $" { send_gdb "y\n" diff --git a/gdb/testsuite/replaytest b/gdb/testsuite/replaytest new file mode 100755 index 00000000000..4c11d9735f3 --- /dev/null +++ b/gdb/testsuite/replaytest @@ -0,0 +1,147 @@ +#!/bin/bash + +# Copyright (C) 2019 Free Software Foundation, Inc. +# +# This file is part of GDB. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# +# This script uses GDB to replay previously run tests using the .in and +# .cmd files from the output directory of a test. +# +# For example: +# $ replaytest gdb testsuite/outputs/gdb.base/store +# +# +# For tests that have been runing using gdbserver (for example with +# board file native-gdbserver), the test script should be run in two +# terminals, using gdbserver and gdb modes. +# +# For example, in two different terminals: +# $ replaytest -b gdbserver testsuite/outputs/gdb.base/store +# $ replaytest -b gdb testsuite/outputs/gdb.base/store +# +# Alternatively, instead of running gdbserver, the gdbreplay tool can +# be used instead. This uses the .replay file which is created when the +# test is run with GDBSERVER_DEBUG="replay". gdbreplay must be in your +# $PATH. +# +# +# Tests that run GDB (and gdbserver) multiple times will create .1, .2 +# .3 etc files for each instance of GDB. You can run a specified +# instance using the -i flag +# +# For example: +# $ replaytest -b -i 23 gdb testsuite/outputs/gdb.base/infcall-nested-structs +# +# Note that to successfully run to the end of the test, the .in file must be +# free from the answers to any interactive gdb prompts (for example the y from +# a y/n question) or any command that causes gdb to report an error. +# +# +# Usage: replaytest [-b] [-i instance] [-e] {gdb|gdbserver|replay} test_output_dir +# +# -b : Run GDB in batch mode (ie: exit GDB at the end of the test). +# -i : Run the given instance of the test. Defaults to 0. +# -e : Don't run the test, just echo the command line instead. +# +# test_output_dir : Directory containing all the test output files. +# + +usage() +{ + echo "$0 [-b] [-i instance] [-e] {gdb|gdbserver|replay} test_output_dir" + exit 2 +} + +# Check file $1 exists, using test operator $2 (defaults to -e). +check_exists() +{ + if [ ! ${2:-"-e"} "${1}" ]; then echo Error: Cannot find $1; exit 2; fi +} + +# Run the given command in $1...n +execute() +{ + if [ -n "$ECHO" ] + then + echo $@ + else + # Pipe the command we want to run to a file then execute the file. This + # ensures spaces within quoatation marks args get interpreted correctly. + OUTFILE=$(mktemp) + echo $@ > ${OUTFILE} + bash ${OUTFILE} + rm ${OUTFILE} + fi +} + +# Parse command line +while getopts bi:e line +do + case $line in + b) EXTRA_GDBCMD="${EXTRA_GDBCMD} --batch";; + i) INSTANCE=".$OPTARG";; + e) ECHO=1;; + *) usage;; + esac +done +shift $(($OPTIND - 1)) +if [[ $# -ne 2 ]]; then usage; fi; +MODE=$1 +TESTDIR=$2 +check_exists ${TESTDIR} -d + + +case $MODE in + +"gdb") + # Run the .in file through GDB by appending it to the end of the .cmd contents. + CMDFILE=${TESTDIR}/gdb.cmd${INSTANCE} + INFILE=${TESTDIR}/gdb.in${INSTANCE} + check_exists ${CMDFILE} + check_exists ${INFILE} + execute "$(cat $CMDFILE) ${EXTRA_GDBCMD} -x ${INFILE}" + ;; + +"gdbserver") + # Run gdbserver using the .cmd file. + GDBSERVER_CMDFILE=${TESTDIR}/gdbserver.cmd${INSTANCE} + check_exists ${GDBSERVER_CMDFILE} + execute $(cat $GDBSERVER_CMDFILE) + ;; + +"replay") + GDBSERVER_REPLAYFILE=${TESTDIR}/gdbserver.replay${INSTANCE} + check_exists ${GDBSERVER_REPLAYFILE} + + # Extract the connection from the .cmd file. + GDBSERVER_CMDFILE=${TESTDIR}/gdbserver.cmd${INSTANCE} + check_exists ${GDBSERVER_CMDFILE} + GDBSERVER_HOST=$(grep -o '[^ ]*:[^ ]*' ${GDBSERVER_CMDFILE}) + + # Run the .replay file through gdbreplay. + execute gdbreplay ${GDBSERVER_REPLAYFILE} ${GDBSERVER_HOST} + ;; + +*) + echo "Unknown mode $MODE" + usage + ;; + +esac + + +