From patchwork Tue Nov 7 18:18:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Burgess X-Patchwork-Id: 79349 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D32EE3857701 for ; Tue, 7 Nov 2023 18:19:16 +0000 (GMT) X-Original-To: gdb-patches@sourceware.org Delivered-To: gdb-patches@sourceware.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 4D9B13858C33 for ; Tue, 7 Nov 2023 18:19:02 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4D9B13858C33 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4D9B13858C33 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699381143; cv=none; b=vwpMaULB7HZXyTmjErMfBUxADVd9a1Ny8Jx1ReftCBtk9ontFtjXqxdFFtbek1eXKKcZudwKH/DCw++15BL94bcHDNqk5j46/Q7qUCrisvmO9s4l/LkOHX/wGTTJeyg14BhPQ1nGOWAu3nKbrbfAi2N4QtKpMyndnDGmkQxVRkc= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699381143; c=relaxed/simple; bh=SC7Cg5t+J4bPfTOhCEJ21haIPajUoDTPLwgwVaoXDZY=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=jD7a3FSxh2/xt/rtmpFpdjsJo2z2NcMCGR+8enw1bmIW/SYy6IhPumvsfywbten2z1796GtvSeBE09P87hhZdzK8AfIZg+tF4HMz9H5btjXgErPohyxsTs4M98x1nlqFGHytlGb72zD/xDTJ1IcvIzjSJ/oROX7yViaR8LorCsg= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1699381141; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nUZEKBXddizE+qYKYBx6GSCegi9REdzFbGY/5oSF4kM=; b=MP3F9TZWxmoaaicg+ON3bWdww0RZ19Exhd1JsOMJe/uj3UEhreIFSH2J5tY1H/Du2Ew+yw XlYB0wOx221+NCwazCyi6MJutIVzKyQVhursRB3eNnk+aEPbC64JhLhqW7kRbCYV/RUIWT MrHE/GPF4Z6ZAyDgApMmPAGqf0LhydQ= Received: from mail-lj1-f199.google.com (mail-lj1-f199.google.com [209.85.208.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-638-oXUacG9YMNWPQPgRHqb8gw-1; Tue, 07 Nov 2023 13:18:59 -0500 X-MC-Unique: oXUacG9YMNWPQPgRHqb8gw-1 Received: by mail-lj1-f199.google.com with SMTP id 38308e7fff4ca-2c50257772bso55872431fa.3 for ; Tue, 07 Nov 2023 10:18:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699381137; x=1699985937; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nUZEKBXddizE+qYKYBx6GSCegi9REdzFbGY/5oSF4kM=; b=WLd3snsod37UKYmOJ9hvbMG06+kjSo+KIi1Zwc1lQNMLAc2MurpF8UsIwJcIA1uUvP OjiZ9jjJqJyySQuwpshqfqiKEvwSFRSZaeu4HIlrBztuHR/7RmfKHzbwkdZi0VQwF5zM LSKpk5vqU9A2Vmre5a5jLazAjfuV6BG8DTIIuQ2uKoSp/MLm8Sk3wc3x8ASYbBcfdmW0 ikfD2ebdx9M0QywtIJFn0LDO7UQak0rqOEHii7wM8gB9iTEnEyr+XymYjdi8oKSmEWEA fNlVYxdD9F0NrCvm8pHRTCj7f0Z44FL6DYhYlshGyDJUDKGJm67VkTHQiwj47tSOeONK Iibw== X-Gm-Message-State: AOJu0Yxn4Yp5udZUQitFOWPuXN2H/FYV6TzTxrZPt25pT2DgL/nkdpVi tfFS0y1G3YPzRUtBKjw8GzhJMEvhvM7EcmJIyvlD0wUEJ/0N+HJhm4mkLrlM7q5kXRBYUaluMh9 PIYZMnJHx5o+GxER8kAtSK8rbTBFnNBgfjXdDwXsogPpOuUWAfxrSiy/F4Y496UYbX6h/BY/fSG 3SifmEjw== X-Received: by 2002:a05:651c:54a:b0:2c7:fa6:718c with SMTP id q10-20020a05651c054a00b002c70fa6718cmr10666833ljp.9.1699381136919; Tue, 07 Nov 2023 10:18:56 -0800 (PST) X-Google-Smtp-Source: AGHT+IGBc+kEFOFfDfh2zSu4oKUZlry9dzGur69mlyUvwvALPrbDFWIBT/FU5VyUA3HzhzGQusOUEg== X-Received: by 2002:a05:651c:54a:b0:2c7:fa6:718c with SMTP id q10-20020a05651c054a00b002c70fa6718cmr10666811ljp.9.1699381136497; Tue, 07 Nov 2023 10:18:56 -0800 (PST) Received: from localhost (105.226.159.143.dyn.plus.net. [143.159.226.105]) by smtp.gmail.com with ESMTPSA id n26-20020a1c721a000000b0040849ce7116sm16193180wmc.43.2023.11.07.10.18.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Nov 2023 10:18:56 -0800 (PST) From: Andrew Burgess To: gdb-patches@sourceware.org Cc: Andrew Burgess Subject: [PATCH 2/2] gdb/testsuite: add a new check-all-boards target Date: Tue, 7 Nov 2023 18:18:49 +0000 Message-Id: X-Mailer: git-send-email 2.25.4 In-Reply-To: References: MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gdb-patches@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gdb-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gdb-patches-bounces+patchwork=sourceware.org@sourceware.org The make-check-all.sh script (gdb/testsuite/make-check-all.sh) is great, it makes it super easy to run some test(s) using all the available board files. This commit aims to make this script even easier to access by adding a check-all-boards target to the GDB Makefile. This new target checks for (and requires) a number of environment variables, so the target should be used like this: make check-all-boards GDB_TARGET_USERNAME=remote-target \ GDB_HOST_USERNAME=remote-host \ TESTS="gdb.base/break.exp" Where GDB_TARGET_USERNAME and GDB_HOST_USERNAME are the user names that should be passed to the make-check-all.sh --target-user and --host-user command line options respectively. My personal intention is to set these variables in my environment, so all I'll need to do is: make check-all-boards TESTS="gdb.base/break.exp" The make rule always passes --keep-results to the make-check-all.sh script, as I find that the most useful. It's super frustrating to run the tests and realise you forgot that option and the results have been discarded. --- gdb/Makefile.in | 8 ++++++++ gdb/testsuite/Makefile.in | 19 ++++++++++++++++++ gdb/testsuite/README | 42 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 69 insertions(+) diff --git a/gdb/Makefile.in b/gdb/Makefile.in index 9c0a0bff2cd..491003c7543 100644 --- a/gdb/Makefile.in +++ b/gdb/Makefile.in @@ -1971,6 +1971,14 @@ check-parallel: force $(MAKE) $(TARGET_FLAGS_TO_PASS) check-parallel; \ else true; fi +check-all-boards: force + @if [ -f testsuite/Makefile ]; then \ + rootme=`pwd`; export rootme; \ + rootsrc=`cd $(srcdir); pwd`; export rootsrc; \ + cd testsuite; \ + $(MAKE) $(TARGET_FLAGS_TO_PASS) check-all-boards; \ + else true; fi + # The idea is to parallelize testing of multilibs, for example: # make -j3 check//sh-hms-sim/{-m1,-m2,-m3,-m3e,-m4}/{,-nofpu} # will run 3 concurrent sessions of check, eventually testing all 10 diff --git a/gdb/testsuite/Makefile.in b/gdb/testsuite/Makefile.in index 3fc9a907f5b..3488e6acc48 100644 --- a/gdb/testsuite/Makefile.in +++ b/gdb/testsuite/Makefile.in @@ -368,6 +368,25 @@ check-perf: all $(abs_builddir)/site.exp @if test ! -d gdb.perf; then mkdir gdb.perf; fi $(DO_RUNTEST) --directory=gdb.perf --outdir gdb.perf GDB_PERFTEST_MODE=$(GDB_PERFTEST_MODE) $(RUNTESTFLAGS) $(TIMESTAMP) + +check-all-boards: all $(abs_builddir)/site.exp + @if [ -z "$(GDB_TARGET_USERNAME)" ]; then \ + echo "Must define GDB_TARGET_USERNAME"; \ + false; \ + fi + @if [ -z "$(GDB_HOST_USERNAME)" ]; then \ + echo "Must define GDB_HOST_USERNAME"; \ + false; \ + fi + @if [ -z "$(TESTS)" ]; then \ + echo "Must define TESTS"; \ + false; \ + fi + ${abs_srcdir}/make-check-all.sh --keep-results \ + --host-user "$(GDB_HOST_USERNAME)" \ + --target-user "$(GDB_TARGET_USERNAME)" \ + "$(TESTS)" + force:; clean mostlyclean: diff --git a/gdb/testsuite/README b/gdb/testsuite/README index 330a385f349..5ad0293649e 100644 --- a/gdb/testsuite/README +++ b/gdb/testsuite/README @@ -328,6 +328,48 @@ For example, to turn on gdbserver debugging, you can do: make check GDBSERVER_DEBUG="debug,replay" +GDB_TARGET_USERNAME +GDB_HOST_USERNAME + +These settings are only used with the check-all-boards target, and +should be the usernames of two separate users on the local machine, +both of which the current user can be ssh to without a password. + +These users will be used when by board files that simulate remote +targets by switching to a different user on the same machine. These +users will have random files copied into their $HOME directories, so +it is a good idea to setup new users just for this purpose. + +Testing All Simple Boards +************************* + +There are a number of boards that ship with GDB that simulate common +debug scenarios. For example by sshing to a different user on the +local machine and running gdbserver as this alternative user we aim to +simulate a true remote debug experience. + +There is a script binutils-gdb/gdb/testssuite/make-check-all.sh which +can be used to run a defined set of tests using all of the available +simple board files. Support for using this script is also included in +GDB's makefile, and can be used as: + + make check-all-boards GDB_TARGET_USERNAME=remote-target \ + GDB_HOST_USERNAME=remote-host \ + TESTS="gdb.base/break.exp" + +The 'remote-target' and 'remote-host' can be replaced with any user +names on the local machine, the only requirements are that the current +user must be able to ssh to these users without a password, and these +users must be happy to have arbitrary files copied into their $HOME +directory. Ideally, these users would be setup just for GDB testing. + +The check-all-boards target requires that TESTS be defined, though it +is fine to include multiple tests. + +There results are preserved and can be found in +gdb/testsuite/check-all/, within this directory the results are split +by the board file used. + Architecture-specific Parameters ******************************