From patchwork Fri Apr 21 21:25:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Frank Ch. Eigler" X-Patchwork-Id: 68151 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 522C83858401 for ; Fri, 21 Apr 2023 21:25:59 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 522C83858401 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1682112359; bh=lbjyFhgBZtfULca8zlfkxC6FZmWhS+lMMmd95J4Z3aM=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Help: List-Subscribe:From:Reply-To:From; b=w8qEhcZale1KRvbhhKEk2qswVos17vn8X+xGunEW9YPtzaebEj+ajA9xxSyLijWCZ ofazHFC44MJoeLX1nwwh00jZNy25qoGKjf7SrsnF2aZikBmorH5nTTCjVzbZDfncB+ crPMpAVk6MH+XxFdqRqXjvFZYeYi9t3I23kbN+DU= X-Original-To: elfutils-devel@sourceware.org Delivered-To: elfutils-devel@sourceware.org Received: from elastic.org (elastic.org [96.126.110.187]) by sourceware.org (Postfix) with ESMTPS id 8B8813858D20 for ; Fri, 21 Apr 2023 21:25:48 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 8B8813858D20 Received: from vpn-home.elastic.org ([10.0.0.2] helo=elastic.org) by elastic.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1ppyGJ-0007rS-26 for elfutils-devel@sourceware.org; Fri, 21 Apr 2023 21:25:47 +0000 Received: from very.elastic.org ([192.168.1.1]) by elastic.org with esmtp (Exim 4.96) (envelope-from ) id 1ppyGJ-000Ca1-1c for elfutils-devel@sourceware.org; Fri, 21 Apr 2023 17:25:47 -0400 Received: from fche by very.elastic.org with local (Exim 4.96) (envelope-from ) id 1ppyGJ-008XN1-0C for elfutils-devel@sourceware.org; Fri, 21 Apr 2023 17:25:47 -0400 Date: Fri, 21 Apr 2023 17:25:47 -0400 To: elfutils-devel@sourceware.org Subject: [patch] PR30377: debuginfod -r -X Message-ID: MIME-Version: 1.0 Content-Disposition: inline X-Sender-Verification: "" X-Spam-Status: No, score=-107.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, USER_IN_WELCOMELIST, USER_IN_WHITELIST autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-Patchwork-Original-From: "Frank Ch. Eigler via Elfutils-devel" From: "Frank Ch. Eigler" Reply-To: "Frank Ch. Eigler" Errors-To: elfutils-devel-bounces+patchwork=sourceware.org@sourceware.org Sender: "Elfutils-devel" Hi - Planning to push this one-liner fix shortly. A new test case exercises the -r / -I / -X ops. commit bc6c774dfb61ac61a66195a269579544f97eeb30 Author: Frank Ch. Eigler Date: Fri Apr 21 17:04:08 2023 -0400 PR30377: fix debuginfod -r -X combination Until this fix, debuginfod -r -X '.*' didn't trigger groom-time removal of everything, because the -I include regex overrode it. Corrected logic to match the scan-time tie-breaking between -I / -X. Signed-off-by: Frank Ch. Eigler diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog index 070dad03fba1..0e4810bba501 100644 --- a/debuginfod/ChangeLog +++ b/debuginfod/ChangeLog @@ -1,3 +1,7 @@ +2023-04-21 Frank Ch. Eigler + + * debuginfod.cxx (groom): Fix -r / -X logic. + 2023-04-13 Frank Ch. Eigler * debuginfod.cxx (archive_classify, scan_archive_file): Catch and diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx index e981d1372233..a1ddeb569a36 100644 --- a/debuginfod/debuginfod.cxx +++ b/debuginfod/debuginfod.cxx @@ -3893,7 +3893,7 @@ void groom() { bool reg_include = !regexec (&file_include_regex, filename, 0, 0, 0); bool reg_exclude = !regexec (&file_exclude_regex, filename, 0, 0, 0); - regex_file_drop = reg_exclude && !reg_include; + regex_file_drop = !reg_include || reg_exclude; // match logic of scan_source_paths } rc = stat(filename, &s); diff --git a/doc/debuginfod.8 b/doc/debuginfod.8 index 07cb01aeed10..5358aaba42c9 100644 --- a/doc/debuginfod.8 +++ b/doc/debuginfod.8 @@ -172,7 +172,10 @@ interrupting a groom pass (if any). .TP .B "\-r" -Apply the -I and -X during groom cycles, so that files excluded by the regexes are removed from the index. These parameters are in addition to what normally qualifies a file for grooming, not a replacement. +Apply the -I and -X during groom cycles, so that most content related +to files excluded by the regexes are removed from the index. Not all +content can be practically removed, so eventually a "\-G" +"maximal-groom" operation may be needed. .TP .B "\-g SECONDS" "\-\-groom\-time=SECONDS" diff --git a/tests/ChangeLog b/tests/ChangeLog index eb3e1118fa00..d816873ce083 100644 --- a/tests/ChangeLog +++ b/tests/ChangeLog @@ -1,3 +1,8 @@ +2023-04-21 Frank Ch. Eigler + + * run-debuginfod-IXr.sh: New test. + * Makefile.am: Run it, ship it. + 2023-02-10 Mark Wielaard * varlocs.c (print_expr): Handle DW_OP_GNU_uninit. diff --git a/tests/Makefile.am b/tests/Makefile.am index 7e32f1170c1b..0ca8aa9e7e81 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -261,7 +261,8 @@ TESTS += run-debuginfod-dlopen.sh \ run-debuginfod-response-headers.sh \ run-debuginfod-extraction-passive.sh \ run-debuginfod-webapi-concurrency.sh \ - run-debuginfod-section.sh + run-debuginfod-section.sh \ + run-debuginfod-IXr.sh endif if !OLD_LIBMICROHTTPD # Will crash on too old libmicrohttpd @@ -578,6 +579,7 @@ EXTRA_DIST = run-arextract.sh run-arsymtest.sh run-ar.sh \ run-debuginfod-extraction-passive.sh \ run-debuginfod-webapi-concurrency.sh \ run-debuginfod-section.sh \ + run-debuginfod-IXr.sh \ debuginfod-rpms/fedora30/hello2-1.0-2.src.rpm \ debuginfod-rpms/fedora30/hello2-1.0-2.x86_64.rpm \ debuginfod-rpms/fedora30/hello2-debuginfo-1.0-2.x86_64.rpm \ diff --git a/tests/run-debuginfod-IXr.sh b/tests/run-debuginfod-IXr.sh new file mode 100755 index 000000000000..631b7bbfbfd2 --- /dev/null +++ b/tests/run-debuginfod-IXr.sh @@ -0,0 +1,94 @@ +#!/usr/bin/env bash +# +# Copyright (C) 2019-2023 Red Hat, Inc. +# This file is part of elfutils. +# +# This file is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# elfutils is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +. $srcdir/debuginfod-subr.sh + +# for test case debugging, uncomment: +set -x +unset VALGRIND_CMD + +mkdir R Z +# This variable is essential and ensures no time-race for claiming ports occurs +# set base to a unique multiple of 100 not used in any other 'run-debuginfod-*' test +base=10100 +get_ports + +DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB +export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache + +cp -rvp ${abs_srcdir}/debuginfod-rpms R + +if [ "$zstd" = "false" ]; then # nuke the zstd fedora 31 ones + rm -vrf R/debuginfod-rpms/fedora31 + rpms=0 + tarxz=1 + groomed=2 +else + rpms=6 + tarxz=1 + groomed=8 +fi + +cp -rvp ${abs_srcdir}/debuginfod-tars Z + +env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -d $DB -X 'hello2' -I 'hello' -R -Z .tar.xz -Z .tar.bz2=bzcat -p $PORT1 -t0 -g0 -v R Z > vlog$PORT1 2>&1 & +PID1=$! +tempfiles vlog$PORT1 +errfiles vlog$PORT1 +# Server must become ready +wait_ready $PORT1 'ready' 1 + +# Wait till both files are in the index and scan/index fully finished +wait_ready $PORT1 'thread_work_total{role="traverse"}' 1 +wait_ready $PORT1 'thread_work_pending{role="scan"}' 0 +wait_ready $PORT1 'thread_busy{role="scan"}' 0 + +# Confirm that the hello3 files are excluded + +wait_ready $PORT1 'scanned_files_total{source=".rpm archive"}' $rpms +wait_ready $PORT1 'scanned_files_total{source=".tar.xz archive"}' $tarxz + + +kill $PID1 +wait $PID1 +PID1=0 + +# Start second debuginfod, with exclusion of everything + +echo 'RERUN WITH -r and -X everything' >> vlog$PORT1 + +env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -d $DB -r -X '.*' -F -R -Z .tar.xz -Z .tar.bz2=bzcat -p $PORT1 -t0 -g0 -v R Z >> vlog$PORT1 2>&1 & +PID1=$! +tempfiles vlog$PORT1 +errfiles vlog$PORT1 +# Server must become ready +wait_ready $PORT1 'ready' 1 + +wait_ready $PORT1 'thread_work_total{role="groom"}' 1 +wait_ready $PORT1 'thread_work_pending{role="scan"}' 0 +wait_ready $PORT1 'thread_busy{role="scan"}' 0 + +wait_ready $PORT1 'groomed_total{decision="stale"}' $groomed +wait_ready $PORT1 'groom{statistic="files scanned (#)"}' 0 + +kill $PID1 +wait $PID1 +PID1=0 + +exit 0