From patchwork Thu May 9 15:15:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Marchi X-Patchwork-Id: 32620 Received: (qmail 38881 invoked by alias); 9 May 2019 15:16:02 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Delivered-To: mailing list gdb-patches@sourceware.org Received: (qmail 38868 invoked by uid 89); 9 May 2019 15:16:02 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-20.4 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.1 spammy=Nice, bonjour, specifying X-HELO: mail.efficios.com Received: from mail.efficios.com (HELO mail.efficios.com) (167.114.142.138) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 09 May 2019 15:15:59 +0000 Received: from localhost (ip6-localhost [IPv6:::1]) by mail.efficios.com (Postfix) with ESMTP id E1B451E29B1; Thu, 9 May 2019 11:15:57 -0400 (EDT) Received: from mail.efficios.com ([IPv6:::1]) by localhost (mail02.efficios.com [IPv6:::1]) (amavisd-new, port 10032) with ESMTP id AiuqV8139EsR; Thu, 9 May 2019 11:15:57 -0400 (EDT) Received: from localhost (ip6-localhost [IPv6:::1]) by mail.efficios.com (Postfix) with ESMTP id 33E741E29A5; Thu, 9 May 2019 11:15:57 -0400 (EDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.efficios.com 33E741E29A5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=efficios.com; s=default; t=1557414957; bh=BTQ5GdJK7MSiN58xrGlZdz/0xDOZx1yJJnd4EzbwWE0=; h=To:From:Message-ID:Date:MIME-Version; b=X9QX6H5ESO1XqydL25b1qF5IKBJzWMj6iid8wCDVVDdhxankzUDstO4P4SiFzI8g4 Q5clcLKe90tw6w3cxV7eyfK9bMAvtxXZ8gU9ZONmMQ+qNl1E2oLXzL/Dm0jZW0PsUT hFhBavvWaiER4o7daan8xlfy6aFKLJBs44IRlBu5uI0ULD8caHgrkD/GvIGW7AXaCR 7HCrHDb9e7ZPHSKBVbrgYq/y9FNBH+Hhxtqpjlww310KTum2yJQxp3TR4sz4x3y0V1 zcJ74PCKDY9LGZYuljkW2xRO+c3/0xJBvPEOvhYLhysd7Ik0PBk3BlT0Z7otm8zqCb xCBmdrLBS+7nQ== Received: from mail.efficios.com ([IPv6:::1]) by localhost (mail02.efficios.com [IPv6:::1]) (amavisd-new, port 10026) with ESMTP id MIQdZR5LmUka; Thu, 9 May 2019 11:15:57 -0400 (EDT) Received: from [172.16.0.120] (192-222-157-41.qc.cable.ebox.net [192.222.157.41]) by mail.efficios.com (Postfix) with ESMTPSA id 0DE991E299E; Thu, 9 May 2019 11:15:57 -0400 (EDT) Subject: Re: [PATCH] cc-with-tweaks: show dwz stderr and check exit code To: Tom de Vries , gdb-patches@sourceware.org Cc: Jakub Jelinek References: <20190508160012.32596-1-simon.marchi@efficios.com> From: Simon Marchi Message-ID: <0ba3b8fa-18d9-65bf-13ff-53f1baf69868@efficios.com> Date: Thu, 9 May 2019 11:15:56 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: On 2019-05-09 7:25 a.m., Tom de Vries wrote: > On 08-05-19 18:00, Simon Marchi wrote: >> When running the gdb.base/index-cache.exp test case with the >> cc-with-dwz-m board, I noticed that the final executable didn't actually >> contain a .gnu_debugaltlink section with the name of the external dwz >> file: >> >> $ readelf --debug-dump=links testsuite/outputs/gdb.base/index-cache/index-cache >> * empty * >> >> Running dwz by hand, I realized it's because dwz complains that the >> output .debug_info section is empty and fails: >> >> $ gcc ~/src/binutils-gdb/gdb/testsuite/gdb.base/index-cache.c -g3 -O0 -o a && cp a b >> $ dwz -m foo a b >> dwz: foo: .debug_info section not present >> $ echo $? >> 1 >> >> This is because index-cache.c is trivial (just an empty main) and dwz >> doesn't find anything to factor out to the dwz file. [1] >> >> I think that cc-with-tweaks should fail in this scenario: > > >> if the user >> asks for an external dwz file to be generated (the -m flag), > > This dwz semantics is open for debate, I think. > > In the case of dwz a, the user asks for a to be compressed, and if that > doesn't happen due to size heuristics, then execution is still > considered a success. > > In the case of dwz a -o b, the user asks for b to be generated, and > failure to generate b (f.i. due to size heuristics) is considered an > error. [ Which I guess is similar to the -o option of many other Linux > commands. ]> > ISTM that the -m option could be interpreted either way. And I prefer > the interpretation of -m as: generate an external dwz file, if beneficial. It would be interesting to get a particular exit code to mean it worked but "I chose not to change anything", so the caller can differentiate that from other kinds of errors. I just noticed something odd, when specifying input files that don't exist, dwz exits with 0: $ dwz -m foo hello bonjour dwz: Failed to open input file hello: No such file or directory dwz: Failed to open input file bonjour: No such file or directory dwz: Too few files for multifile optimization $ echo $? 0 In this case, shouldn't it exit with some non-0 code, as it's clearly an error? >> then it >> should be an error if cc-with-tweaks doesn't manage to produce an >> executable with the proper link to this external dwz file. Otherwise, >> the test runs with a regular non-dwzified executable, which gives a >> false sense of security about whether the feature under test works with >> dwzified executables. >> > > Agreed. Failing in cc-with-tweaks.sh in this case will list the > test-case as UNTESTED, which is a fair representation. > > [ And at some point we can add support in cc-with-tweaks.sh for dwz > --devel-ignore-size (a switch only supported if dwz is compiled with > -DDEVEL) which makes it more likely to transform the input file and > generate an -m file, even if it's not beneficial, which will mean fewer > UNTESTED test-cases. ] Indeed, if we can probe that the dwz executable in use has this feature, we can use it. > > [ nitpick: And coming back to the false sense of security: there's no > guarantee that the DWARF that is intended to be tested by a particular > test-case is in fact transformed by dwz, even if the executable as a > whole is indeed transformed. So running the test-suite with dwz does not > guarantee that the feature under test works with dwzified executables. ] So I guess that if dwz had a particular exit code to indicate that it chose not to change the DWARF because it's not beneficial, we could decide that it's equivalent to a compilation failure in cc-with-tweaks.sh. As you said, it will result in UNTESTED, which we agree is better than letting the user think that the test actually ran with a dwzified executable. It would also be pointless to continue running the test, as it would test the exact same thing as what is tested using the default unix board. Other users of dwz could consider that exit code of dwz as a success and carry on. > >> So this patch adds checks for that after invoking dwz. It also removes >> the 2>&1 to allow the error message to be printed like so: >> >> Running /home/smarchi/src/binutils-gdb/gdb/testsuite/gdb.base/index-cache.exp ... >> gdb compile failed, dwz: /home/smarchi/build/binutils-gdb/gdb/testsuite/outputs/gdb.base/index-cache/index-cache.dwz: .debug_info section not present >> > > Nice. I've used your patch to test, and because of this more verbose > behaviour I already noticed a new error message ("Error mmapping > multi-file temporary files"), and fixed it in dwz. Heh, I just saw that for the first time, thanks for fixing it! >> gdb/contrib/cc-with-tweaks.sh | 8 ++++++-- >> 1 file changed, 6 insertions(+), 2 deletions(-) >> >> diff --git a/gdb/contrib/cc-with-tweaks.sh b/gdb/contrib/cc-with-tweaks.sh >> index 47379cc15875..366b90918c18 100755 >> --- a/gdb/contrib/cc-with-tweaks.sh >> +++ b/gdb/contrib/cc-with-tweaks.sh >> @@ -180,10 +180,14 @@ if [ "$want_index" = true ]; then >> fi >> >> if [ "$want_dwz" = true ]; then >> - $DWZ "$output_file" > /dev/null 2>&1 >> + $DWZ "$output_file" > /dev/null >> + rc=$? >> + [ $rc != 0 ] && exit $rc >> elif [ "$want_multi" = true ]; then >> cp $output_file ${output_file}.alt >> - $DWZ -m ${output_file}.dwz "$output_file" ${output_file}.alt > /dev/null 2>&1 >> + $DWZ -m ${output_file}.dwz "$output_file" ${output_file}.alt > /dev/null >> + rc=$? >> + [ $rc != 0 ] && exit $rc >> rm -f ${output_file}.alt >> fi > > I wonder if it's more reliable to test for presence of -m file rather > than exit status (and likewise, we could copy and compare for the > want_dwz case). Good point. Why not check both actually (exit code and presence of the -m file)? I did this change in the updated patch lower, tell me if you think it makes sense. What do you mean by "we could copy and compare for the want_dwz case"? No new file is created in that case, so what should we test for? > > Also, I propose to run dwz with -q, to inhibit the countless "not > beneficial" messages. It's good to list these cases as untested, but in > the log we'd rather skip the uninteresting messages, to increase the > likelyhood that interesting messages get spotted. Hmm, since this is the reason the test was not ran (it's akin to a compilation failure), I think it's important to show it. Someone could search for a very long time why the test is not running if there's not error message. Here's a new version of the patch updated to check for the presence of the dwz file. From 1e6140468c42aec96058357d64d5557267cface3 Mon Sep 17 00:00:00 2001 From: Simon Marchi Date: Tue, 7 May 2019 14:14:54 -0400 Subject: [PATCH] cc-with-tweaks: show dwz stderr and check exit code When running the gdb.base/index-cache.exp test case with the cc-with-dwz-m board, I noticed that the final executable didn't actually contain a .gnu_debugaltlink section with the name of the external dwz file: $ readelf --debug-dump=links testsuite/outputs/gdb.base/index-cache/index-cache * empty * Running dwz by hand, I realized it's because dwz complains that the output .debug_info section is empty and fails: $ gcc ~/src/binutils-gdb/gdb/testsuite/gdb.base/index-cache.c -g3 -O0 -o a && cp a b $ dwz -m foo a b dwz: foo: .debug_info section not present $ echo $? 1 This is because index-cache.c is trivial (just an empty main) and dwz doesn't find anything to factor out to the dwz file. [1] I think that cc-with-tweaks should fail in this scenario: if the user asks for an external dwz file to be generated (the -m flag), then it should be an error if cc-with-tweaks doesn't manage to produce an executable with the proper link to this external dwz file. Otherwise, the test runs with a regular non-dwzified executable, which gives a false sense of security about whether the feature under test works with dwzified executables. So this patch adds checks for that after invoking dwz. It also removes the 2>&1 to allow the error message to be printed like so: Running /home/smarchi/src/binutils-gdb/gdb/testsuite/gdb.base/index-cache.exp ... gdb compile failed, dwz: /home/smarchi/build/binutils-gdb/gdb/testsuite/outputs/gdb.base/index-cache/index-cache.dwz: .debug_info section not present We also check for the presence of the expected output file in the dwz -m case. With this patch, fewer tests will pass than before with the cc-with-dwz and cc-with-dwz-m boards, but those were false positives anyway. [1] Note that dwz has been patched by Tom de Vries to work correctly in this case, so we can use dwz master to run the test: https://sourceware.org/git/?p=dwz.git;a=commit;h=08becc8b33453b6d013a65e7eeae57fc1881e801 gdb/ChangeLog: * contrib/cc-with-tweaks.sh: Check for return value of dwz. --- gdb/contrib/cc-with-tweaks.sh | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/gdb/contrib/cc-with-tweaks.sh b/gdb/contrib/cc-with-tweaks.sh index 47379cc15875..fcfdbd0589cf 100755 --- a/gdb/contrib/cc-with-tweaks.sh +++ b/gdb/contrib/cc-with-tweaks.sh @@ -180,10 +180,20 @@ if [ "$want_index" = true ]; then fi if [ "$want_dwz" = true ]; then - $DWZ "$output_file" > /dev/null 2>&1 + $DWZ "$output_file" > /dev/null + rc=$? + [ $rc != 0 ] && exit $rc elif [ "$want_multi" = true ]; then cp $output_file ${output_file}.alt - $DWZ -m ${output_file}.dwz "$output_file" ${output_file}.alt > /dev/null 2>&1 + $DWZ -m ${output_file}.dwz "$output_file" ${output_file}.alt > /dev/null + rc=$? + [ $rc != 0 ] && exit $rc + + if [ ! -f "${output_file}.dwz" ]; then + echo "$myname: dwz file ${output_file}.dwz missing." + exit 1 + fi + rm -f ${output_file}.alt fi