Message ID | 5fd01995-2c7a-411d-cc03-58199ffa615c@simark.ca |
---|---|
State | New, archived |
Headers |
Received: (qmail 129755 invoked by alias); 10 May 2019 16:05:56 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: <gdb-patches.sourceware.org> List-Unsubscribe: <mailto:gdb-patches-unsubscribe-##L=##H@sourceware.org> List-Subscribe: <mailto:gdb-patches-subscribe@sourceware.org> List-Archive: <http://sourceware.org/ml/gdb-patches/> List-Post: <mailto:gdb-patches@sourceware.org> List-Help: <mailto:gdb-patches-help@sourceware.org>, <http://sourceware.org/ml/#faqs> Sender: gdb-patches-owner@sourceware.org Delivered-To: mailing list gdb-patches@sourceware.org Received: (qmail 129747 invoked by uid 89); 10 May 2019 16:05:56 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-18.3 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.1 spammy= X-HELO: simark.ca Received: from simark.ca (HELO simark.ca) (158.69.221.121) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 10 May 2019 16:05:54 +0000 Received: from [172.16.0.120] (192-222-157-41.qc.cable.ebox.net [192.222.157.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by simark.ca (Postfix) with ESMTPSA id 3C4D51E481; Fri, 10 May 2019 12:05:52 -0400 (EDT) Subject: Re: [PATCH] cc-with-tweaks: show dwz stderr and check exit code To: Tom de Vries <tdevries@suse.de>, Simon Marchi <simon.marchi@efficios.com>, gdb-patches@sourceware.org Cc: Jakub Jelinek <jakub@redhat.com> References: <20190508160012.32596-1-simon.marchi@efficios.com> <d14c31af-e97e-472c-831f-a13aacfc7ed3@suse.de> <0ba3b8fa-18d9-65bf-13ff-53f1baf69868@efficios.com> <da75fb15-ec40-853e-fa98-bcb3a44f21d0@suse.de> From: Simon Marchi <simark@simark.ca> Message-ID: <5fd01995-2c7a-411d-cc03-58199ffa615c@simark.ca> Date: Fri, 10 May 2019 12:05:51 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <da75fb15-ec40-853e-fa98-bcb3a44f21d0@suse.de> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit |
Commit Message
Simon Marchi
May 10, 2019, 4:05 p.m. UTC
On 2019-05-10 5:03 a.m., Tom de Vries wrote: > That's PR24301 - "dwz fails to return EXIT_FAILURE when processing two > files" ( https://sourceware.org/bugzilla/show_bug.cgi?id=24301 ). I've > just committed a fix. Thanks! >> Good point. Why not check both actually (exit code and presence of the -m file)? >> I did this change in the updated patch lower, tell me if you think it makes sense. > > The rationale for checking for file changed in want_dwz mode and > multifile presence in want_multi mode is because these are measures that > are more reliable than the exit code, and will get better results for > people using older dwz. > > Checking for both goes against that rationale. Ok, I understand your point for older dwzs. >> What do you mean by "we could copy and compare for the want_dwz case"? No new >> file is created in that case, so what should we test for? >> > > I meant: > ... > if [ "$want_dwz" = true ]; then > cp $output_file ${output_file}.copy > $DWZ "$output_file" > /dev/null > if cmp "$output_file" "$output_file.copy"; rc=$?; then > true > fi > rm -f ${output_file}.copy > if [ $rc == 0 ]; then exit 1; fi Ah, interesting. As you said earlier, this doesn't guarantee that the DWARF has been modified, it could be something else in the executable that was changed. But still, it should catch a number of cases where dwz doesn't touch the executable at all. I have included your suggestion in the patch below and made a few adjustments. By the way, do you have a way to reproduce the case where dwz doesn't optimize anything in single file mode? Even with a file with just an empty main, dwz manages to do some optimization. Running dwz twice on the same executable is a way to get "DWARF compression not beneficial", but this shouldn't happen in cc-with-tweaks. Another question: in multifile mode, we check if the output dwz file exists. Should we also check that the original file now contains a .gnu_debugaltlink section? If dwz succeeded to generated the external dwz file, but failed to modify the original executable, we could also end up running a test with a non-dwzified executable. From ae61facce0dd0a480e265be1728fb1741b886180 Mon Sep 17 00:00:00 2001 From: Simon Marchi <simon.marchi@efficios.com> Date: Tue, 7 May 2019 14:14:54 -0400 Subject: [PATCH] cc-with-tweaks: show dwz stderr and verify result When running the gdb.base/index-cache.exp test case with the cc-with-dwz-m board, I noticed that the final executable didn't actually contain a .gnu_debugaltlink section with the name of the external dwz file: $ readelf --debug-dump=links testsuite/outputs/gdb.base/index-cache/index-cache * empty * Running dwz by hand, I realized it's because dwz complains that the output .debug_info section is empty and fails: $ gcc ~/src/binutils-gdb/gdb/testsuite/gdb.base/index-cache.c -g3 -O0 -o a && cp a b $ dwz -m foo a b dwz: foo: .debug_info section not present $ echo $? 1 This is because index-cache.c is trivial (just an empty main) and dwz doesn't find anything to factor out to the dwz file. [1] I think that cc-with-tweaks should fail in this scenario: if the user asks for an external dwz file to be generated (the -m flag), then it should be an error if cc-with-tweaks doesn't manage to produce an executable with the proper link to this external dwz file. Otherwise, the test runs with a regular non-dwzified executable, which gives a false sense of security about whether the feature under test works with dwzified executables. So this patch adds checks for that after invoking dwz. It also removes the 2>&1 to allow the error message to be printed like so: Running /home/smarchi/src/binutils-gdb/gdb/testsuite/gdb.base/index-cache.exp ... gdb compile failed, dwz: /home/smarchi/build/binutils-gdb/gdb/testsuite/outputs/gdb.base/index-cache/index-cache.dwz: .debug_info section not present - In the -m case (multi-file compression), we check if the expected output file exists. - In the -z case (single-file compression), we check if the file contents has changed. This should catch cases where dwz doesn't modify the file because it's not worth it. It was chosen not to check for dwz's exit code, as it is not very reliable up to dwz 0.12. With this patch, fewer tests will pass than before with the cc-with-dwz and cc-with-dwz-m boards, but those were false positives anyway, as the test ran with regular executables. [1] Note that dwz has been patched by Tom de Vries to work correctly in this case, so we can use dwz master to run the test: https://sourceware.org/git/?p=dwz.git;a=commit;h=08becc8b33453b6d013a65e7eeae57fc1881e801 gdb/ChangeLog: * contrib/cc-with-tweaks.sh: Validate dwz's work. --- gdb/contrib/cc-with-tweaks.sh | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-)
Comments
On 10-05-19 18:05, Simon Marchi wrote: > On 2019-05-10 5:03 a.m., Tom de Vries wrote: >> That's PR24301 - "dwz fails to return EXIT_FAILURE when processing two >> files" ( https://sourceware.org/bugzilla/show_bug.cgi?id=24301 ). I've >> just committed a fix. > > Thanks! > >>> Good point. Why not check both actually (exit code and presence of the -m file)? >>> I did this change in the updated patch lower, tell me if you think it makes sense. >> >> The rationale for checking for file changed in want_dwz mode and >> multifile presence in want_multi mode is because these are measures that >> are more reliable than the exit code, and will get better results for >> people using older dwz. >> >> Checking for both goes against that rationale. > > Ok, I understand your point for older dwzs. > >>> What do you mean by "we could copy and compare for the want_dwz case"? No new >>> file is created in that case, so what should we test for? >>> >> >> I meant: >> ... >> if [ "$want_dwz" = true ]; then >> cp $output_file ${output_file}.copy >> $DWZ "$output_file" > /dev/null >> if cmp "$output_file" "$output_file.copy"; rc=$?; then >> true >> fi >> rm -f ${output_file}.copy >> if [ $rc == 0 ]; then exit 1; fi > > Ah, interesting. As you said earlier, this doesn't guarantee that the DWARF has been > modified, it could be something else in the executable that was changed. But still, it > should catch a number of cases where dwz doesn't touch the executable at all. > The point I was trying to make, is that in a test-case there may be dwarf related to the test-case, and other, run-of-the-mill dwarf that is not at all related to the feature being tested, and when dwz has changed the executable, there's no way of knowing which of the two, or both has been affected. So, testing a test-case in combination with dwz, and seeing that the dwarf has been changed by dwz, does not guarantee you that the dwarf related to the feature being tested has in fact been changed. > I have included your suggestion in the patch below and made a few adjustments. > > By the way, do you have a way to reproduce the case where dwz doesn't optimize anything > in single file mode? Even with a file with just an empty main, dwz manages to do some > optimization. > I guess if you make a start.c with an empty _start function and compile with -stdlib (so, the start testcase in the dwz testsuite), and then furhter prune down the generated .s file, you'll end up with triggering that warning. > Running dwz twice on the same executable is a way to get "DWARF compression not beneficial", > but this shouldn't happen in cc-with-tweaks. > Ack. > Another question: in multifile mode, we check if the output dwz file exists. Should we also > check that the original file now contains a .gnu_debugaltlink section? If dwz succeeded to > generated the external dwz file, but failed to modify the original executable, we could also > end up running a test with a non-dwzified executable. > Agreed, we could do that, but I haven't seen any problems with dwz related to that, so I'm not sure it's worth the churn atm. > > From ae61facce0dd0a480e265be1728fb1741b886180 Mon Sep 17 00:00:00 2001 > From: Simon Marchi <simon.marchi@efficios.com> > Date: Tue, 7 May 2019 14:14:54 -0400 > Subject: [PATCH] cc-with-tweaks: show dwz stderr and verify result > > When running the gdb.base/index-cache.exp test case with the > cc-with-dwz-m board, I noticed that the final executable didn't actually > contain a .gnu_debugaltlink section with the name of the external dwz > file: > > $ readelf --debug-dump=links testsuite/outputs/gdb.base/index-cache/index-cache > * empty * > > Running dwz by hand, I realized it's because dwz complains that the > output .debug_info section is empty and fails: > > $ gcc ~/src/binutils-gdb/gdb/testsuite/gdb.base/index-cache.c -g3 -O0 -o a && cp a b > $ dwz -m foo a b > dwz: foo: .debug_info section not present > $ echo $? > 1 > > This is because index-cache.c is trivial (just an empty main) and dwz > doesn't find anything to factor out to the dwz file. [1] > > I think that cc-with-tweaks should fail in this scenario: if the user > asks for an external dwz file to be generated (the -m flag), then it > should be an error if cc-with-tweaks doesn't manage to produce an > executable with the proper link to this external dwz file. Otherwise, > the test runs with a regular non-dwzified executable, which gives a > false sense of security about whether the feature under test works with > dwzified executables. > > So this patch adds checks for that after invoking dwz. It also removes > the 2>&1 to allow the error message to be printed like so: > > Running /home/smarchi/src/binutils-gdb/gdb/testsuite/gdb.base/index-cache.exp ... > gdb compile failed, dwz: /home/smarchi/build/binutils-gdb/gdb/testsuite/outputs/gdb.base/index-cache/index-cache.dwz: .debug_info section not present > > - In the -m case (multi-file compression), we check if the expected output file > exists. > - In the -z case (single-file compression), we check if the file > contents has changed. This should catch cases where dwz doesn't modify the > file because it's not worth it. > > It was chosen not to check for dwz's exit code, as it is not very > reliable up to dwz 0.12. > > With this patch, fewer tests will pass than before with the > cc-with-dwz and cc-with-dwz-m boards, but those were false positives > anyway, as the test ran with regular executables. > > [1] Note that dwz has been patched by Tom de Vries to work correctly in > this case, so we can use dwz master to run the test: > > https://sourceware.org/git/?p=dwz.git;a=commit;h=08becc8b33453b6d013a65e7eeae57fc1881e801 > > gdb/ChangeLog: > > * contrib/cc-with-tweaks.sh: Validate dwz's work. > --- > gdb/contrib/cc-with-tweaks.sh | 34 ++++++++++++++++++++++++++++++++-- > 1 file changed, 32 insertions(+), 2 deletions(-) > > diff --git a/gdb/contrib/cc-with-tweaks.sh b/gdb/contrib/cc-with-tweaks.sh > index 47379cc15875..7df16bc6c1ca 100755 > --- a/gdb/contrib/cc-with-tweaks.sh > +++ b/gdb/contrib/cc-with-tweaks.sh > @@ -180,11 +180,41 @@ if [ "$want_index" = true ]; then > fi > > if [ "$want_dwz" = true ]; then > - $DWZ "$output_file" > /dev/null 2>&1 > + # Validate dwz's result by checking if the executable was modified. > + cp "$output_file" "${output_file}.copy" > + $DWZ "$output_file" > /dev/null > + cmp "$output_file" "$output_file.copy" > /dev/null > + cmp_rc=$? > + rm -f "${output_file}.copy" > + > + case $cmp_rc in > + 0) > + echo "$myname: dwz did not modify ${output_file}." > + exit 1 > + ;; > + 1) > + # File was modified, great. > + ;; > + *) > + # Other cmp error, it presumably has already printed something on > + # stderr. > + exit 1 > + ;; > + esac > elif [ "$want_multi" = true ]; then > + # Remove the dwz output file if it exists, so we don't mistake it for a > + # new file in case dwz fails. > + rm -f "${output_file}.dwz" > + > cp $output_file ${output_file}.alt > - $DWZ -m ${output_file}.dwz "$output_file" ${output_file}.alt > /dev/null 2>&1 > + $DWZ -m ${output_file}.dwz "$output_file" ${output_file}.alt > /dev/null > rm -f ${output_file}.alt > + > + # Validate dwz's work by checking if the expected output file exists. > + if [ ! -f "${output_file}.dwz" ]; then > + echo "$myname: dwz file ${output_file}.dwz missing." > + exit 1 > + fi > fi > > if [ "$want_dwp" = true ]; then > LGTM. Thanks, - Tom
On 2019-05-10 3:49 p.m., Tom de Vries wrote: > The point I was trying to make, is that in a test-case there may be > dwarf related to the test-case, and other, run-of-the-mill dwarf that is > not at all related to the feature being tested, and when dwz has changed > the executable, there's no way of knowing which of the two, or both has > been affected. So, testing a test-case in combination with dwz, and > seeing that the dwarf has been changed by dwz, does not guarantee you > that the dwarf related to the feature being tested has in fact been changed. Indeed. >> I have included your suggestion in the patch below and made a few adjustments. >> >> By the way, do you have a way to reproduce the case where dwz doesn't optimize anything >> in single file mode? Even with a file with just an empty main, dwz manages to do some >> optimization. >> > > I guess if you make a start.c with an empty _start function and compile > with -stdlib (so, the start testcase in the dwz testsuite), and then > furhter prune down the generated .s file, you'll end up with triggering > that warning. Ok, that's a bit further than I am willing to go right now to test the script :). >> Another question: in multifile mode, we check if the output dwz file exists. Should we also >> check that the original file now contains a .gnu_debugaltlink section? If dwz succeeded to >> generated the external dwz file, but failed to modify the original executable, we could also >> end up running a test with a non-dwzified executable. >> > > Agreed, we could do that, but I haven't seen any problems with dwz > related to that, so I'm not sure it's worth the churn atm. Ok. > LGTM. Thanks for all the comments, I am pushing the patch. Simon
diff --git a/gdb/contrib/cc-with-tweaks.sh b/gdb/contrib/cc-with-tweaks.sh index 47379cc15875..7df16bc6c1ca 100755 --- a/gdb/contrib/cc-with-tweaks.sh +++ b/gdb/contrib/cc-with-tweaks.sh @@ -180,11 +180,41 @@ if [ "$want_index" = true ]; then fi if [ "$want_dwz" = true ]; then - $DWZ "$output_file" > /dev/null 2>&1 + # Validate dwz's result by checking if the executable was modified. + cp "$output_file" "${output_file}.copy" + $DWZ "$output_file" > /dev/null + cmp "$output_file" "$output_file.copy" > /dev/null + cmp_rc=$? + rm -f "${output_file}.copy" + + case $cmp_rc in + 0) + echo "$myname: dwz did not modify ${output_file}." + exit 1 + ;; + 1) + # File was modified, great. + ;; + *) + # Other cmp error, it presumably has already printed something on + # stderr. + exit 1 + ;; + esac elif [ "$want_multi" = true ]; then + # Remove the dwz output file if it exists, so we don't mistake it for a + # new file in case dwz fails. + rm -f "${output_file}.dwz" + cp $output_file ${output_file}.alt - $DWZ -m ${output_file}.dwz "$output_file" ${output_file}.alt > /dev/null 2>&1 + $DWZ -m ${output_file}.dwz "$output_file" ${output_file}.alt > /dev/null rm -f ${output_file}.alt + + # Validate dwz's work by checking if the expected output file exists. + if [ ! -f "${output_file}.dwz" ]; then + echo "$myname: dwz file ${output_file}.dwz missing." + exit 1 + fi fi if [ "$want_dwp" = true ]; then