| Message ID | 20150606105711.GA7215@domone |
|---|---|
| State | New, archived |
| Headers |
Received: (qmail 130584 invoked by alias); 6 Jun 2015 10:57:33 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: <libc-alpha.sourceware.org> List-Unsubscribe: <mailto:libc-alpha-unsubscribe-##L=##H@sourceware.org> List-Subscribe: <mailto:libc-alpha-subscribe@sourceware.org> List-Archive: <http://sourceware.org/ml/libc-alpha/> List-Post: <mailto:libc-alpha@sourceware.org> List-Help: <mailto:libc-alpha-help@sourceware.org>, <http://sourceware.org/ml/#faqs> Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 130571 invoked by uid 89); 6 Jun 2015 10:57:32 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-0.5 required=5.0 tests=AWL, BAYES_00, FREEMAIL_FROM, KAM_LOTSOFHASH, SPF_NEUTRAL autolearn=no version=3.3.2 X-HELO: popelka.ms.mff.cuni.cz Date: Sat, 6 Jun 2015 12:57:11 +0200 From: =?utf-8?B?T25kxZllaiBCw61sa2E=?= <neleai@seznam.cz> To: Carlos O'Donell <carlos@redhat.com> Cc: libc-alpha@sourceware.org Subject: Wordexp benchtest: good, bad and unreliable. Message-ID: <20150606105711.GA7215@domone> References: <20150513124945.GA4067@domone> <55538A90.9000908@redhat.com> <20150513183554.GA31157@domone> <55539E20.4010506@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <55539E20.4010506@redhat.com> User-Agent: Mutt/1.5.20 (2009-06-14) |
Commit Message
Ondrej Bilka
June 6, 2015, 10:57 a.m. UTC
On Wed, May 13, 2015 at 02:55:28PM -0400, Carlos O'Donell wrote: > On 05/13/2015 02:35 PM, Ondřej Bílka wrote: > > On Wed, May 13, 2015 at 01:32:00PM -0400, Carlos O'Donell wrote: > >> On 05/13/2015 08:49 AM, Ondřej Bílka wrote: > >>> Hi, as I read Paul's wordexp patch I found that you use inefficient > >>> idiom. Checking character membership with strchr is slow due to startup > >>> cost. Much better is just use table lookup. > >> > >> How did you test it was faster? > >> > > You don't need to look at barometer to see if its raining. It does > > single memory access which takes around 5-6 cycles is faster than strchr that takes ~30 cycles > > and cannot be faster as it also needs to do a memory access. > > > > > >> Could you please add a wordexp microbenchmark to show the gains? > >> > > I could. Its easy to make microbenchmark that shows improved performance. > > Also its easy to make microbenchmark that makes it a regression. Just > > use 128 byte IFS env. variable and call wordexp("x", foo, 0) > > I could also make microbenchmark that is inconclusive as performance > > difference is lost in syscall overhead of finding filenames by glob. > > And those would all represent various workloads which we care about? > > I'm fine having 3 different wordexp microbenchmarks. > > > So I wont make microbenchmark as it would be useless exercise for pretty > > obvious case. > > OK. > > > You would need runtime profiling to get something useful. > > I don't agree with that. I think a microbenchmark of wordexp would be useful > as a regression test for this case. > > Cheers, > Carlos. > As I mentioned before for wordexp performance depends on too many parameters. So here is microbenchmark that shows some improvements and some regressions. There is lot of noise. I ran these three times and on one * and ffffffffoo pattern become twice slower for no reason. So how do you decide with conflicted data like this? old implementation wordexp Pattern foo flags 0: 1081.77 Pattern ffoo flags 0: 621.516 Pattern ffffoo flags 0: 697.562 Pattern ffffffffoo flags 0: 867.922 Pattern ffffffffffffffffoo flags 0: 1110.75 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1567.92 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2541.03 Pattern * flags 0: 141070 Pattern foo flags 0: 1061.08 Pattern ffoo flags 0: 1190.48 Pattern ffffoo flags 0: 1146.83 Pattern ffffffffoo flags 0: 1407.77 Pattern ffffffffffffffffoo flags 0: 1734.33 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2143.02 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3100.41 Pattern * flags 0: 205510 wordexp Pattern foo flags 0: 1081.84 Pattern ffoo flags 0: 672.422 Pattern ffffoo flags 0: 704.828 Pattern ffffffffoo flags 0: 875.188 Pattern ffffffffffffffffoo flags 0: 1089.7 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1594.03 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2575.28 Pattern * flags 0: 144160 Pattern foo flags 0: 1062.45 Pattern ffoo flags 0: 1099.72 Pattern ffffoo flags 0: 1153.95 Pattern ffffffffoo flags 0: 1391.45 Pattern ffffffffffffffffoo flags 0: 1820.36 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2129 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3171.5 Pattern * flags 0: 208475 wordexp Pattern foo flags 0: 1086.2 Pattern ffoo flags 0: 615.078 Pattern ffffoo flags 0: 688.594 Pattern ffffffffoo flags 0: 851.984 Pattern ffffffffffffffffoo flags 0: 1085.48 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1562.95 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2540.08 Pattern * flags 0: 287636 Pattern foo flags 0: 1068.11 Pattern ffoo flags 0: 1092.5 Pattern ffffoo flags 0: 1143.7 Pattern ffffffffoo flags 0: 1380.66 Pattern ffffffffffffffffoo flags 0: 1763.88 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2165.72 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3095.41 Pattern * flags 0: 207373 new wordexp Pattern foo flags 0: 1161.95 Pattern ffoo flags 0: 631.016 Pattern ffffoo flags 0: 714.797 Pattern ffffffffoo flags 0: 2209.84 Pattern ffffffffffffffffoo flags 0: 1024.17 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1408.22 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2223.39 Pattern * flags 0: 146925 Pattern foo flags 0: 1660.66 Pattern ffoo flags 0: 1673.34 Pattern ffffoo flags 0: 1730.23 Pattern ffffffffoo flags 0: 2030.42 Pattern ffffffffffffffffoo flags 0: 2245.59 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2512.95 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3312.95 Pattern * flags 0: 210492 wordexp Pattern foo flags 0: 1184.19 Pattern ffoo flags 0: 634.062 Pattern ffffoo flags 0: 683.688 Pattern ffffffffoo flags 0: 826.891 Pattern ffffffffffffffffoo flags 0: 1020.27 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1405.89 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2185.44 Pattern * flags 0: 140257 Pattern foo flags 0: 1649.77 Pattern ffoo flags 0: 1668.39 Pattern ffffoo flags 0: 1722.23 Pattern ffffffffoo flags 0: 1924.27 Pattern ffffffffffffffffoo flags 0: 2322.56 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2502.8 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3306.39 Pattern * flags 0: 208003 wordexp Pattern foo flags 0: 1097.7 Pattern ffoo flags 0: 655.891 Pattern ffffoo flags 0: 692.453 Pattern ffffffffoo flags 0: 860.203 Pattern ffffffffffffffffoo flags 0: 1009.38 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1426.02 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2179.64 Pattern * flags 0: 144492 Pattern foo flags 0: 1668.47 Pattern ffoo flags 0: 1684.38 Pattern ffffoo flags 0: 1718.75 Pattern ffffffffoo flags 0: 1932.91 Pattern ffffffffffffffffoo flags 0: 2294.95 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2514.89 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3329.14 Pattern * flags 0: 211152
Comments
So Carlos whats your comment on benchmark results? On Sat, Jun 06, 2015 at 12:57:11PM +0200, Ondřej Bílka wrote: > On Wed, May 13, 2015 at 02:55:28PM -0400, Carlos O'Donell wrote: > > On 05/13/2015 02:35 PM, Ondřej Bílka wrote: > > > On Wed, May 13, 2015 at 01:32:00PM -0400, Carlos O'Donell wrote: > > >> On 05/13/2015 08:49 AM, Ondřej Bílka wrote: > > >>> Hi, as I read Paul's wordexp patch I found that you use inefficient > > >>> idiom. Checking character membership with strchr is slow due to startup > > >>> cost. Much better is just use table lookup. > > >> > > >> How did you test it was faster? > > >> > > > You don't need to look at barometer to see if its raining. It does > > > single memory access which takes around 5-6 cycles is faster than strchr that takes ~30 cycles > > > and cannot be faster as it also needs to do a memory access. > > > > > > > > >> Could you please add a wordexp microbenchmark to show the gains? > > >> > > > I could. Its easy to make microbenchmark that shows improved performance. > > > Also its easy to make microbenchmark that makes it a regression. Just > > > use 128 byte IFS env. variable and call wordexp("x", foo, 0) > > > I could also make microbenchmark that is inconclusive as performance > > > difference is lost in syscall overhead of finding filenames by glob. > > > > And those would all represent various workloads which we care about? > > > > I'm fine having 3 different wordexp microbenchmarks. > > > > > So I wont make microbenchmark as it would be useless exercise for pretty > > > obvious case. > > > > OK. > > > > > You would need runtime profiling to get something useful. > > > > I don't agree with that. I think a microbenchmark of wordexp would be useful > > as a regression test for this case. > > > > Cheers, > > Carlos. > > > > As I mentioned before for wordexp performance depends on too many > parameters. So here is microbenchmark that shows some improvements and > some regressions. There is lot of noise. I ran these three times and on > one * and ffffffffoo pattern become twice slower for no reason. > > So how do you decide with conflicted data like this? > > > old implementation > wordexp > Pattern foo flags 0: 1081.77 > Pattern ffoo flags 0: 621.516 > Pattern ffffoo flags 0: 697.562 > Pattern ffffffffoo flags 0: 867.922 > Pattern ffffffffffffffffoo flags 0: 1110.75 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1567.92 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2541.03 > Pattern * flags 0: 141070 > Pattern foo flags 0: 1061.08 > Pattern ffoo flags 0: 1190.48 > Pattern ffffoo flags 0: 1146.83 > Pattern ffffffffoo flags 0: 1407.77 > Pattern ffffffffffffffffoo flags 0: 1734.33 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2143.02 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3100.41 > Pattern * flags 0: 205510 > > > wordexp > Pattern foo flags 0: 1081.84 > Pattern ffoo flags 0: 672.422 > Pattern ffffoo flags 0: 704.828 > Pattern ffffffffoo flags 0: 875.188 > Pattern ffffffffffffffffoo flags 0: 1089.7 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1594.03 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2575.28 > Pattern * flags 0: 144160 > Pattern foo flags 0: 1062.45 > Pattern ffoo flags 0: 1099.72 > Pattern ffffoo flags 0: 1153.95 > Pattern ffffffffoo flags 0: 1391.45 > Pattern ffffffffffffffffoo flags 0: 1820.36 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2129 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3171.5 > Pattern * flags 0: 208475 > > > wordexp > Pattern foo flags 0: 1086.2 > Pattern ffoo flags 0: 615.078 > Pattern ffffoo flags 0: 688.594 > Pattern ffffffffoo flags 0: 851.984 > Pattern ffffffffffffffffoo flags 0: 1085.48 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1562.95 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2540.08 > Pattern * flags 0: 287636 > Pattern foo flags 0: 1068.11 > Pattern ffoo flags 0: 1092.5 > Pattern ffffoo flags 0: 1143.7 > Pattern ffffffffoo flags 0: 1380.66 > Pattern ffffffffffffffffoo flags 0: 1763.88 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2165.72 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3095.41 > Pattern * flags 0: 207373 > > new > > > wordexp > Pattern foo flags 0: 1161.95 > Pattern ffoo flags 0: 631.016 > Pattern ffffoo flags 0: 714.797 > Pattern ffffffffoo flags 0: 2209.84 > Pattern ffffffffffffffffoo flags 0: 1024.17 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1408.22 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2223.39 > Pattern * flags 0: 146925 > Pattern foo flags 0: 1660.66 > Pattern ffoo flags 0: 1673.34 > Pattern ffffoo flags 0: 1730.23 > Pattern ffffffffoo flags 0: 2030.42 > Pattern ffffffffffffffffoo flags 0: 2245.59 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2512.95 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3312.95 > Pattern * flags 0: 210492 > > > wordexp > Pattern foo flags 0: 1184.19 > Pattern ffoo flags 0: 634.062 > Pattern ffffoo flags 0: 683.688 > Pattern ffffffffoo flags 0: 826.891 > Pattern ffffffffffffffffoo flags 0: 1020.27 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1405.89 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2185.44 > Pattern * flags 0: 140257 > Pattern foo flags 0: 1649.77 > Pattern ffoo flags 0: 1668.39 > Pattern ffffoo flags 0: 1722.23 > Pattern ffffffffoo flags 0: 1924.27 > Pattern ffffffffffffffffoo flags 0: 2322.56 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2502.8 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3306.39 > Pattern * flags 0: 208003 > > > wordexp > Pattern foo flags 0: 1097.7 > Pattern ffoo flags 0: 655.891 > Pattern ffffoo flags 0: 692.453 > Pattern ffffffffoo flags 0: 860.203 > Pattern ffffffffffffffffoo flags 0: 1009.38 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1426.02 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2179.64 > Pattern * flags 0: 144492 > Pattern foo flags 0: 1668.47 > Pattern ffoo flags 0: 1684.38 > Pattern ffffoo flags 0: 1718.75 > Pattern ffffffffoo flags 0: 1932.91 > Pattern ffffffffffffffffoo flags 0: 2294.95 > Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2514.89 > Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3329.14 > Pattern * flags 0: 211152 > > diff --git a/benchtests/Makefile b/benchtests/Makefile > index 8e615e5..fb737da 100644 > --- a/benchtests/Makefile > +++ b/benchtests/Makefile > @@ -35,7 +35,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \ > strcat strchr strchrnul strcmp strcpy strcspn strlen \ > strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \ > strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \ > - strcoll > + strcoll wordexp > string-bench-all := $(string-bench) > > # We have to generate locales > diff --git a/benchtests/bench-wordexp.c b/benchtests/bench-wordexp.c > new file mode 100644 > index 0000000..ffadbcc > --- /dev/null > +++ b/benchtests/bench-wordexp.c > @@ -0,0 +1,97 @@ > +/* Measure wordexp functions. > + Copyright (C) 2015 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <http://www.gnu.org/licenses/>. */ > + > +#define TEST_MAIN > +#define TEST_NAME "wordexp" > +#include "bench-string.h" > + > +#include <wordexp.h> > + > +typedef int (*proto_t) (const char *, wordexp_t *, int); > + > +IMPL (wordexp, 1) > + > + > +static void > +do_one_test (impl_t *impl, const char *s1, int flag) > +{ > + size_t i, iters = INNER_LOOP_ITERS; > + timing_t start, stop, cur; > + > + TIMING_NOW (start); > + for (i = 0; i < iters; ++i) > + { > + wordexp_t out; > + CALL (impl, s1, &out, flag); > + } > + TIMING_NOW (stop); > + > + TIMING_DIFF (cur, start, stop); > + > + TIMING_PRINT_MEAN ((double) cur, (double) iters); > +} > + > + > +static void > +do_test (const char *s1, int flag) > +{ > + > + printf ("Pattern %s flags %i:", s1, flag); > + > + FOR_EACH_IMPL (impl, 0) > + do_one_test (impl, s1, flag); > + > + putchar ('\n'); > +} > + > +static int > +test_main (void) > +{ > + test_init (); > + > + printf ("%23s", ""); > + FOR_EACH_IMPL (impl, 0) > + printf ("\t%s", impl->name); > + putchar ('\n'); > + > + do_test ("foo", 0); > + do_test ("ffoo", 0); > + do_test ("ffffoo", 0); > + do_test ("ffffffffoo", 0); > + do_test ("ffffffffffffffffoo", 0); > + do_test ("ffffffffffffffffffffffffffffffffoo", 0); > +do_test ("ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo", 0); > + > + > + do_test ("*", 0); > + setenv("IFS"," \ > + ", 1); > + > + do_test ("foo", 0); > + do_test ("ffoo", 0); > + do_test ("ffffoo", 0); > + do_test ("ffffffffoo", 0); > + do_test ("ffffffffffffffffoo", 0); > + do_test ("ffffffffffffffffffffffffffffffffoo", 0); > +do_test ("ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo", 0); > + do_test ("*", 0); > + > + return ret; > +} > + > +#include "../test-skeleton.c"
On 06/16/2015 01:32 PM, Ondřej Bílka wrote: >> As I mentioned before for wordexp performance depends on too many >> parameters. So here is microbenchmark that shows some improvements and >> some regressions. There is lot of noise. I ran these three times and on >> one * and ffffffffoo pattern become twice slower for no reason. There is a reason, but we just don't know it yet. >> So how do you decide with conflicted data like this? You can't make an "automatic" or "automated" decision, but as a baseline the statistics do help. With noisy benchmarks you can still look at potential trends in the noise. We don't want the bad patterns to get 5x slower. Similarly we will have conflicting workloads modeled as benchmarks, and the hard part is that as an expert we have to make a difficult choice. We have to understand the workload and decide "Yes, we ignore this performance decrease because we expect fewer people will suffer." However, in order to do that we need comments about exactly what workload we're trying to model. For most of our microbenchmarks the workloads are "raw performance of function call" and that's very simplistic, it's also very generic. >> old implementation >> wordexp >> Pattern foo flags 0: 1081.77 >> Pattern ffoo flags 0: 621.516 >> Pattern ffffoo flags 0: 697.562 >> Pattern ffffffffoo flags 0: 867.922 >> Pattern ffffffffffffffffoo flags 0: 1110.75 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1567.92 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2541.03 >> Pattern * flags 0: 141070 >> Pattern foo flags 0: 1061.08 >> Pattern ffoo flags 0: 1190.48 >> Pattern ffffoo flags 0: 1146.83 >> Pattern ffffffffoo flags 0: 1407.77 >> Pattern ffffffffffffffffoo flags 0: 1734.33 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2143.02 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3100.41 >> Pattern * flags 0: 205510 >> >> >> wordexp >> Pattern foo flags 0: 1081.84 >> Pattern ffoo flags 0: 672.422 >> Pattern ffffoo flags 0: 704.828 >> Pattern ffffffffoo flags 0: 875.188 >> Pattern ffffffffffffffffoo flags 0: 1089.7 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1594.03 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2575.28 >> Pattern * flags 0: 144160 >> Pattern foo flags 0: 1062.45 >> Pattern ffoo flags 0: 1099.72 >> Pattern ffffoo flags 0: 1153.95 >> Pattern ffffffffoo flags 0: 1391.45 >> Pattern ffffffffffffffffoo flags 0: 1820.36 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2129 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3171.5 >> Pattern * flags 0: 208475 >> >> >> wordexp >> Pattern foo flags 0: 1086.2 >> Pattern ffoo flags 0: 615.078 >> Pattern ffffoo flags 0: 688.594 >> Pattern ffffffffoo flags 0: 851.984 >> Pattern ffffffffffffffffoo flags 0: 1085.48 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1562.95 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2540.08 >> Pattern * flags 0: 287636 >> Pattern foo flags 0: 1068.11 >> Pattern ffoo flags 0: 1092.5 >> Pattern ffffoo flags 0: 1143.7 >> Pattern ffffffffoo flags 0: 1380.66 >> Pattern ffffffffffffffffoo flags 0: 1763.88 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2165.72 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3095.41 >> Pattern * flags 0: 207373 >> >> new >> >> >> wordexp >> Pattern foo flags 0: 1161.95 >> Pattern ffoo flags 0: 631.016 >> Pattern ffffoo flags 0: 714.797 >> Pattern ffffffffoo flags 0: 2209.84 >> Pattern ffffffffffffffffoo flags 0: 1024.17 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1408.22 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2223.39 >> Pattern * flags 0: 146925 >> Pattern foo flags 0: 1660.66 >> Pattern ffoo flags 0: 1673.34 >> Pattern ffffoo flags 0: 1730.23 >> Pattern ffffffffoo flags 0: 2030.42 >> Pattern ffffffffffffffffoo flags 0: 2245.59 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2512.95 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3312.95 >> Pattern * flags 0: 210492 >> >> >> wordexp >> Pattern foo flags 0: 1184.19 >> Pattern ffoo flags 0: 634.062 >> Pattern ffffoo flags 0: 683.688 >> Pattern ffffffffoo flags 0: 826.891 >> Pattern ffffffffffffffffoo flags 0: 1020.27 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1405.89 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2185.44 >> Pattern * flags 0: 140257 >> Pattern foo flags 0: 1649.77 >> Pattern ffoo flags 0: 1668.39 >> Pattern ffffoo flags 0: 1722.23 >> Pattern ffffffffoo flags 0: 1924.27 >> Pattern ffffffffffffffffoo flags 0: 2322.56 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2502.8 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3306.39 >> Pattern * flags 0: 208003 >> >> >> wordexp >> Pattern foo flags 0: 1097.7 >> Pattern ffoo flags 0: 655.891 >> Pattern ffffoo flags 0: 692.453 >> Pattern ffffffffoo flags 0: 860.203 >> Pattern ffffffffffffffffoo flags 0: 1009.38 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1426.02 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2179.64 >> Pattern * flags 0: 144492 >> Pattern foo flags 0: 1668.47 >> Pattern ffoo flags 0: 1684.38 >> Pattern ffffoo flags 0: 1718.75 >> Pattern ffffffffoo flags 0: 1932.91 >> Pattern ffffffffffffffffoo flags 0: 2294.95 >> Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2514.89 >> Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3329.14 >> Pattern * flags 0: 211152 >> >> diff --git a/benchtests/Makefile b/benchtests/Makefile >> index 8e615e5..fb737da 100644 >> --- a/benchtests/Makefile >> +++ b/benchtests/Makefile >> @@ -35,7 +35,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \ >> strcat strchr strchrnul strcmp strcpy strcspn strlen \ >> strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \ >> strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \ >> - strcoll >> + strcoll wordexp >> string-bench-all := $(string-bench) >> >> # We have to generate locales >> diff --git a/benchtests/bench-wordexp.c b/benchtests/bench-wordexp.c >> new file mode 100644 >> index 0000000..ffadbcc >> --- /dev/null >> +++ b/benchtests/bench-wordexp.c >> @@ -0,0 +1,97 @@ >> +/* Measure wordexp functions. >> + Copyright (C) 2015 Free Software Foundation, Inc. >> + This file is part of the GNU C Library. >> + >> + The GNU C Library is free software; you can redistribute it and/or >> + modify it under the terms of the GNU Lesser General Public >> + License as published by the Free Software Foundation; either >> + version 2.1 of the License, or (at your option) any later version. >> + >> + The GNU C Library is distributed in the hope that it will be useful, >> + but WITHOUT ANY WARRANTY; without even the implied warranty of >> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU >> + Lesser General Public License for more details. >> + >> + You should have received a copy of the GNU Lesser General Public >> + License along with the GNU C Library; if not, see >> + <http://www.gnu.org/licenses/>. */ >> + >> +#define TEST_MAIN >> +#define TEST_NAME "wordexp" >> +#include "bench-string.h" >> + >> +#include <wordexp.h> >> + >> +typedef int (*proto_t) (const char *, wordexp_t *, int); >> + >> +IMPL (wordexp, 1) >> + >> + >> +static void >> +do_one_test (impl_t *impl, const char *s1, int flag) >> +{ >> + size_t i, iters = INNER_LOOP_ITERS; >> + timing_t start, stop, cur; >> + >> + TIMING_NOW (start); >> + for (i = 0; i < iters; ++i) >> + { >> + wordexp_t out; >> + CALL (impl, s1, &out, flag); >> + } >> + TIMING_NOW (stop); >> + >> + TIMING_DIFF (cur, start, stop); >> + >> + TIMING_PRINT_MEAN ((double) cur, (double) iters); >> +} >> + >> + >> +static void >> +do_test (const char *s1, int flag) >> +{ >> + >> + printf ("Pattern %s flags %i:", s1, flag); >> + >> + FOR_EACH_IMPL (impl, 0) >> + do_one_test (impl, s1, flag); >> + >> + putchar ('\n'); >> +} >> + >> +static int >> +test_main (void) >> +{ >> + test_init (); >> + >> + printf ("%23s", ""); >> + FOR_EACH_IMPL (impl, 0) >> + printf ("\t%s", impl->name); >> + putchar ('\n'); >> + >> + do_test ("foo", 0); >> + do_test ("ffoo", 0); >> + do_test ("ffffoo", 0); >> + do_test ("ffffffffoo", 0); >> + do_test ("ffffffffffffffffoo", 0); >> + do_test ("ffffffffffffffffffffffffffffffffoo", 0); >> +do_test ("ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo", 0); This group of tests should have a comment about what's it's testing. What workload are we modeling? >> + >> + >> + do_test ("*", 0); >> + setenv("IFS"," \ >> + ", 1); >> + >> + do_test ("foo", 0); >> + do_test ("ffoo", 0); >> + do_test ("ffffoo", 0); >> + do_test ("ffffffffoo", 0); >> + do_test ("ffffffffffffffffoo", 0); >> + do_test ("ffffffffffffffffffffffffffffffffoo", 0); >> +do_test ("ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo", 0); >> + do_test ("*", 0); >> + Similarly with the IFS change. This is not a normal IFS change for an average workload. Usually one changes IFS to be a series of characters, or just tab, or just newline etc. >> + return ret; >> +} >> + >> +#include "../test-skeleton.c" > The benchmark looks OK to me modulo the added comments. Cheers, Carlos.
diff --git a/benchtests/Makefile b/benchtests/Makefile index 8e615e5..fb737da 100644 --- a/benchtests/Makefile +++ b/benchtests/Makefile @@ -35,7 +35,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \ strcat strchr strchrnul strcmp strcpy strcspn strlen \ strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \ strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \ - strcoll + strcoll wordexp string-bench-all := $(string-bench) # We have to generate locales diff --git a/benchtests/bench-wordexp.c b/benchtests/bench-wordexp.c new file mode 100644 index 0000000..ffadbcc --- /dev/null +++ b/benchtests/bench-wordexp.c @@ -0,0 +1,97 @@ +/* Measure wordexp functions. + Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#define TEST_MAIN +#define TEST_NAME "wordexp" +#include "bench-string.h" + +#include <wordexp.h> + +typedef int (*proto_t) (const char *, wordexp_t *, int); + +IMPL (wordexp, 1) + + +static void +do_one_test (impl_t *impl, const char *s1, int flag) +{ + size_t i, iters = INNER_LOOP_ITERS; + timing_t start, stop, cur; + + TIMING_NOW (start); + for (i = 0; i < iters; ++i) + { + wordexp_t out; + CALL (impl, s1, &out, flag); + } + TIMING_NOW (stop); + + TIMING_DIFF (cur, start, stop); + + TIMING_PRINT_MEAN ((double) cur, (double) iters); +} + + +static void +do_test (const char *s1, int flag) +{ + + printf ("Pattern %s flags %i:", s1, flag); + + FOR_EACH_IMPL (impl, 0) + do_one_test (impl, s1, flag); + + putchar ('\n'); +} + +static int +test_main (void) +{ + test_init (); + + printf ("%23s", ""); + FOR_EACH_IMPL (impl, 0) + printf ("\t%s", impl->name); + putchar ('\n'); + + do_test ("foo", 0); + do_test ("ffoo", 0); + do_test ("ffffoo", 0); + do_test ("ffffffffoo", 0); + do_test ("ffffffffffffffffoo", 0); + do_test ("ffffffffffffffffffffffffffffffffoo", 0); +do_test ("ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo", 0); + + + do_test ("*", 0); + setenv("IFS"," \ + ", 1); + + do_test ("foo", 0); + do_test ("ffoo", 0); + do_test ("ffffoo", 0); + do_test ("ffffffffoo", 0); + do_test ("ffffffffffffffffoo", 0); + do_test ("ffffffffffffffffffffffffffffffffoo", 0); +do_test ("ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo", 0); + do_test ("*", 0); + + return ret; +} + +#include "../test-skeleton.c"