From patchwork Sat Jun 6 10:57:11 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ondrej Bilka X-Patchwork-Id: 7052 Received: (qmail 130584 invoked by alias); 6 Jun 2015 10:57:33 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 130571 invoked by uid 89); 6 Jun 2015 10:57:32 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-0.5 required=5.0 tests=AWL, BAYES_00, FREEMAIL_FROM, KAM_LOTSOFHASH, SPF_NEUTRAL autolearn=no version=3.3.2 X-HELO: popelka.ms.mff.cuni.cz Date: Sat, 6 Jun 2015 12:57:11 +0200 From: =?utf-8?B?T25kxZllaiBCw61sa2E=?= To: Carlos O'Donell Cc: libc-alpha@sourceware.org Subject: Wordexp benchtest: good, bad and unreliable. Message-ID: <20150606105711.GA7215@domone> References: <20150513124945.GA4067@domone> <55538A90.9000908@redhat.com> <20150513183554.GA31157@domone> <55539E20.4010506@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <55539E20.4010506@redhat.com> User-Agent: Mutt/1.5.20 (2009-06-14) On Wed, May 13, 2015 at 02:55:28PM -0400, Carlos O'Donell wrote: > On 05/13/2015 02:35 PM, Ondřej Bílka wrote: > > On Wed, May 13, 2015 at 01:32:00PM -0400, Carlos O'Donell wrote: > >> On 05/13/2015 08:49 AM, Ondřej Bílka wrote: > >>> Hi, as I read Paul's wordexp patch I found that you use inefficient > >>> idiom. Checking character membership with strchr is slow due to startup > >>> cost. Much better is just use table lookup. > >> > >> How did you test it was faster? > >> > > You don't need to look at barometer to see if its raining. It does > > single memory access which takes around 5-6 cycles is faster than strchr that takes ~30 cycles > > and cannot be faster as it also needs to do a memory access. > > > > > >> Could you please add a wordexp microbenchmark to show the gains? > >> > > I could. Its easy to make microbenchmark that shows improved performance. > > Also its easy to make microbenchmark that makes it a regression. Just > > use 128 byte IFS env. variable and call wordexp("x", foo, 0) > > I could also make microbenchmark that is inconclusive as performance > > difference is lost in syscall overhead of finding filenames by glob. > > And those would all represent various workloads which we care about? > > I'm fine having 3 different wordexp microbenchmarks. > > > So I wont make microbenchmark as it would be useless exercise for pretty > > obvious case. > > OK. > > > You would need runtime profiling to get something useful. > > I don't agree with that. I think a microbenchmark of wordexp would be useful > as a regression test for this case. > > Cheers, > Carlos. > As I mentioned before for wordexp performance depends on too many parameters. So here is microbenchmark that shows some improvements and some regressions. There is lot of noise. I ran these three times and on one * and ffffffffoo pattern become twice slower for no reason. So how do you decide with conflicted data like this? old implementation wordexp Pattern foo flags 0: 1081.77 Pattern ffoo flags 0: 621.516 Pattern ffffoo flags 0: 697.562 Pattern ffffffffoo flags 0: 867.922 Pattern ffffffffffffffffoo flags 0: 1110.75 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1567.92 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2541.03 Pattern * flags 0: 141070 Pattern foo flags 0: 1061.08 Pattern ffoo flags 0: 1190.48 Pattern ffffoo flags 0: 1146.83 Pattern ffffffffoo flags 0: 1407.77 Pattern ffffffffffffffffoo flags 0: 1734.33 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2143.02 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3100.41 Pattern * flags 0: 205510 wordexp Pattern foo flags 0: 1081.84 Pattern ffoo flags 0: 672.422 Pattern ffffoo flags 0: 704.828 Pattern ffffffffoo flags 0: 875.188 Pattern ffffffffffffffffoo flags 0: 1089.7 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1594.03 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2575.28 Pattern * flags 0: 144160 Pattern foo flags 0: 1062.45 Pattern ffoo flags 0: 1099.72 Pattern ffffoo flags 0: 1153.95 Pattern ffffffffoo flags 0: 1391.45 Pattern ffffffffffffffffoo flags 0: 1820.36 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2129 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3171.5 Pattern * flags 0: 208475 wordexp Pattern foo flags 0: 1086.2 Pattern ffoo flags 0: 615.078 Pattern ffffoo flags 0: 688.594 Pattern ffffffffoo flags 0: 851.984 Pattern ffffffffffffffffoo flags 0: 1085.48 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1562.95 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2540.08 Pattern * flags 0: 287636 Pattern foo flags 0: 1068.11 Pattern ffoo flags 0: 1092.5 Pattern ffffoo flags 0: 1143.7 Pattern ffffffffoo flags 0: 1380.66 Pattern ffffffffffffffffoo flags 0: 1763.88 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2165.72 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3095.41 Pattern * flags 0: 207373 new wordexp Pattern foo flags 0: 1161.95 Pattern ffoo flags 0: 631.016 Pattern ffffoo flags 0: 714.797 Pattern ffffffffoo flags 0: 2209.84 Pattern ffffffffffffffffoo flags 0: 1024.17 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1408.22 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2223.39 Pattern * flags 0: 146925 Pattern foo flags 0: 1660.66 Pattern ffoo flags 0: 1673.34 Pattern ffffoo flags 0: 1730.23 Pattern ffffffffoo flags 0: 2030.42 Pattern ffffffffffffffffoo flags 0: 2245.59 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2512.95 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3312.95 Pattern * flags 0: 210492 wordexp Pattern foo flags 0: 1184.19 Pattern ffoo flags 0: 634.062 Pattern ffffoo flags 0: 683.688 Pattern ffffffffoo flags 0: 826.891 Pattern ffffffffffffffffoo flags 0: 1020.27 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1405.89 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2185.44 Pattern * flags 0: 140257 Pattern foo flags 0: 1649.77 Pattern ffoo flags 0: 1668.39 Pattern ffffoo flags 0: 1722.23 Pattern ffffffffoo flags 0: 1924.27 Pattern ffffffffffffffffoo flags 0: 2322.56 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2502.8 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3306.39 Pattern * flags 0: 208003 wordexp Pattern foo flags 0: 1097.7 Pattern ffoo flags 0: 655.891 Pattern ffffoo flags 0: 692.453 Pattern ffffffffoo flags 0: 860.203 Pattern ffffffffffffffffoo flags 0: 1009.38 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 1426.02 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 2179.64 Pattern * flags 0: 144492 Pattern foo flags 0: 1668.47 Pattern ffoo flags 0: 1684.38 Pattern ffffoo flags 0: 1718.75 Pattern ffffffffoo flags 0: 1932.91 Pattern ffffffffffffffffoo flags 0: 2294.95 Pattern ffffffffffffffffffffffffffffffffoo flags 0: 2514.89 Pattern ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo flags 0: 3329.14 Pattern * flags 0: 211152 diff --git a/benchtests/Makefile b/benchtests/Makefile index 8e615e5..fb737da 100644 --- a/benchtests/Makefile +++ b/benchtests/Makefile @@ -35,7 +35,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \ strcat strchr strchrnul strcmp strcpy strcspn strlen \ strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \ strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \ - strcoll + strcoll wordexp string-bench-all := $(string-bench) # We have to generate locales diff --git a/benchtests/bench-wordexp.c b/benchtests/bench-wordexp.c new file mode 100644 index 0000000..ffadbcc --- /dev/null +++ b/benchtests/bench-wordexp.c @@ -0,0 +1,97 @@ +/* Measure wordexp functions. + Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define TEST_MAIN +#define TEST_NAME "wordexp" +#include "bench-string.h" + +#include + +typedef int (*proto_t) (const char *, wordexp_t *, int); + +IMPL (wordexp, 1) + + +static void +do_one_test (impl_t *impl, const char *s1, int flag) +{ + size_t i, iters = INNER_LOOP_ITERS; + timing_t start, stop, cur; + + TIMING_NOW (start); + for (i = 0; i < iters; ++i) + { + wordexp_t out; + CALL (impl, s1, &out, flag); + } + TIMING_NOW (stop); + + TIMING_DIFF (cur, start, stop); + + TIMING_PRINT_MEAN ((double) cur, (double) iters); +} + + +static void +do_test (const char *s1, int flag) +{ + + printf ("Pattern %s flags %i:", s1, flag); + + FOR_EACH_IMPL (impl, 0) + do_one_test (impl, s1, flag); + + putchar ('\n'); +} + +static int +test_main (void) +{ + test_init (); + + printf ("%23s", ""); + FOR_EACH_IMPL (impl, 0) + printf ("\t%s", impl->name); + putchar ('\n'); + + do_test ("foo", 0); + do_test ("ffoo", 0); + do_test ("ffffoo", 0); + do_test ("ffffffffoo", 0); + do_test ("ffffffffffffffffoo", 0); + do_test ("ffffffffffffffffffffffffffffffffoo", 0); +do_test ("ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo", 0); + + + do_test ("*", 0); + setenv("IFS"," \ + ", 1); + + do_test ("foo", 0); + do_test ("ffoo", 0); + do_test ("ffffoo", 0); + do_test ("ffffffffoo", 0); + do_test ("ffffffffffffffffoo", 0); + do_test ("ffffffffffffffffffffffffffffffffoo", 0); +do_test ("ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffoo", 0); + do_test ("*", 0); + + return ret; +} + +#include "../test-skeleton.c"