From patchwork Tue Mar 25 10:29:41 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Siddhesh Poyarekar X-Patchwork-Id: 264 Return-Path: X-Original-To: siddhesh@wilcox.dreamhost.com Delivered-To: siddhesh@wilcox.dreamhost.com Received: from homiemail-mx21.g.dreamhost.com (caibbdcaaahb.dreamhost.com [208.113.200.71]) by wilcox.dreamhost.com (Postfix) with ESMTP id 034F8360115 for ; Tue, 25 Mar 2014 03:29:11 -0700 (PDT) Received: by homiemail-mx21.g.dreamhost.com (Postfix, from userid 14307373) id A43081F4B97F; Tue, 25 Mar 2014 03:29:11 -0700 (PDT) X-Original-To: glibc@patchwork.siddhesh.in Delivered-To: x14307373@homiemail-mx21.g.dreamhost.com Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by homiemail-mx21.g.dreamhost.com (Postfix) with ESMTPS id 80885DB2B29 for ; Tue, 25 Mar 2014 03:29:11 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:to:subject:message-id:mime-version :content-type; q=dns; s=default; b=Sdyq7U8Ef3EWtwJA0R+GUzgoz4Epk lzoIAI5zPHdkGhFRvfShy6PllwEQcaa0pBbG3E3p/9YaVs8vr4jMVzCXKd6j+FMX z7pTkiaOTh1NPSer5+GVToMdYYl3Kbr9pbKL5g1ub6lgiL1DDlWc3eQNfp8v4F1A kpe80Y0dRf3btc= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:to:subject:message-id:mime-version :content-type; s=default; bh=8zlXEaIAvkArZrsUkwSjJ5m3FiM=; b=gw2 6hcSTaYIOHsgQz8m/25lS97FwhUjOKorRgA6t9VCanRBktm6qlvA4kscEcMdc4HK iWq0O3GiJHacCI3qH1XBXeewZYywhicynJTTgZdl8PSC2F75jDlJXvKl866nIwi1 UaBu4L6h8RWUTxyK8l5Qw9PJ8NScdGthS/rfGvD8= Received: (qmail 23259 invoked by alias); 25 Mar 2014 10:29:09 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 23247 invoked by uid 89); 25 Mar 2014 10:29:08 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-3.9 required=5.0 tests=AWL, BAYES_00, RP_MATCHES_RCVD, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 X-HELO: mx1.redhat.com Date: Tue, 25 Mar 2014 15:59:41 +0530 From: Siddhesh Poyarekar To: libc-alpha@sourceware.org Subject: [PATCH 2/4] Detailed benchmark outputs for functions Message-ID: <20140325102941.GB1850@spoyarek.pnq.redhat.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.22.1-rc1 (2013-10-16) X-DH-Original-To: glibc@patchwork.siddhesh.in Hi, This patch adds an option to get detailed benchmark output for functions. Invoking the benchmark with 'make DETAILED=1 bench' causes each benchmark program to store a mean execution time for each input it works on. This is useful to give a more comprehensive picture of performance of functions compared to just the single mean figure. Siddhesh * benchtests/Makefile (DETAILED_OPT): New make option. (bench-func): Run benchmark program with -d if DETAILED_OPT is set. * benchtests/bench-skeleton.c: Include stdbool.h. (main): Store and print timings per input. * benchtests/scripts/bench.py (STRUCT_TEMPLATE): Add timing member to each argument value. (EPILOGUE): Define new macros RESULT and RESULT_ACCUM. (_print_arg_data): Initialize per-input timing to 0. --- benchtests/Makefile | 8 +++++++- benchtests/bench-skeleton.c | 22 ++++++++++++++++++++++ benchtests/scripts/bench.py | 6 +++++- 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/benchtests/Makefile b/benchtests/Makefile index be11708..f5488c1 100644 --- a/benchtests/Makefile +++ b/benchtests/Makefile @@ -86,6 +86,12 @@ ifdef USE_CLOCK_GETTIME CPPFLAGS-nonlib += -DUSE_CLOCK_GETTIME endif +DETAILED_OPT := + +ifdef DETAILED +DETAILED_OPT := -d +endif + # This makes sure CPPFLAGS-nonlib and CFLAGS-nonlib are passed # for all these modules. cpp-srcs-left := $(binaries-benchset:=.c) $(binaries-bench:=.c) @@ -126,7 +132,7 @@ bench-func: $(binaries-bench) echo ","; \ fi; \ echo "Running $${run}" >&2; \ - $(run-bench); \ + $(run-bench) $(DETAILED_OPT); \ done; \ echo " }"; \ echo "}"; } > $(objpfx)bench.out-tmp; \ diff --git a/benchtests/bench-skeleton.c b/benchtests/bench-skeleton.c index faef7eb..0c7d744 100644 --- a/benchtests/bench-skeleton.c +++ b/benchtests/bench-skeleton.c @@ -18,6 +18,7 @@ #include #include +#include #include #include #include @@ -48,6 +49,10 @@ main (int argc, char **argv) unsigned long i, k; struct timespec runtime; timing_t start, end; + bool detailed = false; + + if (argc == 2 && !strcmp (argv[1], "-d")) + detailed = true; startup(); @@ -72,6 +77,7 @@ main (int argc, char **argv) double d_total_i = 0; timing_t total = 0, max = 0, min = 0x7fffffffffffffff; + int64_t c = 0; while (1) { for (i = 0; i < NUM_SAMPLES (v); i++) @@ -91,8 +97,13 @@ main (int argc, char **argv) min = cur; TIMING_ACCUM (total, cur); + /* Accumulate timings for the value. In the end we will divide + by the total iterations. */ + RESULT_ACCUM (cur, v, i, c * iters, (c + 1) * iters); + d_total_i += iters; } + c++; struct timespec curtime; memset (&curtime, 0, sizeof (curtime)); @@ -114,6 +125,17 @@ main (int argc, char **argv) d_total_s, d_total_i, max / d_iters, min / d_iters, d_total_s / d_total_i); + if (detailed) + { + printf (",\n\"timings\": ["); + for (int i = 0; i < NUM_SAMPLES (v); i++) + { + if (i > 0) + putc (',', stdout); + printf ("%g", RESULT (v, i)); + } + puts ("]"); + } puts ("}"); } diff --git a/benchtests/scripts/bench.py b/benchtests/scripts/bench.py index 90317b5..492c764 100755 --- a/benchtests/scripts/bench.py +++ b/benchtests/scripts/bench.py @@ -50,6 +50,7 @@ STRUCT_TEMPLATE = ''' struct args { %(args)s + double timing; }; struct _variants @@ -80,6 +81,9 @@ struct _variants variants[%(num_variants)d] = { # Epilogue for the generated source file. EPILOGUE = ''' +#define RESULT(__v, __i) (variants[(__v)].in[(__i)].timing) +#define RESULT_ACCUM(r, v, i, old, new) \\ + ((RESULT ((v), (i))) = (RESULT ((v), (i)) * (old) + (r)) / ((new) + 1)) #define BENCH_FUNC(i, j) ({%(getret)s CALL_BENCH_FUNC (i, j);}) #define FUNCNAME "%(func)s" #include "bench-skeleton.c"''' @@ -168,7 +172,7 @@ def _print_arg_data(func, directives, all_vals): # Now print the values. variants = [] for (k, vals), i in zip(all_vals.items(), itertools.count()): - out = [' {%s},' % v for v in vals] + out = [' {%s, 0},' % v for v in vals] # Members for the variants structure list that we will # print later.