From patchwork Thu Jun 15 12:17:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wilco Dijkstra X-Patchwork-Id: 21035 Received: (qmail 104001 invoked by alias); 15 Jun 2017 12:17:14 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 103978 invoked by uid 89); 15 Jun 2017 12:17:13 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 spammy=timings X-HELO: EUR01-HE1-obe.outbound.protection.outlook.com From: Wilco Dijkstra To: "libc-alpha@sourceware.org" CC: nd Subject: [PATCH] Improve math benchmark infrastructure Date: Thu, 15 Jun 2017 12:17:12 +0000 Message-ID: authentication-results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=arm.com; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM5PR0802MB2611; 7:1LEfe9OvfMettuf4M5U6jINyK4yfyvAM5gNseWcoRi/zLpahJwwdsljkTifvHEEfBirbLH1QFdy6K2lV1d/NGB1GKZEgp4mcnALgWg+D/Y0OYMar/p6o2PmswcWxXjXTHip3Am3NMJ4tvhW0LaIi+0sjflW/N+qAjV+KXUQh4yS2sWqjl9H26CVoQUP8+PxRsJgVWHX5+HOSKX0WZ9RKvIdnFkLZeltyBhM3XpB4Ns8M/iNl5zXkTfkf1zntUzyyjW3U2STo+0EGJ+JJA4cSZof0MN14TWy+oPYcFSPOe+72f0n5dS9Mv05FTZsLYAhxH+p22R66rXh1yO0dBbh1Jw== x-ms-office365-filtering-correlation-id: 6cc808e0-6d26-4e3c-1b0d-08d4b3e87410 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(48565401081)(201703131423075)(201703031133081); SRVR:AM5PR0802MB2611; x-ms-traffictypediagnostic: AM5PR0802MB2611: nodisclaimer: True x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(180628864354917); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(93001095)(3002001)(100000703101)(100105400095)(10201501046)(6055026)(6041248)(20161123558100)(20161123555025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123562025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:AM5PR0802MB2611; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:AM5PR0802MB2611; x-forefront-prvs: 0339F89554 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(39850400002)(39450400003)(39400400002)(39410400002)(39840400002)(377424004)(38730400002)(81166006)(53936002)(4326008)(3660700001)(9686003)(8676002)(110136004)(6506006)(5640700003)(74316002)(33656002)(2906002)(55016002)(72206003)(99286003)(5250100002)(3280700002)(54356999)(305945005)(50986999)(2501003)(5660300001)(6116002)(14454004)(2351001)(86362001)(102836003)(7736002)(189998001)(8936002)(7696004)(6916009)(478600001)(6436002)(2900100001)(25786009); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0802MB2611; H:AM5PR0802MB2610.eurprd08.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jun 2017 12:17:12.5985 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2611 Improve support for math function benchmarking. This patch adds a feature that allows accurate benchmarking of traces extracted from real workloads. This is done by iterating over all samples rather than repeating each sample many times (which completely ignores branch prediction and cache effects). A trace can be added to existing math function inputs via "## name: bench", followed by the trace. OK for commit? ChangeLog: 2017-06-15 Wilco Dijkstra * benchtests/bench-skeleton.c (main): Add support for benchmarking traces from workloads. diff --git a/benchtests/bench-skeleton.c b/benchtests/bench-skeleton.c index 09eb78df1bce2d9f5e410e3e82821eb9b271e70d..48001763e8481592182fd8d67948ccb4078a48cc 100644 --- a/benchtests/bench-skeleton.c +++ b/benchtests/bench-skeleton.c @@ -68,34 +68,50 @@ main (int argc, char **argv) clock_gettime (CLOCK_MONOTONIC_RAW, &runtime); runtime.tv_sec += DURATION; + bool is_bench = strcmp (VARIANT (v), "bench") == 0; double d_total_i = 0; timing_t total = 0, max = 0, min = 0x7fffffffffffffff; int64_t c = 0; + uint64_t cur; while (1) { - for (i = 0; i < NUM_SAMPLES (v); i++) + if (is_bench) { - uint64_t cur; + /* Benchmark a real trace of calls - all samples are iterated + over once before repeating. This models actual use more + accurately than repeating the same sample many times. */ TIMING_NOW (start); for (k = 0; k < iters; k++) - BENCH_FUNC (v, i); + for (i = 0; i < NUM_SAMPLES (v); i++) + BENCH_FUNC (v, i); TIMING_NOW (end); - TIMING_DIFF (cur, start, end); + TIMING_ACCUM (total, cur); + d_total_i += iters; + } + else + for (i = 0; i < NUM_SAMPLES (v); i++) + { + TIMING_NOW (start); + for (k = 0; k < iters; k++) + BENCH_FUNC (v, i); + TIMING_NOW (end); - if (cur > max) - max = cur; + TIMING_DIFF (cur, start, end); - if (cur < min) - min = cur; + if (cur > max) + max = cur; - TIMING_ACCUM (total, cur); - /* Accumulate timings for the value. In the end we will divide - by the total iterations. */ - RESULT_ACCUM (cur, v, i, c * iters, (c + 1) * iters); + if (cur < min) + min = cur; - d_total_i += iters; - } + TIMING_ACCUM (total, cur); + /* Accumulate timings for the value. In the end we will divide + by the total iterations. */ + RESULT_ACCUM (cur, v, i, c * iters, (c + 1) * iters); + + d_total_i += iters; + } c++; struct timespec curtime; @@ -117,11 +133,14 @@ main (int argc, char **argv) json_attr_double (&json_ctx, "duration", d_total_s); json_attr_double (&json_ctx, "iterations", d_total_i); - json_attr_double (&json_ctx, "max", max / d_iters); - json_attr_double (&json_ctx, "min", min / d_iters); + if (!is_bench) + { + json_attr_double (&json_ctx, "max", max / d_iters); + json_attr_double (&json_ctx, "min", min / d_iters); + } json_attr_double (&json_ctx, "mean", d_total_s / d_total_i); - if (detailed) + if (detailed && !is_bench) { json_array_begin (&json_ctx, "timings");