From patchwork Wed Aug 16 11:25:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wilco Dijkstra X-Patchwork-Id: 22137 Received: (qmail 4976 invoked by alias); 16 Aug 2017 11:26:35 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 124142 invoked by uid 89); 16 Aug 2017 11:26:05 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-25.1 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 spammy=7110, Hx-languages-length:4283, 8816 X-HELO: EUR01-DB5-obe.outbound.protection.outlook.com From: Wilco Dijkstra To: "libc-alpha@sourceware.org" CC: nd Subject: [PATCH] Add math benchmark latency test Date: Wed, 16 Aug 2017 11:25:15 +0000 Message-ID: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Wilco.Dijkstra@arm.com; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB6PR0801MB2053; 6:mQz2ejFY9k3NV/akt1sX+WG0NN8fw+bw/mqQV6iBFVbnhBx1qZeT3Oe00TtLc1fheFCsvVNm8PCqvwUznwxoDe9rD9GIh5kt4N9ZxS5DjVjjIw3PeTCiOUNMvRbFojU8J6R/OwDaPF2I8KGPAjqJba/erxy2P5Klt1+QY9n69qvB59QDdQnGifmC1pBG9eay82/8HJT3HZAs9gsdvmGBp8iZAKyKkWLBIsazeItmpJgRjRU+6pe/+oFrlaRlxm/DgxgDPstkRl98P00gdLmcgm0mPLrgglS5bPLQGybRQe/mGMixNuRHPbbdHav/yZobGlpD5B/YJjVOMYozgCGZXw==; 5:sCdWxh/Rx01dcH+F/iq4FT9sXcRFAMrtSj9eBAp0NwMn8cz5vv/50hjPNP1SJBNJp3V9pWCElrO33PmsVlx+Jut0Ax8mCZNpbTX7CsmalNyrZp7SIxHB05gu5pbC+fPyoDOkNnGy8SyuG9Jv0agRlg==; 24:batx3covrAiSFiuJbCh9owCQR4IeFQmFQAZlvaqVo8LNS3PS6JPjjKurAebVB4ipBKZubvz8aVqX58j7y66BZqeZC9a7O7AfzRoo7WdWOlA=; 7:2Xng/4CNXTf1O7YINyZs8iHkZZOCOxb5UYvXouHRY8u5h+utTukDxIrraZvWSbEoi2yCL3lCDWg4CMc4N2L5/MztLsywVqaV1ubwHHq9nYpfPNLeYw0ZJBqs/+qvhFoi+/8NZ7TsmyMJYQNf0BsKo1uBekRCFgCegB51lXXwz6qe4oqHbmz9qN/X21t+weyjVD2H2aYaeyXAs/yk/d41u+pjQ6EaJOw2HVLkQyPYfh4= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: d05294a2-45fc-4ea9-f5d8-08d4e499778a x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(48565401081)(300000503095)(300135400095)(2017052603031)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:DB6PR0801MB2053; x-ms-traffictypediagnostic: DB6PR0801MB2053: nodisclaimer: True x-exchange-antispam-report-test: UriScan:(180628864354917); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(100000703101)(100105400095)(93006095)(93001095)(10201501046)(3002001)(6055026)(6041248)(20161123558100)(20161123560025)(20161123555025)(20161123564025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DB6PR0801MB2053; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DB6PR0801MB2053; x-forefront-prvs: 0401647B7F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(39860400002)(199003)(189002)(54534003)(377424004)(105586002)(8936002)(2900100001)(66066001)(189998001)(102836003)(6116002)(81166006)(3846002)(97736004)(8676002)(81156014)(53936002)(86362001)(110136004)(575784001)(6506006)(6436002)(478600001)(72206003)(99286003)(33656002)(9686003)(25786009)(2501003)(7696004)(4326008)(55016002)(2351001)(6916009)(14454004)(5660300001)(5250100002)(2906002)(7736002)(68736007)(5640700003)(50986999)(54356999)(74316002)(305945005)(3280700002)(101416001)(3660700001)(106356001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR0801MB2053; H:DB6PR0801MB2053.eurprd08.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:3; LANG:en; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Aug 2017 11:25:15.1869 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB2053 This patch further improves math function benchmarking by adding a latency test in addition to throughput. This enables more accurate comparisons of the math functions. The latency test works by creating a dependency on the previous iteration: func_res = F (func_res * zero + input[i]). The multiply by zero avoids changing the input. The powf test now shows: "workload-spec2006.wrf": { "throughput": 200, "latency": 100 } OK for commit? ChangeLog: 2017-08-16 Wilco Dijkstra * benchtests/bench-skeleton.c (main): Add support for latency benchmarking. * benchtests/scripts/bench.py: Add support for latency benchmarking. diff --git a/benchtests/bench-skeleton.c b/benchtests/bench-skeleton.c index 3c6dad705594ac0a53edcb4e09686252c13127cf..60753ede1aa3cc05cc0e9eccc74dd12a609a1a67 100644 --- a/benchtests/bench-skeleton.c +++ b/benchtests/bench-skeleton.c @@ -71,8 +71,10 @@ main (int argc, char **argv) bool is_bench = strncmp (VARIANT (v), "workload-", 9) == 0; double d_total_i = 0; timing_t total = 0, max = 0, min = 0x7fffffffffffffff; + timing_t throughput = 0, latency = 0; int64_t c = 0; uint64_t cur; + BENCH_VARS; while (1) { if (is_bench) @@ -86,7 +88,16 @@ main (int argc, char **argv) BENCH_FUNC (v, i); TIMING_NOW (end); TIMING_DIFF (cur, start, end); - TIMING_ACCUM (total, cur); + TIMING_ACCUM (throughput, cur); + + TIMING_NOW (start); + for (k = 0; k < iters; k++) + for (i = 0; i < NUM_SAMPLES (v); i++) + BENCH_FUNC_LAT (v, i); + TIMING_NOW (end); + TIMING_DIFF (cur, start, end); + TIMING_ACCUM (latency, cur); + d_total_i += iters * NUM_SAMPLES (v); } else @@ -131,12 +142,15 @@ main (int argc, char **argv) /* Begin variant. */ json_attr_object_begin (&json_ctx, VARIANT (v)); - json_attr_double (&json_ctx, "duration", d_total_s); - json_attr_double (&json_ctx, "iterations", d_total_i); if (is_bench) - json_attr_double (&json_ctx, "throughput", d_total_s / d_total_i); + { + json_attr_double (&json_ctx, "throughput", throughput / d_total_i); + json_attr_double (&json_ctx, "latency", latency / d_total_i); + } else { + json_attr_double (&json_ctx, "duration", d_total_s); + json_attr_double (&json_ctx, "iterations", d_total_i); json_attr_double (&json_ctx, "max", max / d_iters); json_attr_double (&json_ctx, "min", min / d_iters); json_attr_double (&json_ctx, "mean", d_total_s / d_total_i); diff --git a/benchtests/scripts/bench.py b/benchtests/scripts/bench.py index 8c1c9eeb2bc67a16cb8a8e010fd2b8a2ef8ab6df..b7ccb7c8c2bf1822202a2377dfb0675516115cc5 100755 --- a/benchtests/scripts/bench.py +++ b/benchtests/scripts/bench.py @@ -45,7 +45,7 @@ DEFINES_TEMPLATE = ''' # variant is represented by the _VARIANT structure. The ARGS structure # represents a single set of arguments. STRUCT_TEMPLATE = ''' -#define CALL_BENCH_FUNC(v, i) %(func)s (%(func_args)s) +#define CALL_BENCH_FUNC(v, i, x) %(func)s (x %(func_args)s) struct args { @@ -84,7 +84,9 @@ EPILOGUE = ''' #define RESULT(__v, __i) (variants[(__v)].in[(__i)].timing) #define RESULT_ACCUM(r, v, i, old, new) \\ ((RESULT ((v), (i))) = (RESULT ((v), (i)) * (old) + (r)) / ((new) + 1)) -#define BENCH_FUNC(i, j) ({%(getret)s CALL_BENCH_FUNC (i, j);}) +#define BENCH_FUNC(i, j) ({%(getret)s CALL_BENCH_FUNC (i, j, );}) +#define BENCH_FUNC_LAT(i, j) ({%(getret)s CALL_BENCH_FUNC (i, j, %(latarg)s);}) +#define BENCH_VARS %(defvar)s #define FUNCNAME "%(func)s" #include "bench-skeleton.c"''' @@ -122,17 +124,22 @@ def gen_source(func, directives, all_vals): # If we have a return value from the function, make sure it is # assigned to prevent the compiler from optimizing out the # call. + getret = '' + latarg = '' + defvar = '' + if directives['ret']: print('static %s volatile ret;' % directives['ret']) - getret = 'ret = ' - else: - getret = '' + print('static %s zero __attribute__((used)) = 0;' % directives['ret']) + getret = 'ret = func_res = ' + latarg = 'func_res * zero +' + defvar = '%s func_res = 0;' % directives['ret'] # Test initialization. if directives['init']: print('#define BENCH_INIT %s' % directives['init']) - print(EPILOGUE % {'getret': getret, 'func': func}) + print(EPILOGUE % {'getret': getret, 'func': func, 'latarg': latarg, 'defvar': defvar }) def _print_arg_data(func, directives, all_vals):