From patchwork Thu Sep 16 02:23:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naohiro Tamura X-Patchwork-Id: 45064 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 51CFC385780F for ; Thu, 16 Sep 2021 02:24:46 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 51CFC385780F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1631759086; bh=8jfO6dOWdQaN3ZEaFNSUzOA5F/2lWqpaWJtVWJtEa7s=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=LPx7gGHikPeakVN6Z48ZjLro69MpzQLqKVkAEj2VaJ7n9ODHkuShyifu79xWhhKQv i8fvnRgTwlejg1PzptC7qTKlBIaazfbiSNQZROQg55xJ4xRD5r4aPH9GA5D2PbWXNG dlikjf+uBKCFWMOA4UUi6ltPbOLwqUoaga46a0YE= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from esa7.hc1455-7.c3s2.iphmx.com (esa7.hc1455-7.c3s2.iphmx.com [139.138.61.252]) by sourceware.org (Postfix) with ESMTPS id 2E0413858020 for ; Thu, 16 Sep 2021 02:24:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 2E0413858020 IronPort-SDR: oFNa23Sxbc2RQEZeSaArnoMBSe+g6nPSxX4ckEU6ugRSKyfR0qp/3d+ptHhIHlgb1+Z2pIDu05 o+mF8gK9avBncKodayRUgoLEvfX3twxtU0zZs6MMvCo5cL9H8oLEHFP/LH1/FGZgqbIiA2NDe3 E0ZPRN36gouLFwGvL/W4Clne2TLKc1ospN6guR3Vi3bK/Qh+n0rR/Z2XxyqGyVIJp0IqNYIBRR xwIZG8rM3zkEOLWDSlc2m+UymkwbzdCOHigNh6/oeUSyA6GyPKor+pKeRxHgBnI1pvfbp1X/zF xT6Nbah/Nupkj/4OlbXCOCqR X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="24031492" X-IronPort-AV: E=Sophos;i="5.85,297,1624287600"; d="scan'208";a="24031492" Received: from unknown (HELO oym-r3.gw.nic.fujitsu.com) ([210.162.30.91]) by esa7.hc1455-7.c3s2.iphmx.com with ESMTP; 16 Sep 2021 11:24:13 +0900 Received: from oym-m3.gw.nic.fujitsu.com (oym-nat-oym-m3.gw.nic.fujitsu.com [192.168.87.60]) by oym-r3.gw.nic.fujitsu.com (Postfix) with ESMTP id D36F666 for ; Thu, 16 Sep 2021 11:24:12 +0900 (JST) Received: from m3051.s.css.fujitsu.com (m3051.s.css.fujitsu.com [10.134.21.209]) by oym-m3.gw.nic.fujitsu.com (Postfix) with ESMTP id 0DB8FC0F24 for ; Thu, 16 Sep 2021 11:24:12 +0900 (JST) Received: from bionic.lxd (unknown [10.126.53.116]) by m3051.s.css.fujitsu.com (Postfix) with ESMTP id E4CC29E; Thu, 16 Sep 2021 11:24:11 +0900 (JST) To: Siddhesh Poyarekar , libc-alpha@sourceware.org Subject: [PATCH v5] benchtests: Fix validate_benchout.py exceptions Date: Thu, 16 Sep 2021 02:23:16 +0000 Message-Id: <20210916022316.1101289-1-naohirot@fujitsu.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <0eb9f04e-0db6-dcc8-3e76-36f18a39ba1d@gotplt.org> References: <0eb9f04e-0db6-dcc8-3e76-36f18a39ba1d@gotplt.org> X-TM-AS-GCONF: 00 X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_PASS, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naohiro Tamura via Libc-alpha From: Naohiro Tamura Reply-To: Naohiro Tamura Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" This patch fixed validate_benchout.py two exceptions, 1) AttributeError if benchout_strings.schema.json is specified, and 2) json.decoder.JSONDecodeError if benchout file is not JSON. $ ~/glibc/benchtests/scripts/validate_benchout.py bench-memset.out \ ~/glibc/benchtests/scripts/benchout_strings.schema.json Traceback (most recent call last): File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 86, in sys.exit(main(sys.argv[1:])) File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 69, in main bench.parse_bench(args[0], args[1]) File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 139, in parse_bench do_for_all_timings(bench, lambda b, f, v: File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 107, in do_for_all_timings if 'timings' not in bench['functions'][func][k].keys(): AttributeError: 'str' object has no attribute 'keys' $ ~/glibc/benchtests/scripts/validate_benchout.py bench-math-inlines.out \ ~/glibc/benchtests/scripts/benchout_strings.schema.json Traceback (most recent call last): File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 86, in sys.exit(main(sys.argv[1:])) File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 69, in main bench.parse_bench(args[0], args[1]) File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 137, in parse_bench bench = json.load(benchfile) File "/usr/lib/python3.6/json/__init__.py", line 299, in load parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) File "/usr/lib/python3.6/json/__init__.py", line 354, in loads return _default_decoder.decode(s) File "/usr/lib/python3.6/json/decoder.py", line 342, in decode raise JSONDecodeError("Extra data", s, end) json.decoder.JSONDecodeError: Extra data: line 1 column 17 (char 16) Reviewed-by: Siddhesh Poyarekar Reviewed-by: Siddhesh Poyarekar --- benchtests/scripts/compare_bench.py | 4 ++++ benchtests/scripts/import_bench.py | 2 -- benchtests/scripts/validate_benchout.py | 6 +++++- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/benchtests/scripts/compare_bench.py b/benchtests/scripts/compare_bench.py index 6fcbd0803808..1a497f8bb61b 100755 --- a/benchtests/scripts/compare_bench.py +++ b/benchtests/scripts/compare_bench.py @@ -163,7 +163,11 @@ def plot_graphs(bench1, bench2): def main(bench1, bench2, schema, threshold, stats): bench1 = bench.parse_bench(bench1, schema) + bench.do_for_all_timings(bench1, lambda b, f, v: + b['functions'][f][v]['timings'].sort()) bench2 = bench.parse_bench(bench2, schema) + bench.do_for_all_timings(bench2, lambda b, f, v: + b['functions'][f][v]['timings'].sort()) plot_graphs(bench1, bench2) diff --git a/benchtests/scripts/import_bench.py b/benchtests/scripts/import_bench.py index a799b4e1b7dc..362c990fe153 100644 --- a/benchtests/scripts/import_bench.py +++ b/benchtests/scripts/import_bench.py @@ -136,6 +136,4 @@ def parse_bench(filename, schema_filename): with open(filename, 'r') as benchfile: bench = json.load(benchfile) validator.validate(bench, schema) - do_for_all_timings(bench, lambda b, f, v: - b['functions'][f][v]['timings'].sort()) return bench diff --git a/benchtests/scripts/validate_benchout.py b/benchtests/scripts/validate_benchout.py index 47df33ed0252..00d5fa0ee5eb 100755 --- a/benchtests/scripts/validate_benchout.py +++ b/benchtests/scripts/validate_benchout.py @@ -73,11 +73,15 @@ def main(args): except bench.validator.ValidationError as e: return print_and_exit("Invalid benchmark output: %s" % e.message, - os.EX_DATAERR) + os.EX_DATAERR) except bench.validator.SchemaError as e: return print_and_exit("Invalid schema: %s" % e.message, os.EX_DATAERR) + except json.decoder.JSONDecodeError as e: + return print_and_exit("Benchmark output in %s is not JSON." % args[0], + os.EX_DATAERR) + print("Benchmark output in %s is valid." % args[0]) return os.EX_OK