From patchwork Mon Nov 8 15:42:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 47217 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id CA5FF3858003 for ; Mon, 8 Nov 2021 15:42:37 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CA5FF3858003 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1636386157; bh=f98yKfUYFZm94PrjJ7fCETGJ9l9kPEhv0utgA6c2YBI=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=D8WekW2HvgnNM3/vTbHdCDnp37avCudKt3HhYf0zwGBtOq+fWff8v31KDrkw20V3O 8oAvK5zBRi6cCGOzxCNaqhGNietCIQSFhOINy4+PnXQzftUr5yyo9Y522i2b6nWb6U pMg/cwNwCA5UkvgsXU/hxLqLrGouZi1utn0VsKNw= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by sourceware.org (Postfix) with ESMTPS id ADA1C3858406 for ; Mon, 8 Nov 2021 15:42:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org ADA1C3858406 Received: by mail-pf1-x42d.google.com with SMTP id c4so4635682pfj.2 for ; Mon, 08 Nov 2021 07:42:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f98yKfUYFZm94PrjJ7fCETGJ9l9kPEhv0utgA6c2YBI=; b=Gfv4AOYuZsUD/BhK1S1N+2olKS4CNBTBmqYTvAboOnCTjRS5Bl9XbuuEwDtFFpZZYJ 0ymE/WWHAQ1rqKnmnSLuaRZvjQJQ6e2Vij6AU3lyGUFr8zLOCeylPCNTr038asnMsDgE z4F6aikVNB4rn8HmnuQPhDRoZPOQLVU3hMa2qyJKCSm8ZdwZktAjxXTxf5ptBQ3l3rlB G37r6ZjHQcuWWcAX/+rd1vwy288ARYr1XAxmdnUC7ncHnqm43+4t9j6kdnsOjBSO10ty 2R6cl0V4hIyDzPEe0NNliKohvTyawG0Vh4svVRL560DgwdrcGVXDqaJb3+W3MxoJCNC/ p3TA== X-Gm-Message-State: AOAM530xnXUFly8lo2yzPqDQEy7ApBt9LNspngTcq/qGRbE/VA1WALcj 89sY2gdtPeznZR59/T6l8UoYX+ZSORk= X-Google-Smtp-Source: ABdhPJw0StR8JSO/ULwQjLPl3h6l+YIse4b8yFhPlMEv3JQ6tgmpbUSczRRFIvxsDvF+1xMW1fMVLQ== X-Received: by 2002:a05:6a00:c95:b0:49f:c8de:9ae7 with SMTP id a21-20020a056a000c9500b0049fc8de9ae7mr11239704pfv.30.1636386133593; Mon, 08 Nov 2021 07:42:13 -0800 (PST) Received: from gnu-cfl-2.localdomain ([172.58.35.133]) by smtp.gmail.com with ESMTPSA id t135sm12836583pgc.51.2021.11.08.07.42.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Nov 2021 07:42:13 -0800 (PST) Received: from gnu-cfl-2.lan (localhost [IPv6:::1]) by gnu-cfl-2.localdomain (Postfix) with ESMTP id CDFBD1A0943; Mon, 8 Nov 2021 07:42:11 -0800 (PST) To: libc-alpha@sourceware.org Subject: [PATCH v2 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550] Date: Mon, 8 Nov 2021 07:42:10 -0800 Message-Id: <20211108154211.167829-2-hjl.tools@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211108154211.167829-1-hjl.tools@gmail.com> References: <20211108154211.167829-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3030.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "H.J. Lu via Libc-alpha" From: "H.J. Lu" Reply-To: "H.J. Lu" Cc: Florian Weimer Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Put all sources for DSO sorting tests in the dso-sort-tests-src directory and compile test relocatable objects with $(objpfx)tst-dso-ordering1-dir/tst-dso-ordering1-a.os: $(objpfx)dso-sort-tests-src/tst-dso-ordering1-a.c $(compile.c) $(OUTPUT_OPTION) to avoid random $< values from $(before-compile) when compiling test relocatable objects with $(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c) compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags) compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS) for 3 "make -j 28" parallel builds on a machine with 112 cores at the same time. This partially fixes BZ #28550. Reviewed-by: Adhemerval Zanella --- scripts/dso-ordering-test.py | 47 ++++++++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 13 deletions(-) diff --git a/scripts/dso-ordering-test.py b/scripts/dso-ordering-test.py index 944ee74052..bde0406be9 100644 --- a/scripts/dso-ordering-test.py +++ b/scripts/dso-ordering-test.py @@ -526,9 +526,13 @@ def process_testcase(t): base_test_name = t.test_name test_subdir = base_test_name + "-dir" testpfx = objpfx + test_subdir + "/" + test_srcdir = "dso-sort-tests-src/" + testpfx_src = objpfx + test_srcdir if not os.path.exists(testpfx): os.mkdir(testpfx) + if not os.path.exists(testpfx_src): + os.mkdir(testpfx_src) def find_objs_not_depended_on(t): objs_not_depended_on = [] @@ -595,6 +599,11 @@ def process_testcase(t): # Print out needed Makefile fragments for use in glibc/elf/Makefile. module_names = "" for o in test_descr.objs: + rule = ("$(objpfx)" + test_subdir + "/" + test_name + + "-" + o + ".os: $(objpfx)" + test_srcdir + + test_name + "-" + o + ".c\n" + "\t$(compile.c) $(OUTPUT_OPTION)\n") + makefile.write (rule) module_names += " " + test_subdir + "/" + test_name + "-" + o makefile.write("modules-names +=%s\n" % (module_names)) @@ -637,7 +646,7 @@ def process_testcase(t): # object. This only needs to be done at most once for # an object name. if not dep in fake_created: - f = open(testpfx + test_name + "-" + dep + f = open(testpfx_src + test_name + "-" + dep + ".FAKE.c", "w") f.write(" \n") f.close() @@ -648,6 +657,12 @@ def process_testcase(t): % (test_name + "-" + dep + ".FAKE.so", ("$(objpfx)" + test_subdir + "/" + test_name + "-" + dep + ".so"))) + rule = ("$(objpfx)" + test_subdir + "/" + + test_name + "-" + dep + ".FAKE.os: " + "$(objpfx)" + test_srcdir + + test_name + "-" + dep + ".FAKE.c\n" + "\t$(compile.c) $(OUTPUT_OPTION)\n") + makefile.write (rule) makefile.write \ ("modules-names += %s\n" % (test_subdir + "/" @@ -687,6 +702,10 @@ def process_testcase(t): + test_descr.soname_map['#'] + ".so") ldflags += (" -Wl,-soname=" + soname) makefile.write("LDFLAGS-%s = %s\n" % (test_name, ldflags)) + rule = ("$(objpfx)" + test_subdir + "/" + test_name + ".o: " + "$(objpfx)" + test_srcdir + test_name + ".c\n" + "\t$(compile.c) $(OUTPUT_OPTION)\n") + makefile.write (rule) not_depended_objs = find_objs_not_depended_on(test_descr) if not_depended_objs: @@ -745,7 +764,7 @@ def process_testcase(t): " something_failed=true\n" "else\n" " diff -wu ${common_objpfx}elf/%s/%s%s.output \\\n" - " ${common_objpfx}elf/%s/%s%s.exp\n" + " ${common_objpfx}elf/%s%s%s.exp\n" " if [ $? -ne 0 ]; then\n" " echo '%sFAIL: %s%s expected output comparison'\n" " something_failed=true\n" @@ -753,14 +772,14 @@ def process_testcase(t): "fi\n" % (("X" if xfail else ""), test_name, tunable_descr, test_subdir, test_name, tunable_sfx, - test_subdir, base_test_name, exp_tunable_sfx, + test_srcdir, base_test_name, exp_tunable_sfx, ("X" if xfail else ""), test_name, tunable_descr)) # Generate C files according to dependency and calling relations from # description string. for obj in test_descr.objs: src_name = test_name + "-" + obj + ".c" - f = open(testpfx + src_name, "w") + f = open(testpfx_src + src_name, "w") if obj in test_descr.callrefs: called_objs = test_descr.callrefs[obj] for callee in called_objs: @@ -804,7 +823,7 @@ def process_testcase(t): f.close() # Open C file for writing main program - f = open(testpfx + test_name + ".c", "w") + f = open(testpfx_src + test_name + ".c", "w") # if there are some operations in main(), it means we need -ldl f.write("#include \n") @@ -885,7 +904,7 @@ def process_testcase(t): for obj in test_descr.objs: src_name = test_name + "-" + obj + ".c" obj_name = test_name + "-" + obj + ".os" - run_cmd([build_gcc, "-c", "-fPIC", testpfx + src_name, + run_cmd([build_gcc, "-c", "-fPIC", testpfx_src + src_name, "-o", testpfx + obj_name]) obj_processed = {} @@ -903,10 +922,12 @@ def process_testcase(t): deps.append(dep + ".FAKE") if not dep in fake_created: base_name = testpfx + test_name + "-" + dep + src_base_name = (testpfx_src + test_name + + "-" + dep) cmd = [build_gcc, "-Wl,--no-as-needed", ("-Wl,-soname=" + base_name + ".so"), "-shared", base_name + ".FAKE.c", - "-o", base_name + ".FAKE.so"] + "-o", src_base_name + ".FAKE.so"] run_cmd(cmd) fake_created[dep] = True dso_deps = map(lambda d: testpfx + test_name + "-" + d + ".so", @@ -932,7 +953,7 @@ def process_testcase(t): main_deps = map(lambda d: testpfx + test_name + "-" + d + ".so", deps) cmd = [build_gcc, "-Wl,--no-as-needed", "-o", testpfx + test_name, - testpfx + test_name + ".c", "-L%s" % (os.getcwd()), + testpfx_src + test_name + ".c", "-L%s" % (os.getcwd()), "-Wl,-rpath-link=%s" % (os.getcwd())] if '#' in test_descr.soname_map: soname = ("-Wl,-soname=" + testpfx + test_name + "-" @@ -987,14 +1008,14 @@ def process_testcase(t): sfx = "" if r[0] != "": sfx = "-" + r[0].replace("=","_") - f = open(testpfx + t.test_name + sfx + ".exp", "w") + f = open(testpfx_src + t.test_name + sfx + ".exp", "w") (output, xfail) = r[1] f.write('%s' % output) f.close() # Create header part of top-level testcase shell script, to wrap execution # and output comparison together. - t.sh = open(testpfx + t.test_name + ".sh", "w") + t.sh = open(testpfx_src + t.test_name + ".sh", "w") t.sh.write("#!/bin/sh\n") t.sh.write("# Test driver for %s, generated by " "dso-ordering-test.py\n" % (t.test_name)) @@ -1022,12 +1043,12 @@ def process_testcase(t): sfx = "" if r[0] != "": sfx = "-" + r[0].replace("=","_") - expected_output_files += " $(objpfx)%s/%s%s.exp" % (test_subdir, + expected_output_files += " $(objpfx)%s%s%s.exp" % (test_srcdir, t.test_name, sfx) makefile.write \ - ("$(objpfx)%s.out: $(objpfx)%s/%s.sh%s " + ("$(objpfx)%s.out: $(objpfx)%s%s.sh%s " "$(common-objpfx)support/test-run-command\n" - % (t.test_name, test_subdir, t.test_name, + % (t.test_name, test_srcdir, t.test_name, expected_output_files)) makefile.write("\t$(SHELL) $< $(common-objpfx) '$(test-wrapper-env)' " "'$(run-program-env)' > $@; $(evaluate-test)\n") From patchwork Mon Nov 8 15:42:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 47219 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C7A763858020 for ; Mon, 8 Nov 2021 15:44:07 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C7A763858020 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1636386247; bh=jEf6dRwo8WBYTjvLXF/Sg3R9Bg78FXXQsuT7cxL9IHc=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=FJ1lGdzdaSJAI9+sAjw3BlGqYKwSXG1qR9PW5pxxwxghwMJ+AeMynSQ3JZU9CRX37 vypqlAOXbfWCwrYTP7+A14x2NlJB5Ipas0ZVlFEhLlcAUAeeL9ffF76lxzp7T4oRnS Mmmp0UJwE2Cqdry01JoK0Vt6mvZHciaaQBx3Eayc= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by sourceware.org (Postfix) with ESMTPS id 3A6963858C2C for ; Mon, 8 Nov 2021 15:42:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 3A6963858C2C Received: by mail-pj1-x1031.google.com with SMTP id v23so6460709pjr.5 for ; Mon, 08 Nov 2021 07:42:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jEf6dRwo8WBYTjvLXF/Sg3R9Bg78FXXQsuT7cxL9IHc=; b=YK76VEdGCXTSrBBw7TEVPb+fwYIeSlMk13ZIwlTvOicSDvoKRdU9TShRibtf+ySt2s qVGCZHPMXjonAbsG+eEdjNtsvoleLSh35gpvvOL3PhjEVJoijFsPwIRJEZSMv0pmqDjG wIUiSnNXK9UpuQ2G7dQLdbBw321sty3FC4tRbGdI0/1ktaVYljH4oeCXnR4Ssnp0NyPG Sfdup1vHzvQYB/tf4TT7Ag7gw5AyAP8r3npyauVi4K7LXgqB54SJ699DdMRO3HiQUAWA YDqng/XPTJ7TZDxSdgub66vTAxp40VyV7DqkEiKRDhoKu1+5vhXYhEo/5UTt8UNBYjSn VA8w== X-Gm-Message-State: AOAM533+aYGTi1ZgDKznn4ZYP7rIrk+LeTwCpI+4uhhaX2O+5KqqPkTW GpqOWo13/v8EF8v6h6kdhtoCGfH+f58= X-Google-Smtp-Source: ABdhPJw1kKxsfiFkeqjwM/7TFrXc306hLD78w9I2000U9KUPHAnTnapliW1eJDhDe2XN7uW82jxW9A== X-Received: by 2002:a17:902:b909:b0:13a:2d8e:12bc with SMTP id bf9-20020a170902b90900b0013a2d8e12bcmr230029plb.6.1636386133265; Mon, 08 Nov 2021 07:42:13 -0800 (PST) Received: from gnu-cfl-2.localdomain ([2607:fb90:a63f:468b:b937:402b:0:c66]) by smtp.gmail.com with ESMTPSA id v10sm4180907pfg.162.2021.11.08.07.42.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Nov 2021 07:42:13 -0800 (PST) Received: from gnu-cfl-2.lan (localhost [IPv6:::1]) by gnu-cfl-2.localdomain (Postfix) with ESMTP id D932D1A0946; Mon, 8 Nov 2021 07:42:11 -0800 (PST) To: libc-alpha@sourceware.org Subject: [PATCH v2 2/2] elf: Pre-generate Makefile fragment for DSO sorting tests [BZ #28550] Date: Mon, 8 Nov 2021 07:42:11 -0800 Message-Id: <20211108154211.167829-3-hjl.tools@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211108154211.167829-1-hjl.tools@gmail.com> References: <20211108154211.167829-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3031.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "H.J. Lu via Libc-alpha" From: "H.J. Lu" Reply-To: "H.J. Lu" Cc: Florian Weimer Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Generate Makefile fragment for DSO sorting tests at configure time. This partially fixes BZ #28550. --- configure | 11 +++++++++++ configure.ac | 12 ++++++++++++ elf/Makefile | 22 +++++----------------- 3 files changed, 28 insertions(+), 17 deletions(-) diff --git a/configure b/configure index 2f9adca064..91993da3b9 100755 --- a/configure +++ b/configure @@ -8484,3 +8484,14 @@ if test -n "$ac_unrecognized_opts" && test "$enable_option_checking" != no; then $as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2;} fi + +if test "$have_tunables" = yes; then + test -d elf || mkdir elf + objdir=`pwd` + rm -f elf/dso-sort-tests.mk + (for d in dso-sort-tests-1.def dso-sort-tests-2.def + do + $PYTHON $srcdir/scripts/dso-ordering-test.py \ + --description-file $srcdir/elf/$d --objpfx $objdir/elf/ + done) > elf/dso-sort-tests.mk +fi diff --git a/configure.ac b/configure.ac index 7eb4239359..faf70b930c 100644 --- a/configure.ac +++ b/configure.ac @@ -1901,3 +1901,15 @@ echo "$config_vars" >> config.make;; esac test -d bits || mkdir bits]],[[config_vars='$config_vars']]) AC_OUTPUT + +dnl Generate Makefile fragments for DSO sorting tests. +if test "$have_tunables" = yes; then + test -d elf || mkdir elf + objdir=`pwd` + rm -f elf/dso-sort-tests.mk + (for d in dso-sort-tests-1.def dso-sort-tests-2.def + do + $PYTHON $srcdir/scripts/dso-ordering-test.py \ + --description-file $srcdir/elf/$d --objpfx $objdir/elf/ + done) > elf/dso-sort-tests.mk +fi diff --git a/elf/Makefile b/elf/Makefile index 41c19668c3..6995ccaa0f 100644 --- a/elf/Makefile +++ b/elf/Makefile @@ -366,6 +366,11 @@ modules-names = testobj1 testobj2 testobj3 testobj4 testobj5 testobj6 \ tst-auxvalmod \ tst-dlmopen-gethostbyname-mod tst-ro-dynamic-mod \ +ifeq (yes,$(have-tunables)) +# Include Makefile fragments for DSO sorting tests. +include $(objpfx)dso-sort-tests.mk +endif + # Most modules build with _ISOMAC defined, but those filtered out # depend on internal headers. modules-names-tests = $(filter-out ifuncmod% tst-tlsmod%,\ @@ -486,23 +491,6 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \ $(objpfx)tst-unused-dep-cmp.out endif -# DSO sorting tests: -# The dso-ordering-test.py script generates testcase source files in $(objpfx), -# creating a $(objpfx)-dir for each testcase, and creates a -# Makefile fragment to be included. -define include_dsosort_tests -$(objpfx)$(1).generated-makefile: $(1) - $(PYTHON) $(..)scripts/dso-ordering-test.py \ - --description-file $$< --objpfx $(objpfx) --output-makefile $$@ -include $(objpfx)$(1).generated-makefile -endef - -# Generate from each testcase description file -ifeq (yes,$(have-tunables)) -$(eval $(call include_dsosort_tests,dso-sort-tests-1.def)) -$(eval $(call include_dsosort_tests,dso-sort-tests-2.def)) -endif - check-abi: $(objpfx)check-abi-ld.out tests-special += $(objpfx)check-abi-ld.out update-abi: update-abi-ld