From patchwork Sun Nov 7 15:07:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 47179 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8EC2D3857C67 for ; Sun, 7 Nov 2021 15:07:52 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 8EC2D3857C67 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1636297672; bh=73yYB4YgwqehoaE44TYPckg+5HQBNAtMSpJwcx2yF1M=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=g50QJNci34hRB2r6agp/2WlqbUtu8gj5tc9vpGpcnNwuWgl7ChkeCH11czdhDzdlH lDCQd9QnIVZy3NEDcONcNmrMRkWqcA0JquM/8zVZ9mjqpIkt7FXS53U3fg3o6pRLgI 6WNw/p31DADC2kwH5ixUvmRuz4mDZwQI4Z2SgXwE= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by sourceware.org (Postfix) with ESMTPS id 789AC3858403 for ; Sun, 7 Nov 2021 15:07:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 789AC3858403 Received: by mail-pg1-x529.google.com with SMTP id n23so12862300pgh.8 for ; Sun, 07 Nov 2021 07:07:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=73yYB4YgwqehoaE44TYPckg+5HQBNAtMSpJwcx2yF1M=; b=VZO8bWGhTBmCi2IUNU9PtvxkdVGcVg/WcD8EbL+2kSOTYkE17UxihKVybh63L9l0N3 vsuh4lBuIIBB6moyHNhe/iiBptvvczV2sRqR0s9X1zRpGrQiG4mqiJq92W3GpOfN4nJe keB/Xnc7f2Gt8HlPtJO4dkyC8G2VjrgOB0nxTMj2LcGZmkFVfj3pLbcawmRUU7SRo3EP 1Gb9s8ZroDFNJcdL+5fma8vcfGDRaS0MyczkYiN+OkGZ1C/NlEgsUor+kLD6RJdZETol EKHghWKwaYaVPKLo/Qke6HyOHtKaLoDeYuKjoxJjCLEfZS+00YWv5bWgkom2D37swDiI oOIg== X-Gm-Message-State: AOAM530GiRtdwOx1Lr2e7r6Ypj3Onovm2zZLD+dI6701kHivEn/i/Shs BJtrUK1h/Zh0gyI86JSJJUUZFbSh/YI= X-Google-Smtp-Source: ABdhPJyjBsrlPGgydBvj7l52UV4Ewcr3s+57H/CA7o7sLb51TVCcXWDERRNeMS3dpSbzx5caI5TiLA== X-Received: by 2002:aa7:8059:0:b0:47e:5de6:5bc7 with SMTP id y25-20020aa78059000000b0047e5de65bc7mr70998445pfm.78.1636297649289; Sun, 07 Nov 2021 07:07:29 -0800 (PST) Received: from gnu-cfl-2.localdomain ([172.58.35.133]) by smtp.gmail.com with ESMTPSA id oa2sm9071120pjb.53.2021.11.07.07.07.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 07 Nov 2021 07:07:29 -0800 (PST) Received: from gnu-cfl-2.lan (localhost [IPv6:::1]) by gnu-cfl-2.localdomain (Postfix) with ESMTP id D7FA81A0943 for ; Sun, 7 Nov 2021 07:07:27 -0800 (PST) To: libc-alpha@sourceware.org Subject: [PATCH 1/2] dso-ordering-test.py: Put all sources in one directory [BZ #28550] Date: Sun, 7 Nov 2021 07:07:26 -0800 Message-Id: <20211107150727.157003-2-hjl.tools@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211107150727.157003-1-hjl.tools@gmail.com> References: <20211107150727.157003-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3030.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "H.J. Lu via Libc-alpha" From: "H.J. Lu" Reply-To: "H.J. Lu" Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Put all sources for DSO sorting tests in the dso-sort-tests-src directory and compile tests with "$(compile.c) $(OUTPUT_OPTION)" to avoid random $< values from $(before-compile) when compiling tests with $(objpfx)%$o: $(objpfx)%.c $(before-compile); $$(compile-command.c) compile-command.c = $(compile.c) $(OUTPUT_OPTION) $(compile-mkdep-flags) compile.c = $(CC) $< -c $(CFLAGS) $(CPPFLAGS) for 3 "make -j 28" parallel builds on a machine with 112 cores at the same time. Also use a temporary file to generate Makefile fragments for DSO sorting tests. This fixes BZ #28550. --- scripts/dso-ordering-test.py | 47 ++++++++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 13 deletions(-) diff --git a/scripts/dso-ordering-test.py b/scripts/dso-ordering-test.py index 944ee74052..26497159a0 100644 --- a/scripts/dso-ordering-test.py +++ b/scripts/dso-ordering-test.py @@ -526,9 +526,13 @@ def process_testcase(t): base_test_name = t.test_name test_subdir = base_test_name + "-dir" testpfx = objpfx + test_subdir + "/" + test_srcdir = "dso-sort-tests-src/" + testpfx_src = objpfx + test_srcdir if not os.path.exists(testpfx): os.mkdir(testpfx) + if not os.path.exists(testpfx_src): + os.mkdir(testpfx_src) def find_objs_not_depended_on(t): objs_not_depended_on = [] @@ -595,6 +599,11 @@ def process_testcase(t): # Print out needed Makefile fragments for use in glibc/elf/Makefile. module_names = "" for o in test_descr.objs: + rule = ("$(objpfx)" + test_subdir + "/" + test_name + + "-" + o + ".os: $(objpfx)" + test_srcdir + + test_name + "-" + o + ".c\n" + "\t$(compile.c) $(OUTPUT_OPTION)\n") + makefile.write (rule); module_names += " " + test_subdir + "/" + test_name + "-" + o makefile.write("modules-names +=%s\n" % (module_names)) @@ -637,7 +646,7 @@ def process_testcase(t): # object. This only needs to be done at most once for # an object name. if not dep in fake_created: - f = open(testpfx + test_name + "-" + dep + f = open(testpfx_src + test_name + "-" + dep + ".FAKE.c", "w") f.write(" \n") f.close() @@ -648,6 +657,12 @@ def process_testcase(t): % (test_name + "-" + dep + ".FAKE.so", ("$(objpfx)" + test_subdir + "/" + test_name + "-" + dep + ".so"))) + rule = ("$(objpfx)" + test_subdir + "/" + + test_name + "-" + dep + ".FAKE.os: " + "$(objpfx)" + test_srcdir + + test_name + "-" + dep + ".FAKE.c\n" + "\t$(compile.c) $(OUTPUT_OPTION)\n") + makefile.write (rule); makefile.write \ ("modules-names += %s\n" % (test_subdir + "/" @@ -687,6 +702,10 @@ def process_testcase(t): + test_descr.soname_map['#'] + ".so") ldflags += (" -Wl,-soname=" + soname) makefile.write("LDFLAGS-%s = %s\n" % (test_name, ldflags)) + rule = ("$(objpfx)" + test_subdir + "/" + test_name + ".o: " + "$(objpfx)" + test_srcdir + test_name + ".c\n" + "\t$(compile.c) $(OUTPUT_OPTION)\n") + makefile.write (rule); not_depended_objs = find_objs_not_depended_on(test_descr) if not_depended_objs: @@ -745,7 +764,7 @@ def process_testcase(t): " something_failed=true\n" "else\n" " diff -wu ${common_objpfx}elf/%s/%s%s.output \\\n" - " ${common_objpfx}elf/%s/%s%s.exp\n" + " ${common_objpfx}elf/%s%s%s.exp\n" " if [ $? -ne 0 ]; then\n" " echo '%sFAIL: %s%s expected output comparison'\n" " something_failed=true\n" @@ -753,14 +772,14 @@ def process_testcase(t): "fi\n" % (("X" if xfail else ""), test_name, tunable_descr, test_subdir, test_name, tunable_sfx, - test_subdir, base_test_name, exp_tunable_sfx, + test_srcdir, base_test_name, exp_tunable_sfx, ("X" if xfail else ""), test_name, tunable_descr)) # Generate C files according to dependency and calling relations from # description string. for obj in test_descr.objs: src_name = test_name + "-" + obj + ".c" - f = open(testpfx + src_name, "w") + f = open(testpfx_src + src_name, "w") if obj in test_descr.callrefs: called_objs = test_descr.callrefs[obj] for callee in called_objs: @@ -804,7 +823,7 @@ def process_testcase(t): f.close() # Open C file for writing main program - f = open(testpfx + test_name + ".c", "w") + f = open(testpfx_src + test_name + ".c", "w") # if there are some operations in main(), it means we need -ldl f.write("#include \n") @@ -885,7 +904,7 @@ def process_testcase(t): for obj in test_descr.objs: src_name = test_name + "-" + obj + ".c" obj_name = test_name + "-" + obj + ".os" - run_cmd([build_gcc, "-c", "-fPIC", testpfx + src_name, + run_cmd([build_gcc, "-c", "-fPIC", testpfx_src + src_name, "-o", testpfx + obj_name]) obj_processed = {} @@ -903,10 +922,12 @@ def process_testcase(t): deps.append(dep + ".FAKE") if not dep in fake_created: base_name = testpfx + test_name + "-" + dep + src_base_name = (testpfx_src + test_name + + "-" + dep) cmd = [build_gcc, "-Wl,--no-as-needed", ("-Wl,-soname=" + base_name + ".so"), "-shared", base_name + ".FAKE.c", - "-o", base_name + ".FAKE.so"] + "-o", src_base_name + ".FAKE.so"] run_cmd(cmd) fake_created[dep] = True dso_deps = map(lambda d: testpfx + test_name + "-" + d + ".so", @@ -932,7 +953,7 @@ def process_testcase(t): main_deps = map(lambda d: testpfx + test_name + "-" + d + ".so", deps) cmd = [build_gcc, "-Wl,--no-as-needed", "-o", testpfx + test_name, - testpfx + test_name + ".c", "-L%s" % (os.getcwd()), + testpfx_src + test_name + ".c", "-L%s" % (os.getcwd()), "-Wl,-rpath-link=%s" % (os.getcwd())] if '#' in test_descr.soname_map: soname = ("-Wl,-soname=" + testpfx + test_name + "-" @@ -987,14 +1008,14 @@ def process_testcase(t): sfx = "" if r[0] != "": sfx = "-" + r[0].replace("=","_") - f = open(testpfx + t.test_name + sfx + ".exp", "w") + f = open(testpfx_src + t.test_name + sfx + ".exp", "w") (output, xfail) = r[1] f.write('%s' % output) f.close() # Create header part of top-level testcase shell script, to wrap execution # and output comparison together. - t.sh = open(testpfx + t.test_name + ".sh", "w") + t.sh = open(testpfx_src + t.test_name + ".sh", "w") t.sh.write("#!/bin/sh\n") t.sh.write("# Test driver for %s, generated by " "dso-ordering-test.py\n" % (t.test_name)) @@ -1022,12 +1043,12 @@ def process_testcase(t): sfx = "" if r[0] != "": sfx = "-" + r[0].replace("=","_") - expected_output_files += " $(objpfx)%s/%s%s.exp" % (test_subdir, + expected_output_files += " $(objpfx)%s%s%s.exp" % (test_srcdir, t.test_name, sfx) makefile.write \ - ("$(objpfx)%s.out: $(objpfx)%s/%s.sh%s " + ("$(objpfx)%s.out: $(objpfx)%s%s.sh%s " "$(common-objpfx)support/test-run-command\n" - % (t.test_name, test_subdir, t.test_name, + % (t.test_name, test_srcdir, t.test_name, expected_output_files)) makefile.write("\t$(SHELL) $< $(common-objpfx) '$(test-wrapper-env)' " "'$(run-program-env)' > $@; $(evaluate-test)\n") From patchwork Sun Nov 7 15:07:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 47180 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 11E0C3858038 for ; Sun, 7 Nov 2021 15:08:41 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 11E0C3858038 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1636297721; bh=Tb/Y1wuLC+37WUp6wbwTTq3sPywPdYaRyM25VkkfWZA=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ZW6NAyMDWFhuM2e/rTLhuGw9jf7uE6hsj5cyYHxKCYIzYom0+PpIWiuSLZo5/1Aiz j5cAQDuEGbd5FnHMiLPmj66Rweimo53q3PTKrffyThDWLT0/dHY4NZ3ig5cBFbA8dP ZUheWoFb5k9u7m84VH8AyQR/3eOStn6sWRo2SFj8= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by sourceware.org (Postfix) with ESMTPS id 957593858406 for ; Sun, 7 Nov 2021 15:07:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 957593858406 Received: by mail-pj1-x1029.google.com with SMTP id nh10-20020a17090b364a00b001a69adad5ebso6866568pjb.2 for ; Sun, 07 Nov 2021 07:07:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Tb/Y1wuLC+37WUp6wbwTTq3sPywPdYaRyM25VkkfWZA=; b=gAwGVSf2eBCWCBXfF+CAEm3M9ZSAikxgeXadqIp5Df/LXjydfvWK4btLmtZK9kk9fm Rs3FKzSf2IPdinit/89sWYPC4AmKytEuLyNKAb9JuWQNum6jMZhO/H49UcIT/+Z/xYBN phrt3JILyjSso+9kYhp2MdovBCxbkc9nEmCXpTopQQmYmjtMscpsPxpe6WTkPRIHtyUT aJ7PGuhyHMLp+ePqYQUZi57aFit07yjED8jn+Dw1CAcK72RAiDW82lHXlpLj8zk/5YQR B6YJKWuyrIYsV4rLhNeJ4XWtMcAftI8DBThdPWS2LDOfa0QJ13pEHcIdqm/h8R5ox0Mu nCLw== X-Gm-Message-State: AOAM530Di8bscHQz5b4nIfhpAP1WpZS9DtKjltGCDUncQ5WJlZLX9Dq6 BZi1dJE/iGDc9DnT0oYdzEpkjHAN5yY= X-Google-Smtp-Source: ABdhPJyufu3JpeZq3GcUvF7Ia1WQnyraE7/cEHX/yH7ngybMrmdVFrNXHGjpgYl/KlClU6TJQMN9IQ== X-Received: by 2002:a17:902:b697:b0:141:c7aa:e10f with SMTP id c23-20020a170902b69700b00141c7aae10fmr51755380pls.18.1636297649452; Sun, 07 Nov 2021 07:07:29 -0800 (PST) Received: from gnu-cfl-2.localdomain ([172.58.35.133]) by smtp.gmail.com with ESMTPSA id lx12sm8583869pjb.5.2021.11.07.07.07.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 07 Nov 2021 07:07:29 -0800 (PST) Received: from gnu-cfl-2.lan (localhost [IPv6:::1]) by gnu-cfl-2.localdomain (Postfix) with ESMTP id D96651A0946 for ; Sun, 7 Nov 2021 07:07:27 -0800 (PST) To: libc-alpha@sourceware.org Subject: [PATCH 2/2] elf: Pre-generate Makefile fragments for DSO sorting tests [BZ #28550] Date: Sun, 7 Nov 2021 07:07:27 -0800 Message-Id: <20211107150727.157003-3-hjl.tools@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211107150727.157003-1-hjl.tools@gmail.com> References: <20211107150727.157003-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3030.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "H.J. Lu via Libc-alpha" From: "H.J. Lu" Reply-To: "H.J. Lu" Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Generate Makefile fragments for DSO sorting tests at configure time to avoid generate them in the elf directory during build. This fixes BZ #28550. --- configure | 11 +++++++++++ configure.ac | 12 ++++++++++++ elf/Makefile | 22 +++++----------------- 3 files changed, 28 insertions(+), 17 deletions(-) diff --git a/configure b/configure index 2f9adca064..91993da3b9 100755 --- a/configure +++ b/configure @@ -8484,3 +8484,14 @@ if test -n "$ac_unrecognized_opts" && test "$enable_option_checking" != no; then $as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2;} fi + +if test "$have_tunables" = yes; then + test -d elf || mkdir elf + objdir=`pwd` + rm -f elf/dso-sort-tests.mk + (for d in dso-sort-tests-1.def dso-sort-tests-2.def + do + $PYTHON $srcdir/scripts/dso-ordering-test.py \ + --description-file $srcdir/elf/$d --objpfx $objdir/elf/ + done) > elf/dso-sort-tests.mk +fi diff --git a/configure.ac b/configure.ac index 7eb4239359..faf70b930c 100644 --- a/configure.ac +++ b/configure.ac @@ -1901,3 +1901,15 @@ echo "$config_vars" >> config.make;; esac test -d bits || mkdir bits]],[[config_vars='$config_vars']]) AC_OUTPUT + +dnl Generate Makefile fragments for DSO sorting tests. +if test "$have_tunables" = yes; then + test -d elf || mkdir elf + objdir=`pwd` + rm -f elf/dso-sort-tests.mk + (for d in dso-sort-tests-1.def dso-sort-tests-2.def + do + $PYTHON $srcdir/scripts/dso-ordering-test.py \ + --description-file $srcdir/elf/$d --objpfx $objdir/elf/ + done) > elf/dso-sort-tests.mk +fi diff --git a/elf/Makefile b/elf/Makefile index 41c19668c3..6995ccaa0f 100644 --- a/elf/Makefile +++ b/elf/Makefile @@ -366,6 +366,11 @@ modules-names = testobj1 testobj2 testobj3 testobj4 testobj5 testobj6 \ tst-auxvalmod \ tst-dlmopen-gethostbyname-mod tst-ro-dynamic-mod \ +ifeq (yes,$(have-tunables)) +# Include Makefile fragments for DSO sorting tests. +include $(objpfx)dso-sort-tests.mk +endif + # Most modules build with _ISOMAC defined, but those filtered out # depend on internal headers. modules-names-tests = $(filter-out ifuncmod% tst-tlsmod%,\ @@ -486,23 +491,6 @@ tests-special += $(objpfx)order-cmp.out $(objpfx)tst-array1-cmp.out \ $(objpfx)tst-unused-dep-cmp.out endif -# DSO sorting tests: -# The dso-ordering-test.py script generates testcase source files in $(objpfx), -# creating a $(objpfx)-dir for each testcase, and creates a -# Makefile fragment to be included. -define include_dsosort_tests -$(objpfx)$(1).generated-makefile: $(1) - $(PYTHON) $(..)scripts/dso-ordering-test.py \ - --description-file $$< --objpfx $(objpfx) --output-makefile $$@ -include $(objpfx)$(1).generated-makefile -endef - -# Generate from each testcase description file -ifeq (yes,$(have-tunables)) -$(eval $(call include_dsosort_tests,dso-sort-tests-1.def)) -$(eval $(call include_dsosort_tests,dso-sort-tests-2.def)) -endif - check-abi: $(objpfx)check-abi-ld.out tests-special += $(objpfx)check-abi-ld.out update-abi: update-abi-ld