From patchwork Tue Jul 19 14:01:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 56161 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 4CCD23857BBA for ; Tue, 19 Jul 2022 14:01:40 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 4CCD23857BBA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1658239300; bh=cQ9TbENJC67b87a8DgVEQVwaTeDJsrAfgdgcSzaR3vw=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=Qtlun8OiRdbTwLL5Z/UaJC0lu4y+2fEcg3SA2d06ry/n3FVr/gOeQ14ixGHY8MXCO GRntsanil40kNaWVboY94rppSQlvKTKpoDnsCXnQH7rTqBuspJ+OqOL5zKW/gsy/iI Xfowb48HQC001O2Lb3JkCjB35/RX2vmVCHdbBeQQ= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oa1-x31.google.com (mail-oa1-x31.google.com [IPv6:2001:4860:4864:20::31]) by sourceware.org (Postfix) with ESMTPS id 4E1B43858439 for ; Tue, 19 Jul 2022 14:01:18 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 4E1B43858439 Received: by mail-oa1-x31.google.com with SMTP id 586e51a60fabf-f2a4c51c45so31873233fac.9 for ; Tue, 19 Jul 2022 07:01:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=cQ9TbENJC67b87a8DgVEQVwaTeDJsrAfgdgcSzaR3vw=; b=l9YNtgjavejkvivHKBQkHhLmCmdt6lN1iZiC0zqQ6/IIQFi7nRQWv/tQW6/8NphZuW 5MaU3eZPdobU+kkY/2GkrMg5pegOzyxD0EEHirHSOJU4+dI+9yXVEhV59k7KKQhM8cQa BeAtarQNy0UBCFXU2PLVRpnGMbZyUR6JMl9lo/IMPZPJvUYQFyO+Yc01fy931ExZCVVh CFniL1vztXGpEyJrd0coKRHxwp+YtGaONC364n5z+YNj+VW71RNgd8gP+pqTc5nQCa+1 54qNnRMnKAweMCSbojWCAkL+AoyfYyseJLg2wKRmIPPZVCorU9RHXEGrvHfI6XO9zTuk ieuA== X-Gm-Message-State: AJIora+5Pehe3F116Dh6JP0JA80qPR+4hnJEQoJqG9drVotxM5OgLlJe f468ewvDtWkl7uFO0Hpw9xoQmPFWE+WbZR+l X-Google-Smtp-Source: AGRyM1siVnyLzjs9tvoqf+OBXRt7HdJ+q6W4zDWNV4gOvowxCC69+oTa5+a6l9D933dTMDmozokdbA== X-Received: by 2002:a05:6870:a18c:b0:10c:29cd:4b2a with SMTP id a12-20020a056870a18c00b0010c29cd4b2amr18143651oaf.261.1658239277272; Tue, 19 Jul 2022 07:01:17 -0700 (PDT) Received: from mandiga.. ([2804:431:c7ca:19c3:f70e:2756:6667:520a]) by smtp.gmail.com with ESMTPSA id k10-20020a9d7dca000000b0061cae9aba22sm1359393otn.71.2022.07.19.07.01.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Jul 2022 07:01:16 -0700 (PDT) To: libc-alpha@sourceware.org, Mark Wielaard , Carlos O'Donell Subject: [PATCH] locale: Optimize tst-localedef-path-norm Date: Tue, 19 Jul 2022 11:01:13 -0300 Message-Id: <20220719140113.1604672-1-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-12.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" The locale generation are issues in parallel to try speed locale generation. The maximum number of jobs are limited to the online CPU (in hope to not overcommit on environments with lower cores than tests). On a Ryzen 9, the test execution improves from ~6.7s to ~1.4s. Tested-by: Mark Wielaard --- locale/Makefile | 2 + locale/tst-localedef-path-norm.c | 237 ++++++++++++++++--------------- 2 files changed, 128 insertions(+), 111 deletions(-) diff --git a/locale/Makefile b/locale/Makefile index c315683b3f..eb55750496 100644 --- a/locale/Makefile +++ b/locale/Makefile @@ -116,3 +116,5 @@ include $(patsubst %,$(..)libof-iterator.mk,$(cpp-srcs-left)) $(objpfx)tst-locale-locpath.out : tst-locale-locpath.sh $(objpfx)locale $(SHELL) $< '$(common-objpfx)' '$(test-wrapper-env)' '$(run-program-env)' > $@; \ $(evaluate-test) + +$(objpfx)tst-localedef-path-norm: $(shared-thread-library) diff --git a/locale/tst-localedef-path-norm.c b/locale/tst-localedef-path-norm.c index 68995a415d..97cc1cc396 100644 --- a/locale/tst-localedef-path-norm.c +++ b/locale/tst-localedef-path-norm.c @@ -29,6 +29,7 @@ present, regardless of verbosity. POSIX requires that any warnings cause the exit status to be non-zero. */ +#include #include #include #include @@ -37,9 +38,10 @@ #include #include #include +#include /* Full path to localedef. */ -char *prog; +static const char *prog; /* Execute localedef in a subprocess. */ static void @@ -63,12 +65,13 @@ struct test_closure /* Run localedef with DATA.ARGV arguments (NULL terminated), and expect path to the compiled locale is "DATA.COMPLOCALEDIR/DATA.EXP". */ -static void -run_test (struct test_closure data) +static void * +run_test (void *closure) { - const char * const *args = data.argv; - const char *exp = data.exp; - const char *complocaledir = data.complocaledir; + struct test_closure *data = closure; + const char * const *args = data->argv; + const char *exp = data->exp; + const char *complocaledir = data->complocaledir; struct stat64 fs; /* Expected output path. */ @@ -82,8 +85,14 @@ run_test (struct test_closure data) /* Verify path is present and is a directory. */ xstat (path, &fs); - TEST_VERIFY_EXIT (S_ISDIR (fs.st_mode)); - printf ("info: Directory '%s' exists.\n", path); + if (!S_ISDIR (fs.st_mode)) + { + support_record_failure (); + printf ("error: Directory '%s' does not exist.\n", path); + return (void *) -1ULL; + } + + return NULL; } static int @@ -99,139 +108,145 @@ do_test (void) #define ABSDIR "/output" xmkdirp (ABSDIR, 0777); - /* It takes ~10 seconds to serially execute 9 localedef test. We - could run the compilations in parallel if we want to reduce test - time. We don't want to split this out into distinct tests because - it would require multiple chroots. Batching the same localedef - tests saves disk space during testing. */ + /* It takes ~10 seconds to serially execute 9 localedef test. We run the + compilations in parallel to reduce test time. We don't want to split + this out into distinct tests because it would require multiple chroots. + Batching the same localedef tests saves disk space during testing. */ + struct test_closure tests[] = + { /* Test 1: Expected normalization. Run localedef and expect output in $(complocaledir)/en_US1.utf8, with normalization changing UTF-8 to utf8. */ - run_test ((struct test_closure) - { - .argv = { prog, - "--no-archive", - "-i", "en_US", - "-f", "UTF-8", - "en_US1.UTF-8", NULL }, - .exp = "en_US1.utf8", - .complocaledir = support_complocaledir_prefix - }); - + { + .argv = { (const char *const) prog, + "--no-archive", + "-i", "en_US", + "-f", "UTF-8", + "en_US1.UTF-8", NULL }, + .exp = "en_US1.utf8", + .complocaledir = support_complocaledir_prefix + }, /* Test 2: No normalization past '@'. Run localedef and expect output in $(complocaledir)/en_US2.utf8@tEsT, with normalization changing UTF-8@tEsT to utf8@tEsT (everything after @ is untouched). */ - run_test ((struct test_closure) - { - .argv = { prog, - "--no-archive", - "-i", "en_US", - "-f", "UTF-8", - "en_US2.UTF-8@tEsT", NULL }, - .exp = "en_US2.utf8@tEsT", - .complocaledir = support_complocaledir_prefix - }); - + { + .argv = { prog, + "--no-archive", + "-i", "en_US", + "-f", "UTF-8", + "en_US2.UTF-8@tEsT", NULL }, + .exp = "en_US2.utf8@tEsT", + .complocaledir = support_complocaledir_prefix + }, /* Test 3: No normalization past '@' despite period. Run localedef and expect output in $(complocaledir)/en_US3@tEsT.UTF-8, with normalization changing nothing (everything after @ is untouched) despite there being a period near the end. */ - run_test ((struct test_closure) - { - .argv = { prog, - "--no-archive", - "-i", "en_US", - "-f", "UTF-8", - "en_US3@tEsT.UTF-8", NULL }, - .exp = "en_US3@tEsT.UTF-8", - .complocaledir = support_complocaledir_prefix - }); - + { + .argv = { prog, + "--no-archive", + "-i", "en_US", + "-f", "UTF-8", + "en_US3@tEsT.UTF-8", NULL }, + .exp = "en_US3@tEsT.UTF-8", + .complocaledir = support_complocaledir_prefix + }, /* Test 4: Normalize numeric codeset by adding 'iso' prefix. Run localedef and expect output in $(complocaledir)/en_US4.88591, with normalization changing 88591 to iso88591. */ - run_test ((struct test_closure) - { - .argv = { prog, - "--no-archive", - "-i", "en_US", - "-f", "UTF-8", - "en_US4.88591", NULL }, - .exp = "en_US4.iso88591", - .complocaledir = support_complocaledir_prefix - }); - + { + .argv = { prog, + "--no-archive", + "-i", "en_US", + "-f", "UTF-8", + "en_US4.88591", NULL }, + .exp = "en_US4.iso88591", + .complocaledir = support_complocaledir_prefix + }, /* Test 5: Don't add 'iso' prefix if first char is alpha. Run localedef and expect output in $(complocaledir)/en_US5.a88591, with normalization changing nothing. */ - run_test ((struct test_closure) - { - .argv = { prog, - "--no-archive", - "-i", "en_US", - "-f", "UTF-8", - "en_US5.a88591", NULL }, - .exp = "en_US5.a88591", - .complocaledir = support_complocaledir_prefix - }); - + { + .argv = { prog, + "--no-archive", + "-i", "en_US", + "-f", "UTF-8", + "en_US5.a88591", NULL }, + .exp = "en_US5.a88591", + .complocaledir = support_complocaledir_prefix + }, /* Test 6: Don't add 'iso' prefix if last char is alpha. Run localedef and expect output in $(complocaledir)/en_US6.88591a, with normalization changing nothing. */ - run_test ((struct test_closure) - { - .argv = { prog, - "--no-archive", - "-i", "en_US", - "-f", "UTF-8", - "en_US6.88591a", NULL }, - .exp = "en_US6.88591a", - .complocaledir = support_complocaledir_prefix - }); - + { + .argv = { prog, + "--no-archive", + "-i", "en_US", + "-f", "UTF-8", + "en_US6.88591a", NULL }, + .exp = "en_US6.88591a", + .complocaledir = support_complocaledir_prefix + }, /* Test 7: Don't normalize anything with an absolute path. Run localedef and expect output in ABSDIR/en_US7.UTF-8, with normalization changing nothing. */ - run_test ((struct test_closure) - { - .argv = { prog, - "--no-archive", - "-i", "en_US", - "-f", "UTF-8", - ABSDIR "/en_US7.UTF-8", NULL }, - .exp = "en_US7.UTF-8", - .complocaledir = ABSDIR - }); - + { + .argv = { prog, + "--no-archive", + "-i", "en_US", + "-f", "UTF-8", + ABSDIR "/en_US7.UTF-8", NULL }, + .exp = "en_US7.UTF-8", + .complocaledir = ABSDIR + }, /* Test 8: Don't normalize anything with an absolute path. Run localedef and expect output in ABSDIR/en_US8.UTF-8@tEsT, with normalization changing nothing. */ - run_test ((struct test_closure) - { - .argv = { prog, - "--no-archive", - "-i", "en_US", - "-f", "UTF-8", - ABSDIR "/en_US8.UTF-8@tEsT", NULL }, - .exp = "en_US8.UTF-8@tEsT", - .complocaledir = ABSDIR - }); - + { + .argv = { prog, + "--no-archive", + "-i", "en_US", + "-f", "UTF-8", + ABSDIR "/en_US8.UTF-8@tEsT", NULL }, + .exp = "en_US8.UTF-8@tEsT", + .complocaledir = ABSDIR + }, /* Test 9: Don't normalize anything with an absolute path. Run localedef and expect output in ABSDIR/en_US9@tEsT.UTF-8, with normalization changing nothing. */ - run_test ((struct test_closure) - { - .argv = { prog, - "--no-archive", - "-i", "en_US", - "-f", "UTF-8", - ABSDIR "/en_US9@tEsT.UTF-8", NULL }, - .exp = "en_US9@tEsT.UTF-8", - .complocaledir = ABSDIR - }); + { + .argv = { prog, + "--no-archive", + "-i", "en_US", + "-f", "UTF-8", + ABSDIR "/en_US9@tEsT.UTF-8", NULL }, + .exp = "en_US9@tEsT.UTF-8", + .complocaledir = ABSDIR + } + }; + + /* Do not run more threads than the maximum of online CPUs. */ + size_t ntests = array_length (tests); + long int cpus = sysconf (_SC_NPROCESSORS_ONLN); + cpus = cpus == -1 ? 1 : cpus; + printf ("info: cpus=%ld ntests=%zu\n", cpus, ntests); + + pthread_t thr[ntests]; + + for (int i = 0; i < ntests; i += cpus) + { + int max = i + cpus; + if (max > ntests) + max = ntests; + + for (int j = i; j < max; j++) + thr[j] = xpthread_create (NULL, run_test, &tests[j]); + + for (int j = i; j < max; j++) + TEST_VERIFY_EXIT (xpthread_join (thr[j]) == NULL); + } return 0; }