From patchwork Fri Oct 15 10:26:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Jelinek X-Patchwork-Id: 46266 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3EAE93857C76 for ; Fri, 15 Oct 2021 10:26:46 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 3EAE93857C76 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1634293606; bh=DZjatpaiwiY6vFdtWZqfjSyrPqmIFjIHqyD4aHZWpfk=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=G126+RhZI5NwGtYNXYUuiDczdrMWzUXN7L0VoujkU5lokqcf0iwpwo1YGgRnixPQH bON8ZTeQnCK7kLRGj6Z29esPyOBgBZs5oB9vGEUPhtvj7606fOi42xKnO3VoE8JPol g58kRXyioDSwGUNOYBTitvSsJcnHOrziSLVVZFHg= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTP id C64D93858005 for ; Fri, 15 Oct 2021 10:26:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C64D93858005 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-75-OGjQLRz2PUKolauGnp1QIQ-1; Fri, 15 Oct 2021 06:26:12 -0400 X-MC-Unique: OGjQLRz2PUKolauGnp1QIQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EF5CB100CC84; Fri, 15 Oct 2021 10:26:10 +0000 (UTC) Received: from tucnak.zalov.cz (unknown [10.39.193.172]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 77BAA2B060; Fri, 15 Oct 2021 10:26:10 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.16.1/8.16.1) with ESMTPS id 19FAQ7QY4132772 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Fri, 15 Oct 2021 12:26:08 +0200 Received: (from jakub@localhost) by tucnak.zalov.cz (8.16.1/8.16.1/Submit) id 19FAQ6IS4132771; Fri, 15 Oct 2021 12:26:06 +0200 Date: Fri, 15 Oct 2021 12:26:06 +0200 To: gcc-patches@gcc.gnu.org Subject: [committed] openmp: Add support for OMP_PLACES=ll_caches Message-ID: <20211015102606.GB304296@tucnak> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jakub Jelinek via Gcc-patches From: Jakub Jelinek Reply-To: Jakub Jelinek Cc: Tobias Burnus Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" Hi! This patch implements support for ll_caches abstract name in OMP_PLACES, which stands for places where logical cpus in each place share the last level cache. This seems to work fine for me on x86 and kernel sources show that it is in common code, but on some machines on CompileFarm the files I'm using, i.e. /sys/devices/system/cpu/cpuN/cache/indexN/level /sys/devices/system/cpu/cpuN/cache/indexN/shared_cpu_list don't exist, is that because they have too old kernel and newer kernels are fine or should I implement some fallback methods (which)? E.g. on gcc112.fsffrance.org I see just shared_cpu_map and not shared_cpu_list (with shared_cpu_map being harder to parse) and on another box I didn't even see the cache subdirectories. Way to test this is OMP_PLACES=ll_caches OMP_DISPLAY_ENV=true LD_PRELOAD=.libs/libgomp.so.1 /bin/true and see what it prints on OMP_PLACES line. Bootstrapped/regtested on x86_64-linux and i686-linux, committed to trunk. 2021-10-15 Jakub Jelinek * env.c (parse_places_var): Handle ll_caches as level 4. * config/linux/affinity.c (gomp_affinity_find_last_cache_level): New function. (gomp_affinity_init_level_1): Handle level 4 as logical cpus sharing last level cache. (gomp_affinity_init_level): Likewise. * testsuite/libgomp.c/places-1.c: New test. * testsuite/libgomp.c/places-2.c: New test. * testsuite/libgomp.c/places-3.c: New test. * testsuite/libgomp.c/places-4.c: New test. Jakub --- libgomp/env.c.jj 2021-10-11 12:20:21.926063118 +0200 +++ libgomp/env.c 2021-10-14 15:25:48.816212823 +0200 @@ -696,6 +696,11 @@ parse_places_var (const char *name, bool env += 7; level = 3; } + else if (strncasecmp (env, "ll_caches", 9) == 0) + { + env += 9; + level = 4; + } if (level) { count = ULONG_MAX; --- libgomp/config/linux/affinity.c.jj 2021-07-13 09:50:46.270677237 +0200 +++ libgomp/config/linux/affinity.c 2021-10-14 17:13:12.811302863 +0200 @@ -223,6 +223,46 @@ gomp_affinity_finalize_place_list (bool return true; } +/* Find the index of the last level cache. We assume the index + of the last level cache is the same for all logical CPUs. + Also, if there are multiple caches with the same highest level, + assume they have the same shared_cpu_list and pick the last one + from them (highest index number). */ + +static int +gomp_affinity_find_last_cache_level (char *name, size_t prefix_len, + unsigned long cpu) +{ + int ret = -1; + unsigned long maxval = 0; + char *line = NULL; + size_t linelen = 0; + FILE *f; + + for (int l = 0; l < 128; l++) + { + sprintf (name + prefix_len, "%lu/cache/index%u/level", cpu, l); + f = fopen (name, "r"); + if (f == NULL) + break; + if (getline (&line, &linelen, f) > 0) + { + unsigned long val; + char *p; + errno = 0; + val = strtoul (line, &p, 10); + if (!errno && val >= maxval) + { + ret = l; + maxval = val; + } + } + fclose (f); + } + free (line); + return ret; +} + static void gomp_affinity_init_level_1 (int level, int this_level, unsigned long count, cpu_set_t *copy, char *name, bool quiet) @@ -232,12 +272,29 @@ gomp_affinity_init_level_1 (int level, i char *line = NULL; size_t linelen = 0; unsigned long i, max = 8 * gomp_cpuset_size; + int init = -1; for (i = 0; i < max && gomp_places_list_len < count; i++) if (CPU_ISSET_S (i, gomp_cpuset_size, copy)) { - sprintf (name + prefix_len, "%lu/topology/%s_siblings_list", - i, this_level == 3 ? "core" : "thread"); + if (level == 4) + { + if (init == -1) + { + init = gomp_affinity_find_last_cache_level (name, prefix_len, + i); + if (init == -1) + { + CPU_CLR_S (i, gomp_cpuset_size, copy); + continue; + } + sprintf (name + prefix_len, + "%lu/cache/index%u/shared_cpu_list", i, init); + } + } + else + sprintf (name + prefix_len, "%lu/topology/%s_siblings_list", + i, this_level == 3 ? "core" : "thread"); f = fopen (name, "r"); if (f == NULL) { @@ -302,7 +359,7 @@ bool gomp_affinity_init_level (int level, unsigned long count, bool quiet) { char name[sizeof ("/sys/devices/system/cpu/cpu/topology/" - "thread_siblings_list") + 3 * sizeof (unsigned long)]; + "thread_siblings_list") + 6 * sizeof (unsigned long)]; cpu_set_t *copy; if (gomp_cpusetp) @@ -320,7 +377,8 @@ gomp_affinity_init_level (int level, uns copy = gomp_alloca (gomp_cpuset_size); strcpy (name, "/sys/devices/system/cpu/cpu"); memcpy (copy, gomp_cpusetp, gomp_cpuset_size); - gomp_affinity_init_level_1 (level, 3, count, copy, name, quiet); + gomp_affinity_init_level_1 (level, level > 3 ? level : 3, count, copy, name, + quiet); if (gomp_places_list_len == 0) { if (!quiet) --- libgomp/testsuite/libgomp.c/places-1.c.jj 2021-10-15 11:19:37.763993802 +0200 +++ libgomp/testsuite/libgomp.c/places-1.c 2021-10-15 11:30:52.161264852 +0200 @@ -0,0 +1,10 @@ +/* { dg-set-target-env-var OMP_PLACES "threads" } */ + +#include + +int +main () +{ + omp_display_env (0); + return 0; +} --- libgomp/testsuite/libgomp.c/places-2.c.jj 2021-10-15 11:19:44.057903005 +0200 +++ libgomp/testsuite/libgomp.c/places-2.c 2021-10-15 11:31:36.367627131 +0200 @@ -0,0 +1,10 @@ +/* { dg-set-target-env-var OMP_PLACES "cores" } */ + +#include + +int +main () +{ + omp_display_env (0); + return 0; +} --- libgomp/testsuite/libgomp.c/places-3.c.jj 2021-10-15 11:20:11.802502758 +0200 +++ libgomp/testsuite/libgomp.c/places-3.c 2021-10-15 11:31:43.388525848 +0200 @@ -0,0 +1,10 @@ +/* { dg-set-target-env-var OMP_PLACES "sockets" } */ + +#include + +int +main () +{ + omp_display_env (0); + return 0; +} --- libgomp/testsuite/libgomp.c/places-4.c.jj 2021-10-15 11:20:27.669273860 +0200 +++ libgomp/testsuite/libgomp.c/places-4.c 2021-10-15 11:31:51.049415333 +0200 @@ -0,0 +1,10 @@ +/* { dg-set-target-env-var OMP_PLACES "ll_caches" } */ + +#include + +int +main () +{ + omp_display_env (0); + return 0; +}