From patchwork Thu Jan 20 13:59:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Stubbs X-Patchwork-Id: 50281 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DB7173857C7F for ; Thu, 20 Jan 2022 14:00:41 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from esa3.mentor.iphmx.com (esa3.mentor.iphmx.com [68.232.137.180]) by sourceware.org (Postfix) with ESMTPS id A08BB385781B for ; Thu, 20 Jan 2022 14:00:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A08BB385781B Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=codesourcery.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=mentor.com IronPort-SDR: Zgf7Y6KXq4tm5oUm0FRpBALfBDVuoYpHNLrfL3KIupe2tDPXrvqpkLp954PhjARj9V08qhXYG9 2B9+nCxPTR06MM1hNPzV23Wg98D0nX88POUayFnVX0a9vh7hBt+4YMWovsnRB1eq7edyD3ruMo 0o3y3m34J+jCQ7Tbq1IUjPUsh/89AbL0zPbctw517OmW3d2ZMBTpXJ4WfTIg7SNaIW6J8+Jkn8 1pLDoHE1V6oGVgnDFothUV7vQwCUczwANpVeFnIsyswQ1Wbwv7dbWHrMgmJJdnLExpkiSusOGq gkc= X-IronPort-AV: E=Sophos;i="5.88,302,1635235200"; d="scan'208";a="70868675" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa3.mentor.iphmx.com with ESMTP; 20 Jan 2022 06:00:06 -0800 IronPort-SDR: XJmIoe+RvuT54D7+WEK3gxfKn9CDImfJTzaUr0pDkGX6Ss47+XWoKeq+2boIXSNfLTjZkid0CB G3q3T1I7J7kcU7EzLqobWaBWgatse3ZxJR2DLgK2DImbLqii08WQOhSgpecpLr+IQP/N7c+8Q6 roHVyu3qwY4+euLMQpzIKfCfK0dSt5K2yT2DLEuJ6uLvulROPiWSno8IS3a35Hmi0xsEUaGfit EcqpPW+PPDnkK7E+7tqW8SluBLJ3l6d4XW1emJYGIg9OwiPKaH749Pi+RZT2MD518EgLRD6p+j iYQ= Message-ID: <1482e076-ecae-80ca-cab5-617fe1bcd666@codesourcery.com> Date: Thu, 20 Jan 2022 13:59:59 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Content-Language: en-GB From: Andrew Stubbs Subject: [PATCH] libgomp, openmp: Add ompx_pinned_mem_alloc To: "gcc-patches@gcc.gnu.org" X-Originating-IP: [137.202.0.90] X-ClientProxiedBy: svr-ies-mbx-09.mgc.mentorg.com (139.181.222.9) To svr-ies-mbx-01.mgc.mentorg.com (139.181.222.1) X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" This patch adds a new predefined allocator named ompx_pinned_mem_alloc as an extension to the OpenMP standard. It is intended as a convenient way to allocate pinned memory using the Linux support patch I posted recently. I anticipate it being used by compiler internals in future as part of a project to improve performance of existing code (such as benchmarks!) It is equivalent to a custom allocator with the pinned trait and a null fallback trait. The name uses the "ompx" extension namespace proposed for OpenMP 5.2, rather than a "gomp" prefix, say, so that it can also be used by other projects implementing OpenMP in other toolchains (this requirement comes from our client who is funding this work). OK for stage 1? Andrew libgomp, openmp: Add ompx_pinned_mem_alloc This creates a new predefined allocator as a shortcut for using pinned memory with OpenMP. The name uses the OpenMP extension space and is intended to be consistent with other OpenMP implementations currently in development. The allocator is equivalent to using a custom allocator with the pinned trait and the null fallback trait. libgomp/ChangeLog: * allocator.c (omp_max_predefined_alloc): Update. (omp_aligned_alloc): Support ompx_pinned_mem_alloc. (omp_free): Likewise. (omp_aligned_calloc): Likewise. (omp_realloc): Likewise. * omp.h.in (omp_allocator_handle_t): Add ompx_pinned_mem_alloc. * omp_lib.f90.in: Add ompx_pinned_mem_alloc. * testsuite/libgomp.c/alloc-pinned-5.c: New test. * testsuite/libgomp.c/alloc-pinned-6.c: New test. * testsuite/libgomp.fortran/alloc-pinned-1.f90: New test. diff --git a/libgomp/allocator.c b/libgomp/allocator.c index 5ab161b6314..b1f41ccc0d4 100644 --- a/libgomp/allocator.c +++ b/libgomp/allocator.c @@ -32,7 +32,7 @@ #include #include -#define omp_max_predefined_alloc omp_thread_mem_alloc +#define omp_max_predefined_alloc ompx_pinned_mem_alloc /* These macros may be overridden in config//allocator.c. */ #ifndef MEMSPACE_ALLOC @@ -64,6 +64,7 @@ static const omp_memspace_handle_t predefined_alloc_mapping[] = { omp_low_lat_mem_space, /* omp_cgroup_mem_alloc. */ omp_low_lat_mem_space, /* omp_pteam_mem_alloc. */ omp_low_lat_mem_space, /* omp_thread_mem_alloc. */ + omp_default_mem_space, /* ompx_pinned_mem_alloc. */ }; struct omp_allocator_data @@ -334,8 +335,11 @@ retry: = (allocator_data ? allocator_data->memspace : predefined_alloc_mapping[allocator]); - ptr = MEMSPACE_ALLOC (memspace, new_size, - allocator_data && allocator_data->pinned); + int pinned __attribute__((unused)) + = (allocator_data + ? allocator_data->pinned + : allocator == ompx_pinned_mem_alloc); + ptr = MEMSPACE_ALLOC (memspace, new_size, pinned); if (ptr == NULL) goto fail; } @@ -355,7 +359,8 @@ retry: fail: int fallback = (allocator_data ? allocator_data->fallback - : allocator == omp_default_mem_alloc + : (allocator == omp_default_mem_alloc + || allocator == ompx_pinned_mem_alloc) ? omp_atv_null_fb : omp_atv_default_mem_fb); switch (fallback) @@ -442,7 +447,10 @@ omp_free (void *ptr, omp_allocator_handle_t allocator) pinned = allocator_data->pinned; } else - memspace = predefined_alloc_mapping[data->allocator]; + { + memspace = predefined_alloc_mapping[data->allocator]; + pinned = (data->allocator == ompx_pinned_mem_alloc); + } MEMSPACE_FREE (memspace, data->ptr, data->size, pinned); } @@ -553,8 +561,11 @@ retry: = (allocator_data ? allocator_data->memspace : predefined_alloc_mapping[allocator]); - ptr = MEMSPACE_CALLOC (memspace, new_size, - allocator_data && allocator_data->pinned); + int pinned __attribute__((unused)) + = (allocator_data + ? allocator_data->pinned + : allocator == ompx_pinned_mem_alloc); + ptr = MEMSPACE_CALLOC (memspace, new_size, pinned); if (ptr == NULL) goto fail; } @@ -574,7 +585,8 @@ retry: fail: int fallback = (allocator_data ? allocator_data->fallback - : allocator == omp_default_mem_alloc + : (allocator == omp_default_mem_alloc + || allocator == ompx_pinned_mem_alloc) ? omp_atv_null_fb : omp_atv_default_mem_fb); switch (fallback) @@ -719,11 +731,15 @@ retry: gomp_mutex_unlock (&allocator_data->lock); #endif if (prev_size) - new_ptr = MEMSPACE_REALLOC (allocator_data->memspace, data->ptr, - data->size, new_size, - (free_allocator_data - && free_allocator_data->pinned), - allocator_data->pinned); + { + int was_pinned __attribute__((unused)) + = (free_allocator_data + ? free_allocator_data->pinned + : free_allocator == ompx_pinned_mem_alloc); + new_ptr = MEMSPACE_REALLOC (allocator_data->memspace, data->ptr, + data->size, new_size, was_pinned, + allocator_data->pinned); + } else new_ptr = MEMSPACE_ALLOC (allocator_data->memspace, new_size, allocator_data->pinned); @@ -758,10 +774,16 @@ retry: = (allocator_data ? allocator_data->memspace : predefined_alloc_mapping[allocator]); + int was_pinned __attribute__((unused)) + = (free_allocator_data + ? free_allocator_data->pinned + : free_allocator == ompx_pinned_mem_alloc); + int pinned __attribute__((unused)) + = (allocator_data + ? allocator_data->pinned + : allocator == ompx_pinned_mem_alloc); new_ptr = MEMSPACE_REALLOC (memspace, data->ptr, data->size, new_size, - (free_allocator_data - && free_allocator_data->pinned), - allocator_data && allocator_data->pinned); + was_pinned, pinned); if (new_ptr == NULL) goto fail; @@ -777,8 +799,11 @@ retry: = (allocator_data ? allocator_data->memspace : predefined_alloc_mapping[allocator]); - new_ptr = MEMSPACE_ALLOC (memspace, new_size, - allocator_data && allocator_data->pinned); + int pinned __attribute__((unused)) + = (allocator_data + ? allocator_data->pinned + : allocator == ompx_pinned_mem_alloc); + new_ptr = MEMSPACE_ALLOC (memspace, new_size, pinned); if (new_ptr == NULL) goto fail; } @@ -814,7 +839,8 @@ retry: fail: int fallback = (allocator_data ? allocator_data->fallback - : allocator == omp_default_mem_alloc + : (allocator == omp_default_mem_alloc + || allocator == ompx_pinned_mem_alloc) ? omp_atv_null_fb : omp_atv_default_mem_fb); switch (fallback) diff --git a/libgomp/omp.h.in b/libgomp/omp.h.in index 89c5d65bcd5..1d002d36aae 100644 --- a/libgomp/omp.h.in +++ b/libgomp/omp.h.in @@ -134,6 +134,7 @@ typedef enum omp_allocator_handle_t __GOMP_UINTPTR_T_ENUM omp_cgroup_mem_alloc = 6, omp_pteam_mem_alloc = 7, omp_thread_mem_alloc = 8, + ompx_pinned_mem_alloc = 9, __omp_allocator_handle_t_max__ = __UINTPTR_MAX__ } omp_allocator_handle_t; diff --git a/libgomp/omp_lib.f90.in b/libgomp/omp_lib.f90.in index daf40dc8e6d..a095dad8962 100644 --- a/libgomp/omp_lib.f90.in +++ b/libgomp/omp_lib.f90.in @@ -158,6 +158,8 @@ parameter :: omp_pteam_mem_alloc = 7 integer (kind=omp_allocator_handle_kind), & parameter :: omp_thread_mem_alloc = 8 + integer (kind=omp_allocator_handle_kind), & + parameter :: ompx_pinned_mem_alloc = 9 integer (omp_memspace_handle_kind), & parameter :: omp_default_mem_space = 0 integer (omp_memspace_handle_kind), & diff --git a/libgomp/testsuite/libgomp.c/alloc-pinned-5.c b/libgomp/testsuite/libgomp.c/alloc-pinned-5.c new file mode 100644 index 00000000000..8355ca83790 --- /dev/null +++ b/libgomp/testsuite/libgomp.c/alloc-pinned-5.c @@ -0,0 +1,76 @@ +/* { dg-do run } */ + +/* { dg-xfail-run-if "Pinning not implemented on this host" { ! *-*-linux-gnu } } */ + +/* Test that ompx_pinned_mem_alloc works. */ + +#ifdef __linux__ +#include +#include +#include +#include + +#include + +int +get_pinned_mem () +{ + int pid = getpid (); + char buf[100]; + sprintf (buf, "/proc/%d/status", pid); + + FILE *proc = fopen (buf, "r"); + if (!proc) + abort (); + while (fgets (buf, 100, proc)) + { + int val; + if (sscanf (buf, "VmLck: %d", &val)) + { + fclose (proc); + return val; + } + } + abort (); +} +#else +int +get_pinned_mem () +{ + return 0; +} +#endif + +#include + +/* Allocate more than a page each time, but stay within the ulimit. */ +#define SIZE 10*1024 + +int +main () +{ + // Sanity check + if (get_pinned_mem () != 0) + abort (); + + void *p = omp_alloc (SIZE, ompx_pinned_mem_alloc); + if (!p) + abort (); + + int amount = get_pinned_mem (); + if (amount == 0) + abort (); + + p = omp_realloc (p, SIZE*2, ompx_pinned_mem_alloc, ompx_pinned_mem_alloc); + + int amount2 = get_pinned_mem (); + if (amount2 <= amount) + abort (); + + p = omp_calloc (1, SIZE, ompx_pinned_mem_alloc); + + if (get_pinned_mem () <= amount2) + abort (); + + return 0; +} diff --git a/libgomp/testsuite/libgomp.c/alloc-pinned-6.c b/libgomp/testsuite/libgomp.c/alloc-pinned-6.c new file mode 100644 index 00000000000..80fd37ab875 --- /dev/null +++ b/libgomp/testsuite/libgomp.c/alloc-pinned-6.c @@ -0,0 +1,96 @@ +/* { dg-do run } */ + +/* Test that ompx_pinned_mem_alloc fails correctly. */ + +#ifdef __linux__ +#include +#include +#include +#include + +#include +#include + +int +get_pinned_mem () +{ + int pid = getpid (); + char buf[100]; + sprintf (buf, "/proc/%d/status", pid); + + FILE *proc = fopen (buf, "r"); + if (!proc) + abort (); + while (fgets (buf, 100, proc)) + { + int val; + if (sscanf (buf, "VmLck: %d", &val)) + { + fclose (proc); + return val; + } + } + abort (); +} + +void +set_pin_limit (int size) +{ + struct rlimit limit; + if (getrlimit (RLIMIT_MEMLOCK, &limit)) + abort (); + limit.rlim_cur = (limit.rlim_max < size ? limit.rlim_max : size); + if (setrlimit (RLIMIT_MEMLOCK, &limit)) + abort (); +} +#else +int +get_pinned_mem () +{ + return 0; +} + +void +set_pin_limit () +{ +} +#endif + +#include + +/* This should be large enough to cover multiple pages. */ +#define SIZE 10000*1024 + +int +main () +{ + /* Ensure that the limit is smaller than the allocation. */ + set_pin_limit (SIZE/2); + + // Sanity check + if (get_pinned_mem () != 0) + abort (); + + // Should fail + void *p = omp_alloc (SIZE, ompx_pinned_mem_alloc); + if (p) + abort (); + + // Should fail + p = omp_calloc (1, SIZE, ompx_pinned_mem_alloc); + if (p) + abort (); + + // Should fail to realloc + void *notpinned = omp_alloc (SIZE, omp_default_mem_alloc); + p = omp_realloc (notpinned, SIZE, ompx_pinned_mem_alloc, omp_default_mem_alloc); + if (!notpinned || p) + abort (); + + // No memory should have been pinned + int amount = get_pinned_mem (); + if (amount != 0) + abort (); + + return 0; +} diff --git a/libgomp/testsuite/libgomp.fortran/alloc-pinned-1.f90 b/libgomp/testsuite/libgomp.fortran/alloc-pinned-1.f90 new file mode 100644 index 00000000000..798dc3d5a12 --- /dev/null +++ b/libgomp/testsuite/libgomp.fortran/alloc-pinned-1.f90 @@ -0,0 +1,16 @@ +! Ensure that the ompx_pinned_mem_alloc predefined allocator is present and +! accepted. The majority of the functionality testing lives in the C tests. +! +! { dg-xfail-run-if "Pinning not implemented on this host" { ! *-*-linux-gnu } } + +program main + use omp_lib + use ISO_C_Binding + implicit none (external, type) + + type(c_ptr) :: p + + p = omp_alloc (10_c_size_t, ompx_pinned_mem_alloc); + if (.not. c_associated (p)) stop 1 + call omp_free (p, ompx_pinned_mem_alloc); +end program main