[1/5] openmp: Add -foffload-memory

  From: Andrew Stubbs <ams@codesourcery.com>

Add a new option.  It will be used in follow-up patches.

gcc/ChangeLog:

	* common.opt: Add -foffload-memory and its enum values.
	* coretypes.h (enum offload_memory): New.
	* doc/invoke.texi: Document -foffload-memory.
---
 gcc/common.opt      | 16 ++++++++++++++++
 gcc/coretypes.h     |  7 +++++++
 gcc/doc/invoke.texi | 16 +++++++++++++++-
 3 files changed, 38 insertions(+), 1 deletion(-)
  

Hi!

On 2022-03-08T11:30:55+0000, Hafiz Abid Qadeer <abidh@codesourcery.com> wrote:
> From: Andrew Stubbs <ams@codesourcery.com>
>
> Add a new option.  It will be used in follow-up patches.

> --- a/gcc/doc/invoke.texi
> +++ b/gcc/doc/invoke.texi

> +@option{-foffload-memory=pinned} forces all host memory to be pinned (this
> +mode may require the user to increase the ulimit setting for locked memory).

So, this is currently implemented via 'mlockall', which, as discussed,
(a) has issues ('ulimit -l'), and (b) doesn't actually achieve what it
meant to achieve (because it doesn't register the page-locked memory with
the GPU driver).

So one idea was to re-purpose the unified shared memory
'gcc/omp-low.cc:pass_usm_transform' (compiler pass that "changes calls to
malloc/free/calloc/realloc and operator new to memory allocation
functions in libgomp with allocator=ompx_unified_shared_mem_alloc"),
<https://inbox.sourceware.org/gcc-patches/20220308113059.688551-5-abidh@codesourcery.com>.
(I have not yet looked into that in detail.)

Here's now a different idea.  As '-foffload-memory=pinned', per the name
of the option, concerns itself with memory used in offloading but not
host execution generally, why are we actually attempting to "[force] all
host memory to be pinned" -- why not just the memory that's being used
with offloading?  That is, if '-foffload-memory=pinned' is set, register
as page-locked with the GPU driver all memory that appears in OMP
offloading data regions, such as OpenMP 'target' 'map' clauses etc.  That
way, this is directed at the offloading data transfers, as itended, but
at the same time we don't "waste" page-locked memory for generic host
memory allocations.  What do you think -- you, who've spent a lot more
time on this topic than I have, so it's likely possible that I fail to
realize some "details"?


Grüße
 Thomas
-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955
  

On 13/02/2023 14:38, Thomas Schwinge wrote:
> Hi!
> 
> On 2022-03-08T11:30:55+0000, Hafiz Abid Qadeer <abidh@codesourcery.com> wrote:
>> From: Andrew Stubbs <ams@codesourcery.com>
>>
>> Add a new option.  It will be used in follow-up patches.
> 
>> --- a/gcc/doc/invoke.texi
>> +++ b/gcc/doc/invoke.texi
> 
>> +@option{-foffload-memory=pinned} forces all host memory to be pinned (this
>> +mode may require the user to increase the ulimit setting for locked memory).
> 
> So, this is currently implemented via 'mlockall', which, as discussed,
> (a) has issues ('ulimit -l'), and (b) doesn't actually achieve what it
> meant to achieve (because it doesn't register the page-locked memory with
> the GPU driver).
> 
> So one idea was to re-purpose the unified shared memory
> 'gcc/omp-low.cc:pass_usm_transform' (compiler pass that "changes calls to
> malloc/free/calloc/realloc and operator new to memory allocation
> functions in libgomp with allocator=ompx_unified_shared_mem_alloc"),
> <https://inbox.sourceware.org/gcc-patches/20220308113059.688551-5-abidh@codesourcery.com>> (I have not yet looked into that in detail.)
> 
> Here's now a different idea.  As '-foffload-memory=pinned', per the name
> of the option, concerns itself with memory used in offloading but not
> host execution generally, why are we actually attempting to "[force] all
> host memory to be pinned" -- why not just the memory that's being used
> with offloading?  That is, if '-foffload-memory=pinned' is set, register
> as page-locked with the GPU driver all memory that appears in OMP
> offloading data regions, such as OpenMP 'target' 'map' clauses etc.  That
> way, this is directed at the offloading data transfers, as itended, but
> at the same time we don't "waste" page-locked memory for generic host
> memory allocations.  What do you think -- you, who've spent a lot more
> time on this topic than I have, so it's likely possible that I fail to
> realize some "details"?

The main reason it is the way it is is because in general it's not 
possible to know what memory is going to be offloaded at the time it is 
allocated (and stack/static memory is never allocated that way).

If there's a way to pin it after the fact then maybe that's not a 
terrible idea? The downside is that the memory might already have been 
paged out at that point, and we'd have to track what we'd previously 
pinned, or else re-pin it every time we launch a kernel. We'd also have 
no way to unpin previously pinned memory (not that that's relevant to 
the "lock all" case).

My original plan was to use omp_alloc for both the standard OpenMP 
support and the -foffload-memory option (to get the benefit of pinning 
without modifying any source), but then I decided that the mlockall 
option was much less invasive. This is still the best way to implement 
target-independent pinning, when there's no driver registration option.

Andrew