From patchwork Sun Nov 19 10:23:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Weimer X-Patchwork-Id: 24357 Received: (qmail 15454 invoked by alias); 19 Nov 2017 10:23:08 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 15441 invoked by uid 89); 19 Nov 2017 10:23:07 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KB_WAM_FROM_NAME_SINGLEWORD, SPF_HELO_PASS, T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 spammy=Shouldnt, Shouldn't X-HELO: mx1.redhat.com Subject: Re: [PATCH] manual: Document the MAP_HUGETLB, MADV_HUGEPAGE, MADV_NOHUGEPAGE flags To: Rical Jasan Cc: libc-alpha@sourceware.org References: <20171118152204.A2F3243426A3C@oldenburg.str.redhat.com> <6e16a231-6a3c-d8e4-a50c-0c3de2a1c0ef@pacific.net> From: Florian Weimer Message-ID: <27e02057-52f6-f6b9-daec-3812c7d9229a@redhat.com> Date: Sun, 19 Nov 2017 11:23:01 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <6e16a231-6a3c-d8e4-a50c-0c3de2a1c0ef@pacific.net> On 11/19/2017 03:29 AM, Rical Jasan wrote: >> -These functions are declared in @file{sys/mman.h}. >> +Note that @samp{sysconf (_SC_PAGESIZE)} only returns the default page >> +size of the system. On some systems, mappings can use larger page sizes >> +for certain files, and applications can request larger page sizes for >> +anonymous mappings as well (see the @code{MAP_HUGETLB} flag below). > > I think the first sentence can be dropped since you already clarified > it's the default page size above, and it would avoid having the function > call in the sentence (which was also just shown). Okay. >> +The following functions are declared in @file{sys/mman.h}. > > As a general rule, I end sentences containing "the following" with a > colon, but +1 for adding that bit here. Colon added. >> @deftypefun {void *} mmap (void *@var{address}, size_t @var{length}, int @var{protect}, int @var{flags}, int @var{filedes}, off_t @var{offset}) >> @standards{POSIX, sys/mman.h} >> @@ -1460,6 +1464,28 @@ On some systems using private anonymous mmaps is more efficient than using >> @code{malloc} for large blocks. This is not an issue with @theglibc{}, >> as the included @code{malloc} automatically uses @code{mmap} where appropriate. >> >> +@item MAP_HUGETLB > > @standards{Linux, sys/mman.h} > >> +This requests that the system uses an alternative page size which is >> +larger than the default page size for the mapping. For some workloads, >> +increasing the page size for large mappings improves performance because >> +the system needs to handle far fewer pages. For other workloads which >> +require frequent transfer of pages between storage or different nodes, >> +the increased page granularity may cause performance problems due to the > > Shouldn't this be "decreased page granularity"? Right, fixed. >> +increased page size and larger transfers. >> + >> +In order to create the mapping, the system needs physically contiguous >> +memory of the size of the increased page size. As a result, >> +@code{MAP_HUGETLB} mappings are affected by memory fragmentation, and >> +their creation can fail even if plenty of memory is available in the >> +system. >> + >> +Not all file systems support mappings with an increased page size. >> + >> +The @code{MAP_HUGETLB} flag is specific to Linux. >> + >> +@c There is a mechanism to select different hugepage sizes, see > > "sizes; see" Fixed. >> +@c include/uapi/asm-generic/hugetlb_encode.h in the kernel sources. >> + >> @c Linux has some other MAP_ options, which I have not discussed here. >> @c MAP_DENYWRITE, MAP_EXECUTABLE and MAP_GROWSDOWN don't seem applicable to >> @c user programs (and I don't understand the last two). MAP_LOCKED does >> @@ -1476,8 +1502,11 @@ Possible errors include: >> >> @item EINVAL >> >> -Either @var{address} was unusable, or inconsistent @var{flags} were >> -given. >> +Either @var{address} was unusable (because it is not a multiple of the >> +applicable page size), or inconsistent @var{flags} were given. >> + >> +If @code{MAP_HUGETLB} was specified: the file or system does not support > > "specified," Okay. >> +large page sizes. >> >> @item EACCES >> >> @@ -1678,6 +1707,20 @@ The region is no longer needed. The kernel may free these pages, >> causing any changes to the pages to be lost, as well as swapped >> out pages to be discarded. >> >> +@item MADV_HUGEPAGE > > @standards{Linux, sys/mman.h} > >> +Indicate that it is beneficial to increase the page size for this >> +mapping. This can improve performance larger mappings because the > > "performance for" Fixed. >> +system needs to handle far fewer pages. However, if parts of the >> +mapping are frequently transferred between storage or different nodes, >> +performance may suffer because individual transfers can become >> +substantially larger due to the increased page size. >> + >> +This flag is specific to Linux. >> + >> +@item MADV_NOHUGEPAGE >> +Undo the effect of a previous @code{MADV_NOHUGEPAGE} advice. This flag > > Undoes MADV_HUGEPAGE, right? Fixed as well. Thanks for your review, updated patch attached. Florian Subject: [PATCH] manual: Document the MAP_HUGETLB, MADV_HUGEPAGE, MADV_NOHUGEPAGE flags To: libc-alpha@sourceware.org 2017-11-19 Florian Weimer * manual/llio.texi (Memory-mapped I/O): Document MAP_HUGETLB, MADV_HUGEPAGE, MADV_NOHUGEPAGE. diff --git a/manual/llio.texi b/manual/llio.texi index ff88805369..85e1687c50 100644 --- a/manual/llio.texi +++ b/manual/llio.texi @@ -1377,15 +1377,18 @@ available. Memory mapping only works on entire pages of memory. Thus, addresses for mapping must be page-aligned, and length values will be rounded up. -To determine the size of a page the machine uses one should use +To determine the default size of a page the machine uses one should use: @vindex _SC_PAGESIZE @smallexample size_t page_size = (size_t) sysconf (_SC_PAGESIZE); @end smallexample -@noindent -These functions are declared in @file{sys/mman.h}. +On some systems, mappings can use larger page sizes +for certain files, and applications can request larger page sizes for +anonymous mappings as well (see the @code{MAP_HUGETLB} flag below). + +The following functions are declared in @file{sys/mman.h}: @deftypefun {void *} mmap (void *@var{address}, size_t @var{length}, int @var{protect}, int @var{flags}, int @var{filedes}, off_t @var{offset}) @standards{POSIX, sys/mman.h} @@ -1460,6 +1463,29 @@ On some systems using private anonymous mmaps is more efficient than using @code{malloc} for large blocks. This is not an issue with @theglibc{}, as the included @code{malloc} automatically uses @code{mmap} where appropriate. +@item MAP_HUGETLB +@standards{Linux, sys/mman.h} +This requests that the system uses an alternative page size which is +larger than the default page size for the mapping. For some workloads, +increasing the page size for large mappings improves performance because +the system needs to handle far fewer pages. For other workloads which +require frequent transfer of pages between storage or different nodes, +the decreased page granularity may cause performance problems due to the +increased page size and larger transfers. + +In order to create the mapping, the system needs physically contiguous +memory of the size of the increased page size. As a result, +@code{MAP_HUGETLB} mappings are affected by memory fragmentation, and +their creation can fail even if plenty of memory is available in the +system. + +Not all file systems support mappings with an increased page size. + +The @code{MAP_HUGETLB} flag is specific to Linux. + +@c There is a mechanism to select different hugepage sizes; see +@c include/uapi/asm-generic/hugetlb_encode.h in the kernel sources. + @c Linux has some other MAP_ options, which I have not discussed here. @c MAP_DENYWRITE, MAP_EXECUTABLE and MAP_GROWSDOWN don't seem applicable to @c user programs (and I don't understand the last two). MAP_LOCKED does @@ -1476,8 +1502,11 @@ Possible errors include: @item EINVAL -Either @var{address} was unusable, or inconsistent @var{flags} were -given. +Either @var{address} was unusable (because it is not a multiple of the +applicable page size), or inconsistent @var{flags} were given. + +If @code{MAP_HUGETLB} was specified, the file or system does not support +large page sizes. @item EACCES @@ -1678,6 +1707,21 @@ The region is no longer needed. The kernel may free these pages, causing any changes to the pages to be lost, as well as swapped out pages to be discarded. +@item MADV_HUGEPAGE +@standards{Linux, sys/mman.h} +Indicate that it is beneficial to increase the page size for this +mapping. This can improve performance for larger mappings because the +system needs to handle far fewer pages. However, if parts of the +mapping are frequently transferred between storage or different nodes, +performance may suffer because individual transfers can become +substantially larger due to the increased page size. + +This flag is specific to Linux. + +@item MADV_NOHUGEPAGE +Undo the effect of a previous @code{MADV_HUGEPAGE} advice. This flag +is specific to Linux. + @end vtable The POSIX names are slightly different, but with the same meanings: