[4/4] User manual documentation for tunables

Message ID 1483070938-2308-5-git-send-email-siddhesh@sourceware.org
State New, archived
Headers

Commit Message

Siddhesh Poyarekar Dec. 30, 2016, 4:08 a.m. UTC
  Create a new node for tunables documentation and add notes for the
malloc tunables.

	* manual/tunables.texi: New chapter.
	* manual/Makefile (chapters): Add it.
	* manual/probes.texi (@node): Point to the Tunables chapter.
---
 manual/Makefile      |   3 +-
 manual/probes.texi   |   2 +-
 manual/tunables.texi | 185 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 188 insertions(+), 2 deletions(-)
 create mode 100644 manual/tunables.texi
  

Comments

Florian Weimer Dec. 30, 2016, 9:50 a.m. UTC | #1
On 12/30/2016 05:08 AM, Siddhesh Poyarekar wrote:
> Create a new node for tunables documentation and add notes for the
> malloc tunables.
>
> 	* manual/tunables.texi: New chapter.
> 	* manual/Makefile (chapters): Add it.
> 	* manual/probes.texi (@node): Point to the Tunables chapter.
> ---
>  manual/Makefile      |   3 +-
>  manual/probes.texi   |   2 +-
>  manual/tunables.texi | 185 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 188 insertions(+), 2 deletions(-)
>  create mode 100644 manual/tunables.texi
>
> diff --git a/manual/Makefile b/manual/Makefile
> index f2f694f..ecc2bf6 100644
> --- a/manual/Makefile
> +++ b/manual/Makefile
> @@ -38,7 +38,8 @@ chapters = $(addsuffix .texi, \
>  		       message search pattern io stdio llio filesys	\
>  		       pipe socket terminal syslog math arith time	\
>  		       resource setjmp signal startup process ipc job	\
> -		       nss users sysinfo conf crypt debug threads probes)
> +		       nss users sysinfo conf crypt debug threads	\
> +		       probes tunables)
>  add-chapters = $(wildcard $(foreach d, $(add-ons), ../$d/$d.texi))
>  appendices = lang.texi header.texi install.texi maint.texi platform.texi \
>  	     contrib.texi
> diff --git a/manual/probes.texi b/manual/probes.texi
> index 237a918..eb91c62 100644
> --- a/manual/probes.texi
> +++ b/manual/probes.texi
> @@ -1,5 +1,5 @@
>  @node Internal Probes
> -@c @node Internal Probes, , POSIX Threads, Top
> +@c @node Internal Probes, Tunables, POSIX Threads, Top
>  @c %MENU% Probes to monitor libc internal behavior
>  @chapter Internal probes
>
> diff --git a/manual/tunables.texi b/manual/tunables.texi
> new file mode 100644
> index 0000000..1f1f9fc
> --- /dev/null
> +++ b/manual/tunables.texi
> @@ -0,0 +1,185 @@
> +@node Tunables
> +@c @node Tunables, , Internal Probes, Top
> +@c %MENU% Tunable switches to alter libc internal behavior
> +@chapter Tunables
> +
> +Tunables is a feature in @theglibc{} that allows application authors and

“Tunables are”?  Both variants are probably fine, but you already use 
the plural variant below.

> +distribution maintainers to alter the runtime library behaviour to match

Please use “behavior” consistently.

> +their workload. These are implemented as a set of switches that may be
> +modified in different ways. The current default method to do this is via
> +the @env{GLIBC_TUNABLES} environment variable by setting it to a string
> +of colon-separated @var{name}=@var{value} pairs.  For example, the following
> +environment variable export enables malloc checking and sets the malloc
> +trim threshold to 128 bytes.

Please end the sentence with a colon:

> +@code{export GLIBC_TUNABLES=glibc.malloc.trim_threshold=128:glibc.malloc.check=3}

This should use @example and perhaps POSIX shell syntax:

@example
GLIBC_TUNABLES=glibc.malloc.trim_threshold=128:glibc.malloc.check=3
export GLIBC_TUNABLES
@end example

amespace, tunable namespace and tunable name, separated by periods.
> +
> +@node Memory Allocation Tunables
> +@section Memory Allocation Tunables
> +
> +@deftp Namespace glibc.malloc

I think this should use:

@deftp {Tunable namespace} glibc.malloc

“Namespace” is too generic.

> +@deftp Tunable glibc.malloc.check
> +This tunable supersedes the @code{MALLOC_CHECK_} environment variable and is
> +identical in features.
> +
> +Setting this tunable enables a special (less efficient) memory allocator for
> +the malloc family of functions that is designed to be tolerant against simple
> +errors such as double calls of free with the same argument, or overruns of a
> +single byte (off-by-one bugs).

I don't think this is true for the value 0.  0 uses the standard 
allocator and continues execution after that malloc detected errors.

> +@itemize @bullet
> +@item @code{0} Disable all error reporting.  The alternate allocator is
> +selected and heap corruption detection is in place, but any such errors
> +detected are ignored.  This is currently a supported use, but is not
> +recommended.

See above.

Thanks,
Florian
  
Siddhesh Poyarekar Dec. 31, 2016, 12:40 p.m. UTC | #2
On Friday 30 December 2016 03:20 PM, Florian Weimer wrote:
> I don't think this is true for the value 0.  0 uses the standard
> allocator and continues execution after that malloc detected errors.

Ahh right, I'll fix that.

Siddhesh
  

Patch

diff --git a/manual/Makefile b/manual/Makefile
index f2f694f..ecc2bf6 100644
--- a/manual/Makefile
+++ b/manual/Makefile
@@ -38,7 +38,8 @@  chapters = $(addsuffix .texi, \
 		       message search pattern io stdio llio filesys	\
 		       pipe socket terminal syslog math arith time	\
 		       resource setjmp signal startup process ipc job	\
-		       nss users sysinfo conf crypt debug threads probes)
+		       nss users sysinfo conf crypt debug threads	\
+		       probes tunables)
 add-chapters = $(wildcard $(foreach d, $(add-ons), ../$d/$d.texi))
 appendices = lang.texi header.texi install.texi maint.texi platform.texi \
 	     contrib.texi
diff --git a/manual/probes.texi b/manual/probes.texi
index 237a918..eb91c62 100644
--- a/manual/probes.texi
+++ b/manual/probes.texi
@@ -1,5 +1,5 @@ 
 @node Internal Probes
-@c @node Internal Probes, , POSIX Threads, Top
+@c @node Internal Probes, Tunables, POSIX Threads, Top
 @c %MENU% Probes to monitor libc internal behavior
 @chapter Internal probes
 
diff --git a/manual/tunables.texi b/manual/tunables.texi
new file mode 100644
index 0000000..1f1f9fc
--- /dev/null
+++ b/manual/tunables.texi
@@ -0,0 +1,185 @@ 
+@node Tunables
+@c @node Tunables, , Internal Probes, Top
+@c %MENU% Tunable switches to alter libc internal behavior
+@chapter Tunables
+
+Tunables is a feature in @theglibc{} that allows application authors and
+distribution maintainers to alter the runtime library behaviour to match
+their workload. These are implemented as a set of switches that may be
+modified in different ways. The current default method to do this is via
+the @env{GLIBC_TUNABLES} environment variable by setting it to a string
+of colon-separated @var{name}=@var{value} pairs.  For example, the following
+environment variable export enables malloc checking and sets the malloc
+trim threshold to 128 bytes.
+
+@code{export GLIBC_TUNABLES=glibc.malloc.trim_threshold=128:glibc.malloc.check=3}
+
+Tunables are not part of the @glibcadj{} stable ABI, and they are
+subject to change or removal across releases.  Additionally, the method to
+modify tunable values may change between releases and across distributions.
+It is possible to implement multiple `frontends' for the tunables allowing
+distributions to choose their preferred method at build time.
+
+Finally, the set of tunables available may vary between distributions as
+the tunables feature allows distributions to add their own tunables under
+their own namespace.
+
+@menu
+* Tunable names::  The structure of a tunable name
+* Memory Allocation Tunables::  Tunables in the memory allocation subsystem
+@end menu
+
+@node Tunable names
+@section Tunable names
+
+A tunable name is split into three components, a top namespace, a tunable
+namespace and the tunable name. The top namespace for tunables
+implemented in @theglibc{} project is @code{glibc}. Distributions that
+choose to add custom tunables in their maintained versions of @theglibc{}
+may choose to do so under their own top namespace.
+
+The tunable namespace is a logical grouping of tunables in a single
+module. This currently holds no special significance, although that may
+change in future.
+
+The tunable name is the actual name of the tunable. It is possible that
+different tunable namespaces may have tunables within them that have the
+same name, likewise for top namespaces. Hence, we only support
+identification of tunables by their full name, i.e. with the top
+namespace, tunable namespace and tunable name, separated by periods.
+
+@node Memory Allocation Tunables
+@section Memory Allocation Tunables
+
+@deftp Namespace glibc.malloc
+Memory allocation behaviour can be modified by setting any of the
+following tunables in the @code{malloc} namespace.
+@end deftp
+
+@deftp Tunable glibc.malloc.check
+This tunable supersedes the @code{MALLOC_CHECK_} environment variable and is
+identical in features.
+
+Setting this tunable enables a special (less efficient) memory allocator for
+the malloc family of functions that is designed to be tolerant against simple
+errors such as double calls of free with the same argument, or overruns of a
+single byte (off-by-one bugs). Not all such errors can be protected against,
+however, and memory leaks can result.  The following list describes the values
+that this tunable can take and the effect they have on malloc functionality.
+
+@itemize @bullet
+@item @code{0} Disable all error reporting.  The alternate allocator is
+selected and heap corruption detection is in place, but any such errors
+detected are ignored.  This is currently a supported use, but is not
+recommended.
+@item @code{1} Report errors.  The alternate allocator is selected and heap
+corruption, if detected, is reported as diagnostic messages to @code{stderr}
+and the program continues execution.
+@item @code{2} Abort on errors.  The alternate allocator is selected and if
+heap corruption is detected, the program is ended immediately by calling
+@code{abort}.
+@item @code{3} Fully enabled.  The alternate allocator is selected and is fully
+functional.  That is, if heap corruption is detected, a verbose diagnostic
+message is printed to @code{stderr} and the program is ended by calling
+@code{abort}.
+@end itemize
+
+Like @env{MALLOC_CHECK_}, @code{glibc.malloc.check} has a problem in that it
+diverges from normal program behavior by writing to @code{stderr}, which could
+by exploited in SUID and SGID binaries.  Therefore, @code{glibc.malloc.check}
+is disabled by default for SUID and SGID binaries.  This can be enabled again
+by the system administrator by adding a file @file{/etc/suid-debug}; the
+content of the file could be anything or even empty.
+@end deftp
+
+@deftp Tunable glibc.malloc.top_pad
+This tunable supersedes the @code{MALLOC_TOP_PAD_} environment variable and is
+identical in features.
+
+This tunable determines the amount of extra memory to obtain from the system
+when any of the arenas need to be extended.  It also specifies the number of
+bytes to retain when shrinking any of the arenas.  This provides the necessary
+hysteresis in heap size such that excessive amounts of system calls can be
+avoided.
+
+The default value of this tunable is @samp{0}.
+@end deftp
+
+@deftp Tunable glibc.malloc.perturb
+This tunable supersedes the @code{MALLOC_PERTURB_} environment variable and is
+identical in features.
+
+If set to a non-zero value, memory blocks are initialized with values depending
+on some low order bits of this tunable when they are allocated (except when
+allocated by calloc) and freed.  This can be used to debug the use of
+uninitialized or freed heap memory. Note that this option does not guarantee
+that the freed block will have any specific values. It only guarantees that the
+content the block had before it was freed will be overwritten.
+
+The default value of this tunable is @samp{0}.
+@end deftp
+
+@deftp Tunable glibc.malloc.mmap_threshold
+This tunable supersedes the @code{MALLOC_MMAP_THRESHOLD_} environment variable
+and is identical in features.
+
+When this tunable is set, all chunks larger than this value are allocated
+outside the normal heap, using the @code{mmap} system call. This way it is
+guaranteed that the memory for these chunks can be returned to the system on
+@code{free}. Note that requests smaller than this threshold might still be
+allocated via @code{mmap}.
+
+If this tunable is not set, the default value is set as 128 KB and the
+threshold is adjusted dynamically to suit the allocation patterns of the
+program.  If the tunable is set, the dynamic adjustment is disabled and the
+value is set as static.
+@end deftp
+
+@deftp Tunable glibc.malloc.trim_threshold
+This tunable supersedes the @code{MALLOC_TRIM_THRESHOLD_} environment variable
+and is identical in features.
+
+The value of this tunable is the minimum size (in bytes) of the top-most,
+releasable chunk in an arena that will trigger a system call in order to return
+memory to the system from that arena.
+
+If this tunable is not set, the default value is set as 128 KB and the
+threshold is adjusted dynamically to suit the allocation patterns of the
+program.  If the tunable is set, the dynamic adjustment is disabled and the
+value is set as static.
+@end deftp
+
+@deftp Tunable glibc.malloc.mmap_max
+This tunable supersedes the @code{MALLOC_MMAP_MAX_} environment variable and is
+identical in features.
+
+The value of this tunable is maximum number of chunks to allocate with
+@code{mmap}.  Setting this to zero disables all use of @code{mmap}.
+
+The default value of this tunable is @samp{65536}.
+@end deftp
+
+@deftp Tunable glibc.malloc.arena_test
+This tunable supersedes the @code{MALLOC_ARENA_TEST} environment variable and is
+identical in features.
+
+The @code{glibc.malloc.arena_test} tunable specifies the number of arenas that
+can be created before the test on the limit to the number of arenas is
+conducted.  The value is ignored if @code{glibc.malloc.arena_max} is set.
+
+The default value of this tunable is 2 for 32-bit systems and 8 for 64-bit
+systems.
+@end deftp
+
+@deftp Tunable glibc.malloc.arena_max
+This tunable supersedes the @code{MALLOC_ARENA_MAX} environment variable and is
+identical in features.
+
+This tunable sets the number of arenas to use in a process regardless of the
+number of cores in the system.
+
+The default value of this tunable is @code{0}, meaning that the limit on the
+number of arenas is determined by the number of CPU cores online.  For 32-bit
+systems the limit is twice the number of cores online and on 64-bit systems, it
+is 8 times the number of cores online.
+@end deftp