manual: memory: document support custom allocators

  While glibc has long officially supported replacing things like malloc,
it hasn't really been documented.  The calloc section happens to have a
small aside, but doesn't provide enough info.

Create a new section that explicitly lays out what is supported.
---
 manual/memory.texi | 47 +++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 4 deletions(-)
  

Mike Frysinger <vapier@gentoo.org> writes:

> +If you wish to replace the @code{malloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.

@ref is only useful at the end of a sentence.

Andreas.
  

On 13 Feb 2016 10:09, Andreas Schwab wrote:
> Mike Frysinger <vapier@gentoo.org> writes:
> > +If you wish to replace the @code{malloc} function with your own
> > +implementation, be sure to read the @ref{Custom Allocators} section.
> 
> @ref is only useful at the end of a sentence.

seems to render fine in html & info, and there are other cases in the
manual where @ref isn't at the end of a sentence.
-mike
  

On 02/13/2016 03:46 AM, Mike Frysinger wrote:

> +@node Function Replacements
> +@subsection API for custom allocators
> +
> +Your replacement must implement all of these functions.  If you omit any of
> +them, correct behavior cannot be guaranteed.  This is because some functions
> +might not internally call the others.  E.g. @code{calloc} might not call
> +@code{malloc}, so passing the returned pointer to @code{free} can confuse the
> +allocator.
> +
> +@itemize @bullet
> +@item @code{malloc} @xref{Basic Allocation}
> +@item @code{free} @xref{Freeing after Malloc}
> +@item @code{realloc} @xref{Changing Block Size}
> +@item @code{calloc} @xref{Allocating Cleared Space}
> +@item @code{memalign} @xref{Aligned Memory Blocks}
> +@end itemize
>  
> +All other memory functions that the @glibcadj{} provides will utilize the
> +ones listed above.  You may replace them, but the @glibcadj{} will not call
> +them directly, only through the aforementioned list.

I think this should note that other system libraries may call the other
functions, and that for full compatibility. an interposed malloc should
override these functions as well.

Florian
  

Mike Frysinger <vapier@gentoo.org> writes:

> On 13 Feb 2016 10:09, Andreas Schwab wrote:
>> Mike Frysinger <vapier@gentoo.org> writes:
>> > +If you wish to replace the @code{malloc} function with your own
>> > +implementation, be sure to read the @ref{Custom Allocators} section.
>> 
>> @ref is only useful at the end of a sentence.
>
> seems to render fine in html & info,

Try pdf or dvi.

Andreas.
  

On 13 Feb 2016 11:38, Florian Weimer wrote:
> On 02/13/2016 03:46 AM, Mike Frysinger wrote:
> > +@node Function Replacements
> > +@subsection API for custom allocators
> > +
> > +Your replacement must implement all of these functions.  If you omit any of
> > +them, correct behavior cannot be guaranteed.  This is because some functions
> > +might not internally call the others.  E.g. @code{calloc} might not call
> > +@code{malloc}, so passing the returned pointer to @code{free} can confuse the
> > +allocator.
> > +
> > +@itemize @bullet
> > +@item @code{malloc} @xref{Basic Allocation}
> > +@item @code{free} @xref{Freeing after Malloc}
> > +@item @code{realloc} @xref{Changing Block Size}
> > +@item @code{calloc} @xref{Allocating Cleared Space}
> > +@item @code{memalign} @xref{Aligned Memory Blocks}
> > +@end itemize
> >  
> > +All other memory functions that the @glibcadj{} provides will utilize the
> > +ones listed above.  You may replace them, but the @glibcadj{} will not call
> > +them directly, only through the aforementioned list.
> 
> I think this should note that other system libraries may call the other
> functions, and that for full compatibility. an interposed malloc should
> override these functions as well.

i don't think that's what we want.  even if another library (be it part of
glibc or otherwise) calls a function not overridden, those will eventually
call into malloc().  the funcs i listed where the only ones that show up
in a `readelf -r` listing of libc.so.
-mike
  

On 02/13/2016 12:07 PM, Mike Frysinger wrote:
> On 13 Feb 2016 11:38, Florian Weimer wrote:
>> On 02/13/2016 03:46 AM, Mike Frysinger wrote:
>>> +@node Function Replacements +@subsection API for custom
>>> allocators + +Your replacement must implement all of these
>>> functions.  If you omit any of +them, correct behavior cannot
>>> be guaranteed.  This is because some functions +might not
>>> internally call the others.  E.g. @code{calloc} might not call 
>>> +@code{malloc}, so passing the returned pointer to @code{free}
>>> can confuse the +allocator. + +@itemize @bullet +@item
>>> @code{malloc} @xref{Basic Allocation} +@item @code{free}
>>> @xref{Freeing after Malloc} +@item @code{realloc}
>>> @xref{Changing Block Size} +@item @code{calloc}
>>> @xref{Allocating Cleared Space} +@item @code{memalign}
>>> @xref{Aligned Memory Blocks} +@end itemize
>>> 
>>> +All other memory functions that the @glibcadj{} provides will
>>> utilize the +ones listed above.  You may replace them, but the
>>> @glibcadj{} will not call +them directly, only through the
>>> aforementioned list.
>> 
>> I think this should note that other system libraries may call the
>> other functions, and that for full compatibility. an interposed
>> malloc should override these functions as well.
> 
> i don't think that's what we want.  even if another library (be it
> part of glibc or otherwise) calls a function not overridden, those
> will eventually call into malloc().  the funcs i listed where the
> only ones that show up in a `readelf -r` listing of libc.so.

To be clear, I am talking about malloc_usable_size, valloc, pvalloc,
aligned_alloc, and posix_memalign.  Some system libraries do call some
of these functions, and if your interposed malloc library does not
override them as well, crashes are the result.

Florian
  

On 02/12/2016 06:46 PM, Mike Frysinger wrote:
> While glibc has long officially supported replacing things like malloc,
> it hasn't really been documented.  The calloc section happens to have a
> small aside, but doesn't provide enough info.
> 
> Create a new section that explicitly lays out what is supported.
> ---
>  manual/memory.texi | 47 +++++++++++++++++++++++++++++++++++++++++++----
>  1 file changed, 43 insertions(+), 4 deletions(-)
> 
> diff --git a/manual/memory.texi b/manual/memory.texi
> index 700555e..e1426f1 100644
> --- a/manual/memory.texi
> +++ b/manual/memory.texi
> @@ -18,6 +18,7 @@ and allocation of real memory.
>  * Memory Allocation::           Allocating storage for your program data
>  * Resizing the Data Segment::   @code{brk}, @code{sbrk}
>  * Locking Pages::               Preventing page faults
> +* Custom Allocators::           Using different memory allocators at runtime
>  @end menu
>  
>  Memory mapped I/O is not discussed in this chapter.  @xref{Memory-mapped I/O}.
> @@ -577,6 +578,9 @@ ptr = (char *) malloc (length + 1);
>  @noindent
>  @xref{Representation of Strings}, for more information about this.
>  
> +If you wish to replace the @code{malloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.

+implementation, @pxref{Custom Allocators}.

That won't render correctly across all documentation types.

> +
>  @node Malloc Examples
>  @subsubsection Examples of @code{malloc}
>  
> @@ -707,6 +711,9 @@ There is no point in freeing blocks at the end of a program, because all
>  of the program's space is given back to the system when the process
>  terminates.
>  
> +If you wish to replace the @code{free} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.

+implementation, @pxref{Custom Allocators}.

> +
>  @node Changing Block Size
>  @subsubsection Changing the Size of a Block
>  @cindex changing the size of a block (@code{malloc})
> @@ -815,6 +822,9 @@ necessitates copying it, so it can fail if no other space is available.
>  If the new size you specify is the same as the old size, @code{realloc}
>  is guaranteed to change nothing and return the same address that you gave.
>  
> +If you wish to replace the @code{realloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.

+implementation, @pxref{Custom Allocators}.

> +
>  @node Allocating Cleared Space
>  @subsubsection Allocating Cleared Space
>  
> @@ -862,10 +872,8 @@ calloc (size_t count, size_t eltsize)
>  @}
>  @end smallexample
>  
> -But in general, it is not guaranteed that @code{calloc} calls
> -@code{malloc} internally.  Therefore, if an application provides its own
> -@code{malloc}/@code{realloc}/@code{free} outside the C library, it
> -should always define @code{calloc}, too.
> +If you wish to replace the @code{calloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.

+implementation, @pxref{Custom Allocators}.

>  
>  @node Efficiency and Malloc
>  @subsubsection Efficiency Considerations for @code{malloc}
> @@ -3280,8 +3288,39 @@ calls can fail, so there are no specific @code{errno} values.
>  
>  @end deftypefun
>  
> +@node Custom Allocators
> +@section Custom Allocators
>  
> +The @glibcadj{} memory allocator is designed to allow implementations to
> +provide their own replacement.  I.e. You want to write your own @code{malloc}

+provide their own replacement---you can write your own @code{malloc}

I don't think Latin abbreviations should start a sentence.  "Want" also
colors the subordinate clause in an odd way; I think "can" is more in
keeping with the intent.

> +function or use a library like tcmalloc or jemalloc, and all @glibcadj{} code
> +will call into this for memory requests.  However, care must be taken to make
> +this work correctly.
> +
> +@menu
> +* Function Replacements::       List of functions that must be implemented
> +@end menu
> +
> +@node Function Replacements
> +@subsection API for custom allocators
> +
> +Your replacement must implement all of these functions.  If you omit any of
> +them, correct behavior cannot be guaranteed.  This is because some functions
> +might not internally call the others.  E.g. @code{calloc} might not call

+might not internally call the others.  For example, @code{calloc} might
not call

Alternate strategy for avoiding the abbreviation.

> +@code{malloc}, so passing the returned pointer to @code{free} can confuse the
> +allocator.
> +
> +@itemize @bullet
> +@item @code{malloc} @xref{Basic Allocation}
> +@item @code{free} @xref{Freeing after Malloc}
> +@item @code{realloc} @xref{Changing Block Size}
> +@item @code{calloc} @xref{Allocating Cleared Space}
> +@item @code{memalign} @xref{Aligned Memory Blocks}
> +@end itemize
>  
> +All other memory functions that the @glibcadj{} provides will utilize the

+All other memory functions @theglibc{} provides will utilize the

Not an adjective here.  "That" is unnecessary.

> +ones listed above.  You may replace them, but the @glibcadj{} will not call

+ones listed above.  You may replace them, but @theglibc{} will not call

Same.

> +them directly, only through the aforementioned list.
>  
>  @ignore
>  @c This was never actually implemented.  -zw
> 

My 2￠.

Rical