diff mbox

manual: memory: document support custom allocators

Message ID 1455331567-12658-1-git-send-email-vapier@gentoo.org
State New
Delegated to: Mike Frysinger
Headers show

Commit Message

Mike Frysinger Feb. 13, 2016, 2:46 a.m. UTC
While glibc has long officially supported replacing things like malloc,
it hasn't really been documented.  The calloc section happens to have a
small aside, but doesn't provide enough info.

Create a new section that explicitly lays out what is supported.
---
 manual/memory.texi | 47 +++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 4 deletions(-)

Comments

Andreas Schwab Feb. 13, 2016, 9:09 a.m. UTC | #1
Mike Frysinger <vapier@gentoo.org> writes:

> +If you wish to replace the @code{malloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.

@ref is only useful at the end of a sentence.

Andreas.
Mike Frysinger Feb. 13, 2016, 10:23 a.m. UTC | #2
On 13 Feb 2016 10:09, Andreas Schwab wrote:
> Mike Frysinger <vapier@gentoo.org> writes:
> > +If you wish to replace the @code{malloc} function with your own
> > +implementation, be sure to read the @ref{Custom Allocators} section.
> 
> @ref is only useful at the end of a sentence.

seems to render fine in html & info, and there are other cases in the
manual where @ref isn't at the end of a sentence.
-mike
Florian Weimer Feb. 13, 2016, 10:38 a.m. UTC | #3
On 02/13/2016 03:46 AM, Mike Frysinger wrote:

> +@node Function Replacements
> +@subsection API for custom allocators
> +
> +Your replacement must implement all of these functions.  If you omit any of
> +them, correct behavior cannot be guaranteed.  This is because some functions
> +might not internally call the others.  E.g. @code{calloc} might not call
> +@code{malloc}, so passing the returned pointer to @code{free} can confuse the
> +allocator.
> +
> +@itemize @bullet
> +@item @code{malloc} @xref{Basic Allocation}
> +@item @code{free} @xref{Freeing after Malloc}
> +@item @code{realloc} @xref{Changing Block Size}
> +@item @code{calloc} @xref{Allocating Cleared Space}
> +@item @code{memalign} @xref{Aligned Memory Blocks}
> +@end itemize
>  
> +All other memory functions that the @glibcadj{} provides will utilize the
> +ones listed above.  You may replace them, but the @glibcadj{} will not call
> +them directly, only through the aforementioned list.

I think this should note that other system libraries may call the other
functions, and that for full compatibility. an interposed malloc should
override these functions as well.

Florian
Andreas Schwab Feb. 13, 2016, 10:45 a.m. UTC | #4
Mike Frysinger <vapier@gentoo.org> writes:

> On 13 Feb 2016 10:09, Andreas Schwab wrote:
>> Mike Frysinger <vapier@gentoo.org> writes:
>> > +If you wish to replace the @code{malloc} function with your own
>> > +implementation, be sure to read the @ref{Custom Allocators} section.
>> 
>> @ref is only useful at the end of a sentence.
>
> seems to render fine in html & info,

Try pdf or dvi.

Andreas.
Mike Frysinger Feb. 13, 2016, 11:07 a.m. UTC | #5
On 13 Feb 2016 11:38, Florian Weimer wrote:
> On 02/13/2016 03:46 AM, Mike Frysinger wrote:
> > +@node Function Replacements
> > +@subsection API for custom allocators
> > +
> > +Your replacement must implement all of these functions.  If you omit any of
> > +them, correct behavior cannot be guaranteed.  This is because some functions
> > +might not internally call the others.  E.g. @code{calloc} might not call
> > +@code{malloc}, so passing the returned pointer to @code{free} can confuse the
> > +allocator.
> > +
> > +@itemize @bullet
> > +@item @code{malloc} @xref{Basic Allocation}
> > +@item @code{free} @xref{Freeing after Malloc}
> > +@item @code{realloc} @xref{Changing Block Size}
> > +@item @code{calloc} @xref{Allocating Cleared Space}
> > +@item @code{memalign} @xref{Aligned Memory Blocks}
> > +@end itemize
> >  
> > +All other memory functions that the @glibcadj{} provides will utilize the
> > +ones listed above.  You may replace them, but the @glibcadj{} will not call
> > +them directly, only through the aforementioned list.
> 
> I think this should note that other system libraries may call the other
> functions, and that for full compatibility. an interposed malloc should
> override these functions as well.

i don't think that's what we want.  even if another library (be it part of
glibc or otherwise) calls a function not overridden, those will eventually
call into malloc().  the funcs i listed where the only ones that show up
in a `readelf -r` listing of libc.so.
-mike
Florian Weimer Feb. 13, 2016, 11:12 a.m. UTC | #6
On 02/13/2016 12:07 PM, Mike Frysinger wrote:
> On 13 Feb 2016 11:38, Florian Weimer wrote:
>> On 02/13/2016 03:46 AM, Mike Frysinger wrote:
>>> +@node Function Replacements +@subsection API for custom
>>> allocators + +Your replacement must implement all of these
>>> functions.  If you omit any of +them, correct behavior cannot
>>> be guaranteed.  This is because some functions +might not
>>> internally call the others.  E.g. @code{calloc} might not call 
>>> +@code{malloc}, so passing the returned pointer to @code{free}
>>> can confuse the +allocator. + +@itemize @bullet +@item
>>> @code{malloc} @xref{Basic Allocation} +@item @code{free}
>>> @xref{Freeing after Malloc} +@item @code{realloc}
>>> @xref{Changing Block Size} +@item @code{calloc}
>>> @xref{Allocating Cleared Space} +@item @code{memalign}
>>> @xref{Aligned Memory Blocks} +@end itemize
>>> 
>>> +All other memory functions that the @glibcadj{} provides will
>>> utilize the +ones listed above.  You may replace them, but the
>>> @glibcadj{} will not call +them directly, only through the
>>> aforementioned list.
>> 
>> I think this should note that other system libraries may call the
>> other functions, and that for full compatibility. an interposed
>> malloc should override these functions as well.
> 
> i don't think that's what we want.  even if another library (be it
> part of glibc or otherwise) calls a function not overridden, those
> will eventually call into malloc().  the funcs i listed where the
> only ones that show up in a `readelf -r` listing of libc.so.

To be clear, I am talking about malloc_usable_size, valloc, pvalloc,
aligned_alloc, and posix_memalign.  Some system libraries do call some
of these functions, and if your interposed malloc library does not
override them as well, crashes are the result.

Florian
Rical Jasan Feb. 14, 2016, 8:17 p.m. UTC | #7
On 02/12/2016 06:46 PM, Mike Frysinger wrote:
> While glibc has long officially supported replacing things like malloc,
> it hasn't really been documented.  The calloc section happens to have a
> small aside, but doesn't provide enough info.
> 
> Create a new section that explicitly lays out what is supported.
> ---
>  manual/memory.texi | 47 +++++++++++++++++++++++++++++++++++++++++++----
>  1 file changed, 43 insertions(+), 4 deletions(-)
> 
> diff --git a/manual/memory.texi b/manual/memory.texi
> index 700555e..e1426f1 100644
> --- a/manual/memory.texi
> +++ b/manual/memory.texi
> @@ -18,6 +18,7 @@ and allocation of real memory.
>  * Memory Allocation::           Allocating storage for your program data
>  * Resizing the Data Segment::   @code{brk}, @code{sbrk}
>  * Locking Pages::               Preventing page faults
> +* Custom Allocators::           Using different memory allocators at runtime
>  @end menu
>  
>  Memory mapped I/O is not discussed in this chapter.  @xref{Memory-mapped I/O}.
> @@ -577,6 +578,9 @@ ptr = (char *) malloc (length + 1);
>  @noindent
>  @xref{Representation of Strings}, for more information about this.
>  
> +If you wish to replace the @code{malloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.

+implementation, @pxref{Custom Allocators}.

That won't render correctly across all documentation types.

> +
>  @node Malloc Examples
>  @subsubsection Examples of @code{malloc}
>  
> @@ -707,6 +711,9 @@ There is no point in freeing blocks at the end of a program, because all
>  of the program's space is given back to the system when the process
>  terminates.
>  
> +If you wish to replace the @code{free} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.

+implementation, @pxref{Custom Allocators}.

> +
>  @node Changing Block Size
>  @subsubsection Changing the Size of a Block
>  @cindex changing the size of a block (@code{malloc})
> @@ -815,6 +822,9 @@ necessitates copying it, so it can fail if no other space is available.
>  If the new size you specify is the same as the old size, @code{realloc}
>  is guaranteed to change nothing and return the same address that you gave.
>  
> +If you wish to replace the @code{realloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.

+implementation, @pxref{Custom Allocators}.

> +
>  @node Allocating Cleared Space
>  @subsubsection Allocating Cleared Space
>  
> @@ -862,10 +872,8 @@ calloc (size_t count, size_t eltsize)
>  @}
>  @end smallexample
>  
> -But in general, it is not guaranteed that @code{calloc} calls
> -@code{malloc} internally.  Therefore, if an application provides its own
> -@code{malloc}/@code{realloc}/@code{free} outside the C library, it
> -should always define @code{calloc}, too.
> +If you wish to replace the @code{calloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.

+implementation, @pxref{Custom Allocators}.

>  
>  @node Efficiency and Malloc
>  @subsubsection Efficiency Considerations for @code{malloc}
> @@ -3280,8 +3288,39 @@ calls can fail, so there are no specific @code{errno} values.
>  
>  @end deftypefun
>  
> +@node Custom Allocators
> +@section Custom Allocators
>  
> +The @glibcadj{} memory allocator is designed to allow implementations to
> +provide their own replacement.  I.e. You want to write your own @code{malloc}

+provide their own replacement---you can write your own @code{malloc}

I don't think Latin abbreviations should start a sentence.  "Want" also
colors the subordinate clause in an odd way; I think "can" is more in
keeping with the intent.

> +function or use a library like tcmalloc or jemalloc, and all @glibcadj{} code
> +will call into this for memory requests.  However, care must be taken to make
> +this work correctly.
> +
> +@menu
> +* Function Replacements::       List of functions that must be implemented
> +@end menu
> +
> +@node Function Replacements
> +@subsection API for custom allocators
> +
> +Your replacement must implement all of these functions.  If you omit any of
> +them, correct behavior cannot be guaranteed.  This is because some functions
> +might not internally call the others.  E.g. @code{calloc} might not call

+might not internally call the others.  For example, @code{calloc} might
not call

Alternate strategy for avoiding the abbreviation.

> +@code{malloc}, so passing the returned pointer to @code{free} can confuse the
> +allocator.
> +
> +@itemize @bullet
> +@item @code{malloc} @xref{Basic Allocation}
> +@item @code{free} @xref{Freeing after Malloc}
> +@item @code{realloc} @xref{Changing Block Size}
> +@item @code{calloc} @xref{Allocating Cleared Space}
> +@item @code{memalign} @xref{Aligned Memory Blocks}
> +@end itemize
>  
> +All other memory functions that the @glibcadj{} provides will utilize the

+All other memory functions @theglibc{} provides will utilize the

Not an adjective here.  "That" is unnecessary.

> +ones listed above.  You may replace them, but the @glibcadj{} will not call

+ones listed above.  You may replace them, but @theglibc{} will not call

Same.

> +them directly, only through the aforementioned list.
>  
>  @ignore
>  @c This was never actually implemented.  -zw
> 

My 2¢.

Rical
diff mbox

Patch

diff --git a/manual/memory.texi b/manual/memory.texi
index 700555e..e1426f1 100644
--- a/manual/memory.texi
+++ b/manual/memory.texi
@@ -18,6 +18,7 @@  and allocation of real memory.
 * Memory Allocation::           Allocating storage for your program data
 * Resizing the Data Segment::   @code{brk}, @code{sbrk}
 * Locking Pages::               Preventing page faults
+* Custom Allocators::           Using different memory allocators at runtime
 @end menu
 
 Memory mapped I/O is not discussed in this chapter.  @xref{Memory-mapped I/O}.
@@ -577,6 +578,9 @@  ptr = (char *) malloc (length + 1);
 @noindent
 @xref{Representation of Strings}, for more information about this.
 
+If you wish to replace the @code{malloc} function with your own
+implementation, be sure to read the @ref{Custom Allocators} section.
+
 @node Malloc Examples
 @subsubsection Examples of @code{malloc}
 
@@ -707,6 +711,9 @@  There is no point in freeing blocks at the end of a program, because all
 of the program's space is given back to the system when the process
 terminates.
 
+If you wish to replace the @code{free} function with your own
+implementation, be sure to read the @ref{Custom Allocators} section.
+
 @node Changing Block Size
 @subsubsection Changing the Size of a Block
 @cindex changing the size of a block (@code{malloc})
@@ -815,6 +822,9 @@  necessitates copying it, so it can fail if no other space is available.
 If the new size you specify is the same as the old size, @code{realloc}
 is guaranteed to change nothing and return the same address that you gave.
 
+If you wish to replace the @code{realloc} function with your own
+implementation, be sure to read the @ref{Custom Allocators} section.
+
 @node Allocating Cleared Space
 @subsubsection Allocating Cleared Space
 
@@ -862,10 +872,8 @@  calloc (size_t count, size_t eltsize)
 @}
 @end smallexample
 
-But in general, it is not guaranteed that @code{calloc} calls
-@code{malloc} internally.  Therefore, if an application provides its own
-@code{malloc}/@code{realloc}/@code{free} outside the C library, it
-should always define @code{calloc}, too.
+If you wish to replace the @code{calloc} function with your own
+implementation, be sure to read the @ref{Custom Allocators} section.
 
 @node Efficiency and Malloc
 @subsubsection Efficiency Considerations for @code{malloc}
@@ -3280,8 +3288,39 @@  calls can fail, so there are no specific @code{errno} values.
 
 @end deftypefun
 
+@node Custom Allocators
+@section Custom Allocators
 
+The @glibcadj{} memory allocator is designed to allow implementations to
+provide their own replacement.  I.e. You want to write your own @code{malloc}
+function or use a library like tcmalloc or jemalloc, and all @glibcadj{} code
+will call into this for memory requests.  However, care must be taken to make
+this work correctly.
+
+@menu
+* Function Replacements::       List of functions that must be implemented
+@end menu
+
+@node Function Replacements
+@subsection API for custom allocators
+
+Your replacement must implement all of these functions.  If you omit any of
+them, correct behavior cannot be guaranteed.  This is because some functions
+might not internally call the others.  E.g. @code{calloc} might not call
+@code{malloc}, so passing the returned pointer to @code{free} can confuse the
+allocator.
+
+@itemize @bullet
+@item @code{malloc} @xref{Basic Allocation}
+@item @code{free} @xref{Freeing after Malloc}
+@item @code{realloc} @xref{Changing Block Size}
+@item @code{calloc} @xref{Allocating Cleared Space}
+@item @code{memalign} @xref{Aligned Memory Blocks}
+@end itemize
 
+All other memory functions that the @glibcadj{} provides will utilize the
+ones listed above.  You may replace them, but the @glibcadj{} will not call
+them directly, only through the aforementioned list.
 
 @ignore
 @c This was never actually implemented.  -zw