manual: memory: document support custom allocators
Commit Message
While glibc has long officially supported replacing things like malloc,
it hasn't really been documented. The calloc section happens to have a
small aside, but doesn't provide enough info.
Create a new section that explicitly lays out what is supported.
---
manual/memory.texi | 47 +++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 43 insertions(+), 4 deletions(-)
Comments
Mike Frysinger <vapier@gentoo.org> writes:
> +If you wish to replace the @code{malloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.
@ref is only useful at the end of a sentence.
Andreas.
On 13 Feb 2016 10:09, Andreas Schwab wrote:
> Mike Frysinger <vapier@gentoo.org> writes:
> > +If you wish to replace the @code{malloc} function with your own
> > +implementation, be sure to read the @ref{Custom Allocators} section.
>
> @ref is only useful at the end of a sentence.
seems to render fine in html & info, and there are other cases in the
manual where @ref isn't at the end of a sentence.
-mike
On 02/13/2016 03:46 AM, Mike Frysinger wrote:
> +@node Function Replacements
> +@subsection API for custom allocators
> +
> +Your replacement must implement all of these functions. If you omit any of
> +them, correct behavior cannot be guaranteed. This is because some functions
> +might not internally call the others. E.g. @code{calloc} might not call
> +@code{malloc}, so passing the returned pointer to @code{free} can confuse the
> +allocator.
> +
> +@itemize @bullet
> +@item @code{malloc} @xref{Basic Allocation}
> +@item @code{free} @xref{Freeing after Malloc}
> +@item @code{realloc} @xref{Changing Block Size}
> +@item @code{calloc} @xref{Allocating Cleared Space}
> +@item @code{memalign} @xref{Aligned Memory Blocks}
> +@end itemize
>
> +All other memory functions that the @glibcadj{} provides will utilize the
> +ones listed above. You may replace them, but the @glibcadj{} will not call
> +them directly, only through the aforementioned list.
I think this should note that other system libraries may call the other
functions, and that for full compatibility. an interposed malloc should
override these functions as well.
Florian
Mike Frysinger <vapier@gentoo.org> writes:
> On 13 Feb 2016 10:09, Andreas Schwab wrote:
>> Mike Frysinger <vapier@gentoo.org> writes:
>> > +If you wish to replace the @code{malloc} function with your own
>> > +implementation, be sure to read the @ref{Custom Allocators} section.
>>
>> @ref is only useful at the end of a sentence.
>
> seems to render fine in html & info,
Try pdf or dvi.
Andreas.
On 13 Feb 2016 11:38, Florian Weimer wrote:
> On 02/13/2016 03:46 AM, Mike Frysinger wrote:
> > +@node Function Replacements
> > +@subsection API for custom allocators
> > +
> > +Your replacement must implement all of these functions. If you omit any of
> > +them, correct behavior cannot be guaranteed. This is because some functions
> > +might not internally call the others. E.g. @code{calloc} might not call
> > +@code{malloc}, so passing the returned pointer to @code{free} can confuse the
> > +allocator.
> > +
> > +@itemize @bullet
> > +@item @code{malloc} @xref{Basic Allocation}
> > +@item @code{free} @xref{Freeing after Malloc}
> > +@item @code{realloc} @xref{Changing Block Size}
> > +@item @code{calloc} @xref{Allocating Cleared Space}
> > +@item @code{memalign} @xref{Aligned Memory Blocks}
> > +@end itemize
> >
> > +All other memory functions that the @glibcadj{} provides will utilize the
> > +ones listed above. You may replace them, but the @glibcadj{} will not call
> > +them directly, only through the aforementioned list.
>
> I think this should note that other system libraries may call the other
> functions, and that for full compatibility. an interposed malloc should
> override these functions as well.
i don't think that's what we want. even if another library (be it part of
glibc or otherwise) calls a function not overridden, those will eventually
call into malloc(). the funcs i listed where the only ones that show up
in a `readelf -r` listing of libc.so.
-mike
On 02/13/2016 12:07 PM, Mike Frysinger wrote:
> On 13 Feb 2016 11:38, Florian Weimer wrote:
>> On 02/13/2016 03:46 AM, Mike Frysinger wrote:
>>> +@node Function Replacements +@subsection API for custom
>>> allocators + +Your replacement must implement all of these
>>> functions. If you omit any of +them, correct behavior cannot
>>> be guaranteed. This is because some functions +might not
>>> internally call the others. E.g. @code{calloc} might not call
>>> +@code{malloc}, so passing the returned pointer to @code{free}
>>> can confuse the +allocator. + +@itemize @bullet +@item
>>> @code{malloc} @xref{Basic Allocation} +@item @code{free}
>>> @xref{Freeing after Malloc} +@item @code{realloc}
>>> @xref{Changing Block Size} +@item @code{calloc}
>>> @xref{Allocating Cleared Space} +@item @code{memalign}
>>> @xref{Aligned Memory Blocks} +@end itemize
>>>
>>> +All other memory functions that the @glibcadj{} provides will
>>> utilize the +ones listed above. You may replace them, but the
>>> @glibcadj{} will not call +them directly, only through the
>>> aforementioned list.
>>
>> I think this should note that other system libraries may call the
>> other functions, and that for full compatibility. an interposed
>> malloc should override these functions as well.
>
> i don't think that's what we want. even if another library (be it
> part of glibc or otherwise) calls a function not overridden, those
> will eventually call into malloc(). the funcs i listed where the
> only ones that show up in a `readelf -r` listing of libc.so.
To be clear, I am talking about malloc_usable_size, valloc, pvalloc,
aligned_alloc, and posix_memalign. Some system libraries do call some
of these functions, and if your interposed malloc library does not
override them as well, crashes are the result.
Florian
On 02/12/2016 06:46 PM, Mike Frysinger wrote:
> While glibc has long officially supported replacing things like malloc,
> it hasn't really been documented. The calloc section happens to have a
> small aside, but doesn't provide enough info.
>
> Create a new section that explicitly lays out what is supported.
> ---
> manual/memory.texi | 47 +++++++++++++++++++++++++++++++++++++++++++----
> 1 file changed, 43 insertions(+), 4 deletions(-)
>
> diff --git a/manual/memory.texi b/manual/memory.texi
> index 700555e..e1426f1 100644
> --- a/manual/memory.texi
> +++ b/manual/memory.texi
> @@ -18,6 +18,7 @@ and allocation of real memory.
> * Memory Allocation:: Allocating storage for your program data
> * Resizing the Data Segment:: @code{brk}, @code{sbrk}
> * Locking Pages:: Preventing page faults
> +* Custom Allocators:: Using different memory allocators at runtime
> @end menu
>
> Memory mapped I/O is not discussed in this chapter. @xref{Memory-mapped I/O}.
> @@ -577,6 +578,9 @@ ptr = (char *) malloc (length + 1);
> @noindent
> @xref{Representation of Strings}, for more information about this.
>
> +If you wish to replace the @code{malloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.
+implementation, @pxref{Custom Allocators}.
That won't render correctly across all documentation types.
> +
> @node Malloc Examples
> @subsubsection Examples of @code{malloc}
>
> @@ -707,6 +711,9 @@ There is no point in freeing blocks at the end of a program, because all
> of the program's space is given back to the system when the process
> terminates.
>
> +If you wish to replace the @code{free} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.
+implementation, @pxref{Custom Allocators}.
> +
> @node Changing Block Size
> @subsubsection Changing the Size of a Block
> @cindex changing the size of a block (@code{malloc})
> @@ -815,6 +822,9 @@ necessitates copying it, so it can fail if no other space is available.
> If the new size you specify is the same as the old size, @code{realloc}
> is guaranteed to change nothing and return the same address that you gave.
>
> +If you wish to replace the @code{realloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.
+implementation, @pxref{Custom Allocators}.
> +
> @node Allocating Cleared Space
> @subsubsection Allocating Cleared Space
>
> @@ -862,10 +872,8 @@ calloc (size_t count, size_t eltsize)
> @}
> @end smallexample
>
> -But in general, it is not guaranteed that @code{calloc} calls
> -@code{malloc} internally. Therefore, if an application provides its own
> -@code{malloc}/@code{realloc}/@code{free} outside the C library, it
> -should always define @code{calloc}, too.
> +If you wish to replace the @code{calloc} function with your own
> +implementation, be sure to read the @ref{Custom Allocators} section.
+implementation, @pxref{Custom Allocators}.
>
> @node Efficiency and Malloc
> @subsubsection Efficiency Considerations for @code{malloc}
> @@ -3280,8 +3288,39 @@ calls can fail, so there are no specific @code{errno} values.
>
> @end deftypefun
>
> +@node Custom Allocators
> +@section Custom Allocators
>
> +The @glibcadj{} memory allocator is designed to allow implementations to
> +provide their own replacement. I.e. You want to write your own @code{malloc}
+provide their own replacement---you can write your own @code{malloc}
I don't think Latin abbreviations should start a sentence. "Want" also
colors the subordinate clause in an odd way; I think "can" is more in
keeping with the intent.
> +function or use a library like tcmalloc or jemalloc, and all @glibcadj{} code
> +will call into this for memory requests. However, care must be taken to make
> +this work correctly.
> +
> +@menu
> +* Function Replacements:: List of functions that must be implemented
> +@end menu
> +
> +@node Function Replacements
> +@subsection API for custom allocators
> +
> +Your replacement must implement all of these functions. If you omit any of
> +them, correct behavior cannot be guaranteed. This is because some functions
> +might not internally call the others. E.g. @code{calloc} might not call
+might not internally call the others. For example, @code{calloc} might
not call
Alternate strategy for avoiding the abbreviation.
> +@code{malloc}, so passing the returned pointer to @code{free} can confuse the
> +allocator.
> +
> +@itemize @bullet
> +@item @code{malloc} @xref{Basic Allocation}
> +@item @code{free} @xref{Freeing after Malloc}
> +@item @code{realloc} @xref{Changing Block Size}
> +@item @code{calloc} @xref{Allocating Cleared Space}
> +@item @code{memalign} @xref{Aligned Memory Blocks}
> +@end itemize
>
> +All other memory functions that the @glibcadj{} provides will utilize the
+All other memory functions @theglibc{} provides will utilize the
Not an adjective here. "That" is unnecessary.
> +ones listed above. You may replace them, but the @glibcadj{} will not call
+ones listed above. You may replace them, but @theglibc{} will not call
Same.
> +them directly, only through the aforementioned list.
>
> @ignore
> @c This was never actually implemented. -zw
>
My 2¢.
Rical
@@ -18,6 +18,7 @@ and allocation of real memory.
* Memory Allocation:: Allocating storage for your program data
* Resizing the Data Segment:: @code{brk}, @code{sbrk}
* Locking Pages:: Preventing page faults
+* Custom Allocators:: Using different memory allocators at runtime
@end menu
Memory mapped I/O is not discussed in this chapter. @xref{Memory-mapped I/O}.
@@ -577,6 +578,9 @@ ptr = (char *) malloc (length + 1);
@noindent
@xref{Representation of Strings}, for more information about this.
+If you wish to replace the @code{malloc} function with your own
+implementation, be sure to read the @ref{Custom Allocators} section.
+
@node Malloc Examples
@subsubsection Examples of @code{malloc}
@@ -707,6 +711,9 @@ There is no point in freeing blocks at the end of a program, because all
of the program's space is given back to the system when the process
terminates.
+If you wish to replace the @code{free} function with your own
+implementation, be sure to read the @ref{Custom Allocators} section.
+
@node Changing Block Size
@subsubsection Changing the Size of a Block
@cindex changing the size of a block (@code{malloc})
@@ -815,6 +822,9 @@ necessitates copying it, so it can fail if no other space is available.
If the new size you specify is the same as the old size, @code{realloc}
is guaranteed to change nothing and return the same address that you gave.
+If you wish to replace the @code{realloc} function with your own
+implementation, be sure to read the @ref{Custom Allocators} section.
+
@node Allocating Cleared Space
@subsubsection Allocating Cleared Space
@@ -862,10 +872,8 @@ calloc (size_t count, size_t eltsize)
@}
@end smallexample
-But in general, it is not guaranteed that @code{calloc} calls
-@code{malloc} internally. Therefore, if an application provides its own
-@code{malloc}/@code{realloc}/@code{free} outside the C library, it
-should always define @code{calloc}, too.
+If you wish to replace the @code{calloc} function with your own
+implementation, be sure to read the @ref{Custom Allocators} section.
@node Efficiency and Malloc
@subsubsection Efficiency Considerations for @code{malloc}
@@ -3280,8 +3288,39 @@ calls can fail, so there are no specific @code{errno} values.
@end deftypefun
+@node Custom Allocators
+@section Custom Allocators
+The @glibcadj{} memory allocator is designed to allow implementations to
+provide their own replacement. I.e. You want to write your own @code{malloc}
+function or use a library like tcmalloc or jemalloc, and all @glibcadj{} code
+will call into this for memory requests. However, care must be taken to make
+this work correctly.
+
+@menu
+* Function Replacements:: List of functions that must be implemented
+@end menu
+
+@node Function Replacements
+@subsection API for custom allocators
+
+Your replacement must implement all of these functions. If you omit any of
+them, correct behavior cannot be guaranteed. This is because some functions
+might not internally call the others. E.g. @code{calloc} might not call
+@code{malloc}, so passing the returned pointer to @code{free} can confuse the
+allocator.
+
+@itemize @bullet
+@item @code{malloc} @xref{Basic Allocation}
+@item @code{free} @xref{Freeing after Malloc}
+@item @code{realloc} @xref{Changing Block Size}
+@item @code{calloc} @xref{Allocating Cleared Space}
+@item @code{memalign} @xref{Aligned Memory Blocks}
+@end itemize
+All other memory functions that the @glibcadj{} provides will utilize the
+ones listed above. You may replace them, but the @glibcadj{} will not call
+them directly, only through the aforementioned list.
@ignore
@c This was never actually implemented. -zw