glibc 2.24 -- Release blockers
Commit Message
On 07/15/2016 12:52 PM, Florian Weimer wrote:
> The problem is that this happens at dump time only, while the issue
> occurs when running the dumped Emacs. Furthermore, the personality
> system call (or at least ADDR_NO_RANDOMIZE) only applies to new
> process images, it does not affected the randomization decisions made
> at load time for the current process.
I wrote a revised patch to emacs-25 (attached) which I hope addresses
the former issue, by disabling randomization in both the undumped and
dumped Emacs when Emacs uses its own allocator. Emacs already addresses
the latter issue by re-execing itself when it changes personalities, so
this shouldn't be an issue.
Comments
On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 07/15/2016 12:52 PM, Florian Weimer wrote:
>>
>> The problem is that this happens at dump time only, while the issue occurs
>> when running the dumped Emacs. Furthermore, the personality system call (or
>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not
>> affected the randomization decisions made at load time for the current
>> process.
>
>
> I wrote a revised patch to emacs-25 (attached) which I hope addresses the
> former issue, by disabling randomization in both the undumped and dumped
> Emacs when Emacs uses its own allocator. Emacs already addresses the latter
> issue by re-execing itself when it changes personalities, so this shouldn't
> be an issue.
After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le),
Fedora rawhide. It no longer consumes memory in GB !
For reference, scratch build link is
http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651
On Fri, Jul 15, 2016 at 7:49 PM, Sinny Kumari <ksinny@gmail.com> wrote:
> On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
>> On 07/15/2016 12:52 PM, Florian Weimer wrote:
>>>
>>> The problem is that this happens at dump time only, while the issue occurs
>>> when running the dumped Emacs. Furthermore, the personality system call (or
>>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not
>>> affected the randomization decisions made at load time for the current
>>> process.
>>
>>
>> I wrote a revised patch to emacs-25 (attached) which I hope addresses the
>> former issue, by disabling randomization in both the undumped and dumped
>> Emacs when Emacs uses its own allocator. Emacs already addresses the latter
>> issue by re-execing itself when it changes personalities, so this shouldn't
>> be an issue.
>
> After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le),
> Fedora rawhide. It no longer consumes memory in GB !
> For reference, scratch build link is
> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651
Also, output of "cat /proc/self/personality" inside emacs shell
contains value 00040000
On 07/15/2016 04:28 PM, Sinny Kumari wrote:
> On Fri, Jul 15, 2016 at 7:49 PM, Sinny Kumari <ksinny@gmail.com> wrote:
>> On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
>>> On 07/15/2016 12:52 PM, Florian Weimer wrote:
>>>>
>>>> The problem is that this happens at dump time only, while the issue occurs
>>>> when running the dumped Emacs. Furthermore, the personality system call (or
>>>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not
>>>> affected the randomization decisions made at load time for the current
>>>> process.
>>>
>>>
>>> I wrote a revised patch to emacs-25 (attached) which I hope addresses the
>>> former issue, by disabling randomization in both the undumped and dumped
>>> Emacs when Emacs uses its own allocator. Emacs already addresses the latter
>>> issue by re-execing itself when it changes personalities, so this shouldn't
>>> be an issue.
>>
>> After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le),
>> Fedora rawhide. It no longer consumes memory in GB !
>> For reference, scratch build link is
>> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651
>
> Also, output of "cat /proc/self/personality" inside emacs shell
> contains value 00040000
Thanks!
This means that new processes spawned by Emacs will have ASLR disable.
Paul, I think the patch should call personality again after it is active
for the Emacs process, to disable it for subprocesses.
Florian
On 07/15/2016 04:33 PM, Florian Weimer wrote:
>
> This means that new processes spawned by Emacs will have ASLR disable.
Yes, on second thought that's too drastic. The patch I just now
installed into master (and sent via email to this list) doesn't do that.
It disables ASLR only in temacs, not in ordinary Emacs invocations.
(Enabling ASLR in ordinary Emacs broke the Emacs build on Fedora 23
x86-64.) This is how Emacs has worked for many years; although you
indicated that it might not work with bleeding-edge glibc in some
situations, I don't understand why not, and would like to understand it
better before changing this aspect of Emacs..
On Fri, Jul 15, 2016 at 4:00 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 07/15/2016 04:33 PM, Florian Weimer wrote:
>>
>> This means that new processes spawned by Emacs will have ASLR disable.
>
> Yes, on second thought that's too drastic. The patch I just now installed
> into master (and sent via email to this list) doesn't do that. It disables
> ASLR only in temacs, not in ordinary Emacs invocations. (Enabling ASLR in
> ordinary Emacs broke the Emacs build on Fedora 23 x86-64.) This is how Emacs
> has worked for many years; although you indicated that it might not work
> with bleeding-edge glibc in some situations, I don't understand why not, and
> would like to understand it better before changing this aspect of Emacs..
I don't have affected hardware to test this on, but the phenomenon
seems pretty clear to me. Emacs' bundled malloc assumes that the data
segment is contiguous with, or at least close to, the memory region
controlled by brk(). This is not true with ASLR enabled, on certain
architectures (e.g. ppc64).
That assumption is a fundamental design constraint, rather than
something done to support dumping. Therefore, if the bundled malloc
is used in the "ordinary Emacs", ASLR must be disabled in the
"ordinary Emacs"; disabling it in temacs is not enough. It looks like
HYBRID_MALLOC mode would avoid the problem, but from upthread, that
mode isn't ready for prime time yet...?
zw
On 07/15/2016 11:19 PM, Zack Weinberg wrote:
> Emacs' bundled malloc assumes that the data
> segment is contiguous with, or at least close to, the memory region
> controlled by brk(). This is not true with ASLR enabled, on certain
> architectures (e.g. ppc64).
Unfortunately Emacs with glibc 2.22 malloc seems to be making the
opposite assumption, at least on Fedora 23 x86-64. That is, if I disable
ASLR in an ordinary (dumped) Emacs with glibc 2.22, Emacs stops working
(I don't know why). This leads me to be leery of disabling ASLR in an
ordinary Emacs, except for platforms like ppc64 where it seems to be
necessary. The patch I just sent in therefore has a special case for
__PPC64__. Quite a hack, but without knowing the problem better I don't
know what other hack to try.
> It looks like
> HYBRID_MALLOC mode would avoid the problem, but from upthread, that
> mode isn't ready for prime time yet...?
Yes, my sense is that HYBRID_MALLOC should work but it is not ready for
emacs-25. It is already enabled in the master branch and we should be
able to get it to work with ppc64 too, at least assuming the patch I
just emailed here works on emacs-25.
From 6c7e7b23abe495da5ac83be6cf8e714d3160af45 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 15 Jul 2016 13:07:09 +0200
Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64
Inspired by a suggestion by Florian Weimer in:
https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html
* configure.ac (HAVE_PERSONALITY_ADDR_NO_RANDOMIZE):
Rename from HAVE_PERSONALITY_LINUX32, and check for
ADDR_NO_RANDOMIZE (the crucial thing) instead of for LINUX32.
All uses changed.
* src/emacs.c (main) [HAVE_PERSONALITY_ADDR_NO_RANDOMIZE]:
Use ADDR_NO_RANDOMIZE from personality.h rather than inventing the
flag ourselves. Just set that flag, rather than also setting the
persona. Do all this earlier, so as to avoid problems with calls
to brk in the interim. When doing it, avoid functions like
putenv that may allocate memory.
---
admin/CPP-DEFINES | 2 +-
configure.ac | 20 +++++++++++---------
src/emacs.c | 55 ++++++++++++++++++++++++++++++++-----------------------
3 files changed, 44 insertions(+), 33 deletions(-)
@@ -237,7 +237,7 @@ HAVE_NET_IF_DL_H
HAVE_NET_IF_H
HAVE_NLIST_H
HAVE_OTF_GET_VARIATION_GLYPHS
-HAVE_PERSONALITY_LINUX32
+HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
HAVE_PNG
HAVE_PNG_H
HAVE_POSIX_MEMALIGN
@@ -1615,15 +1615,17 @@ AC_CHECK_HEADERS_ONCE(
sys/resource.h
sys/utsname.h pwd.h utmp.h util.h)
-AC_MSG_CHECKING(if personality LINUX32 can be set)
-AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/personality.h>]], [[personality (PER_LINUX32)]])],
- emacs_cv_personality_linux32=yes,
- emacs_cv_personality_linux32=no)
-AC_MSG_RESULT($emacs_cv_personality_linux32)
-
-if test $emacs_cv_personality_linux32 = yes; then
- AC_DEFINE(HAVE_PERSONALITY_LINUX32, 1,
- [Define to 1 if personality LINUX32 can be set.])
+AC_CACHE_CHECK([if personality ADDR_NO_RANDOMIZE flag exists],
+ [emacs_cv_personality_addr_no_randomize],
+ [AC_COMPILE_IFELSE(
+ [AC_LANG_PROGRAM([[#include <sys/personality.h>]],
+ [[personality (personality (0xffffffff)
+ | ADDR_NO_RANDOMIZE)]])],
+ [emacs_cv_personality_addr_no_randomize=yes],
+ [emacs_cv_personality_addr_no_randomize=no])])
+if test $emacs_cv_personality_addr_no_randomize = yes; then
+ AC_DEFINE([HAVE_PERSONALITY_ADDR_NO_RANDOMIZE], [1],
+ [Define to 1 if personality flag ADDR_NO_RANDOMIZE exists.])
fi
# Note that Solaris has sys/sysinfo.h which defines struct
@@ -106,7 +106,7 @@ extern void moncontrol (int mode);
#include <sys/resource.h>
#endif
-#ifdef HAVE_PERSONALITY_LINUX32
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
#include <sys/personality.h>
#endif
@@ -674,6 +674,37 @@ main (int argc, char **argv)
stack_base = &dummy;
+ dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
+ || strcmp (argv[argc - 1], "bootstrap") == 0);
+
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
+
+ /* True if address randomization interferes with memory allocaiton. */
+# ifdef HYBRID_MALLOC
+ bool disable_aslr = dumping;
+# elif defined SYSTEM_MALLOC || defined DOUG_LEA_MALLOC
+ bool disable_aslr = false;
+# else
+ bool disable_aslr = true;
+# endif
+
+ if (disable_aslr)
+ {
+ int pers = personality (0xffffffff);
+ if (! (pers & ADDR_NO_RANDOMIZE)
+ && 0 <= personality (pers | ADDR_NO_RANDOMIZE))
+ {
+ /* Address randomization was enabled, but is now disabled.
+ Re-execute Emacs to get a clean slate. */
+ execvp (argv[0], argv);
+
+ /* If the exec fails, warn the user and then try without a
+ clean slate. */
+ fprintf (stderr, "%s: %s\n", argv[0], strerror (errno));
+ }
+ }
+#endif
+
#ifndef CANNOT_DUMP
might_dump = !initialized;
#endif
@@ -781,28 +812,6 @@ main (int argc, char **argv)
}
}
- dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
- || strcmp (argv[argc - 1], "bootstrap") == 0);
-
-#ifdef HAVE_PERSONALITY_LINUX32
- if (dumping && ! getenv ("EMACS_HEAP_EXEC"))
- {
- /* Set this so we only do this once. */
- xputenv ("EMACS_HEAP_EXEC=true");
-
- /* A flag to turn off address randomization which is introduced
- in linux kernel shipped with fedora core 4 */
-#define ADD_NO_RANDOMIZE 0x0040000
- personality (PER_LINUX32 | ADD_NO_RANDOMIZE);
-#undef ADD_NO_RANDOMIZE
-
- execvp (argv[0], argv);
-
- /* If the exec fails, try to dump anyway. */
- emacs_perror (argv[0]);
- }
-#endif /* HAVE_PERSONALITY_LINUX32 */
-
#if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN)
/* Extend the stack space available. Don't do that if dumping,
since some systems (e.g. DJGPP) might define a smaller stack
--
2.5.5