diff mbox

glibc 2.24 -- Release blockers

Message ID 5788E12C.9010205@cs.ucla.edu
State New, archived
Headers show

Commit Message

Paul Eggert July 15, 2016, 1:12 p.m. UTC
On 07/15/2016 12:52 PM, Florian Weimer wrote:
> The problem is that this happens at dump time only, while the issue 
> occurs when running the dumped Emacs. Furthermore, the personality 
> system call (or at least ADDR_NO_RANDOMIZE) only applies to new 
> process images, it does not affected the randomization decisions made 
> at load time for the current process.

I wrote a revised patch to emacs-25 (attached) which I hope addresses 
the former issue, by disabling randomization in both the undumped and 
dumped Emacs when Emacs uses its own allocator. Emacs already addresses 
the latter issue by re-execing itself when it changes personalities, so 
this shouldn't be an issue.

Comments

Sinny Kumari July 15, 2016, 2:19 p.m. UTC | #1
On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 07/15/2016 12:52 PM, Florian Weimer wrote:
>>
>> The problem is that this happens at dump time only, while the issue occurs
>> when running the dumped Emacs. Furthermore, the personality system call (or
>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not
>> affected the randomization decisions made at load time for the current
>> process.
>
>
> I wrote a revised patch to emacs-25 (attached) which I hope addresses the
> former issue, by disabling randomization in both the undumped and dumped
> Emacs when Emacs uses its own allocator. Emacs already addresses the latter
> issue by re-execing itself when it changes personalities, so this shouldn't
> be an issue.

After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le),
Fedora rawhide. It no longer consumes memory in GB !
For reference, scratch build link is
http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651
Sinny Kumari July 15, 2016, 2:28 p.m. UTC | #2
On Fri, Jul 15, 2016 at 7:49 PM, Sinny Kumari <ksinny@gmail.com> wrote:
> On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
>> On 07/15/2016 12:52 PM, Florian Weimer wrote:
>>>
>>> The problem is that this happens at dump time only, while the issue occurs
>>> when running the dumped Emacs. Furthermore, the personality system call (or
>>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not
>>> affected the randomization decisions made at load time for the current
>>> process.
>>
>>
>> I wrote a revised patch to emacs-25 (attached) which I hope addresses the
>> former issue, by disabling randomization in both the undumped and dumped
>> Emacs when Emacs uses its own allocator. Emacs already addresses the latter
>> issue by re-execing itself when it changes personalities, so this shouldn't
>> be an issue.
>
> After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le),
> Fedora rawhide. It no longer consumes memory in GB !
> For reference, scratch build link is
> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651

Also, output of "cat /proc/self/personality" inside emacs shell
contains value 00040000
Florian Weimer July 15, 2016, 2:33 p.m. UTC | #3
On 07/15/2016 04:28 PM, Sinny Kumari wrote:
> On Fri, Jul 15, 2016 at 7:49 PM, Sinny Kumari <ksinny@gmail.com> wrote:
>> On Fri, Jul 15, 2016 at 6:42 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
>>> On 07/15/2016 12:52 PM, Florian Weimer wrote:
>>>>
>>>> The problem is that this happens at dump time only, while the issue occurs
>>>> when running the dumped Emacs. Furthermore, the personality system call (or
>>>> at least ADDR_NO_RANDOMIZE) only applies to new process images, it does not
>>>> affected the randomization decisions made at load time for the current
>>>> process.
>>>
>>>
>>> I wrote a revised patch to emacs-25 (attached) which I hope addresses the
>>> former issue, by disabling randomization in both the undumped and dumped
>>> Emacs when Emacs uses its own allocator. Emacs already addresses the latter
>>> issue by re-execing itself when it changes personalities, so this shouldn't
>>> be an issue.
>>
>> After applying this patch in emacs-25.0.95, it builds and run fine on ppc64(le),
>> Fedora rawhide. It no longer consumes memory in GB !
>> For reference, scratch build link is
>> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3538651
>
> Also, output of "cat /proc/self/personality" inside emacs shell
> contains value 00040000

Thanks!

This means that new processes spawned by Emacs will have ASLR disable.

Paul, I think the patch should call personality again after it is active 
for the Emacs process, to disable it for subprocesses.

Florian
Paul Eggert July 15, 2016, 8 p.m. UTC | #4
On 07/15/2016 04:33 PM, Florian Weimer wrote:
>
> This means that new processes spawned by Emacs will have ASLR disable.

Yes, on second thought that's too drastic. The patch I just now 
installed into master (and sent via email to this list) doesn't do that. 
It disables ASLR only in temacs, not in ordinary Emacs invocations. 
(Enabling ASLR in ordinary Emacs broke the Emacs build on Fedora 23 
x86-64.) This is how Emacs has worked for many years; although you 
indicated that it might not work with bleeding-edge glibc in some 
situations, I don't understand why not, and would like to understand it 
better before changing this aspect of Emacs..
Zack Weinberg July 15, 2016, 9:19 p.m. UTC | #5
On Fri, Jul 15, 2016 at 4:00 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 07/15/2016 04:33 PM, Florian Weimer wrote:
>>
>> This means that new processes spawned by Emacs will have ASLR disable.
>
> Yes, on second thought that's too drastic. The patch I just now installed
> into master (and sent via email to this list) doesn't do that. It disables
> ASLR only in temacs, not in ordinary Emacs invocations. (Enabling ASLR in
> ordinary Emacs broke the Emacs build on Fedora 23 x86-64.) This is how Emacs
> has worked for many years; although you indicated that it might not work
> with bleeding-edge glibc in some situations, I don't understand why not, and
> would like to understand it better before changing this aspect of Emacs..

I don't have affected hardware to test this on, but the phenomenon
seems pretty clear to me.  Emacs' bundled malloc assumes that the data
segment is contiguous with, or at least close to, the memory region
controlled by brk().  This is not true with ASLR enabled, on certain
architectures (e.g. ppc64).

That assumption is a fundamental design constraint, rather than
something done to support dumping.  Therefore, if the bundled malloc
is used in the "ordinary Emacs", ASLR must be disabled in the
"ordinary Emacs"; disabling it in temacs is not enough.  It looks like
HYBRID_MALLOC mode would avoid the problem, but from upthread, that
mode isn't ready for prime time yet...?

zw
Paul Eggert July 19, 2016, 4:32 p.m. UTC | #6
On 07/15/2016 11:19 PM, Zack Weinberg wrote:
> Emacs' bundled malloc assumes that the data
> segment is contiguous with, or at least close to, the memory region
> controlled by brk().  This is not true with ASLR enabled, on certain
> architectures (e.g. ppc64).

Unfortunately Emacs with glibc 2.22 malloc seems to be making the 
opposite assumption, at least on Fedora 23 x86-64. That is, if I disable 
ASLR in an ordinary (dumped) Emacs with glibc 2.22, Emacs stops working 
(I don't know why). This leads me to be leery of disabling ASLR in an 
ordinary Emacs, except for platforms like ppc64 where it seems to be 
necessary. The patch I just sent in therefore has a special case for 
__PPC64__. Quite a hack, but without knowing the problem better I don't 
know what other hack to try.

> It looks like
> HYBRID_MALLOC mode would avoid the problem, but from upthread, that
> mode isn't ready for prime time yet...?


Yes, my sense is that HYBRID_MALLOC should work but it is not ready for 
emacs-25. It is already enabled in the master branch and we should be 
able to get it to work with ppc64 too, at least assuming the patch I 
just emailed here works on emacs-25.
diff mbox

Patch

From 6c7e7b23abe495da5ac83be6cf8e714d3160af45 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 15 Jul 2016 13:07:09 +0200
Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64

Inspired by a suggestion by Florian Weimer in:
https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html
* configure.ac (HAVE_PERSONALITY_ADDR_NO_RANDOMIZE):
Rename from HAVE_PERSONALITY_LINUX32, and check for
ADDR_NO_RANDOMIZE (the crucial thing) instead of for LINUX32.
All uses changed.
* src/emacs.c (main) [HAVE_PERSONALITY_ADDR_NO_RANDOMIZE]:
Use ADDR_NO_RANDOMIZE from personality.h rather than inventing the
flag ourselves.  Just set that flag, rather than also setting the
persona.  Do all this earlier, so as to avoid problems with calls
to brk in the interim.  When doing it, avoid functions like
putenv that may allocate memory.
---
 admin/CPP-DEFINES |  2 +-
 configure.ac      | 20 +++++++++++---------
 src/emacs.c       | 55 ++++++++++++++++++++++++++++++++-----------------------
 3 files changed, 44 insertions(+), 33 deletions(-)

diff --git a/admin/CPP-DEFINES b/admin/CPP-DEFINES
index 796b57d..d404dee 100644
--- a/admin/CPP-DEFINES
+++ b/admin/CPP-DEFINES
@@ -237,7 +237,7 @@  HAVE_NET_IF_DL_H
 HAVE_NET_IF_H
 HAVE_NLIST_H
 HAVE_OTF_GET_VARIATION_GLYPHS
-HAVE_PERSONALITY_LINUX32
+HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 HAVE_PNG
 HAVE_PNG_H
 HAVE_POSIX_MEMALIGN
diff --git a/configure.ac b/configure.ac
index 678e98e..9da23d1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1615,15 +1615,17 @@  AC_CHECK_HEADERS_ONCE(
   sys/resource.h
   sys/utsname.h pwd.h utmp.h util.h)
 
-AC_MSG_CHECKING(if personality LINUX32 can be set)
-AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/personality.h>]], [[personality (PER_LINUX32)]])],
-               emacs_cv_personality_linux32=yes,
-	       emacs_cv_personality_linux32=no)
-AC_MSG_RESULT($emacs_cv_personality_linux32)
-
-if test $emacs_cv_personality_linux32 = yes; then
-  AC_DEFINE(HAVE_PERSONALITY_LINUX32, 1,
-            [Define to 1 if personality LINUX32 can be set.])
+AC_CACHE_CHECK([if personality ADDR_NO_RANDOMIZE flag exists],
+  [emacs_cv_personality_addr_no_randomize],
+  [AC_COMPILE_IFELSE(
+     [AC_LANG_PROGRAM([[#include <sys/personality.h>]],
+		      [[personality (personality (0xffffffff)
+				     | ADDR_NO_RANDOMIZE)]])],
+     [emacs_cv_personality_addr_no_randomize=yes],
+     [emacs_cv_personality_addr_no_randomize=no])])
+if test $emacs_cv_personality_addr_no_randomize = yes; then
+  AC_DEFINE([HAVE_PERSONALITY_ADDR_NO_RANDOMIZE], [1],
+            [Define to 1 if personality flag ADDR_NO_RANDOMIZE exists.])
 fi
 
 # Note that Solaris has sys/sysinfo.h which defines struct
diff --git a/src/emacs.c b/src/emacs.c
index 5c187e7..901bc0c 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -106,7 +106,7 @@  extern void moncontrol (int mode);
 #include <sys/resource.h>
 #endif
 
-#ifdef HAVE_PERSONALITY_LINUX32
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 #include <sys/personality.h>
 #endif
 
@@ -674,6 +674,37 @@  main (int argc, char **argv)
 
   stack_base = &dummy;
 
+  dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
+			     || strcmp (argv[argc - 1], "bootstrap") == 0);
+
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
+
+  /* True if address randomization interferes with memory allocaiton.  */
+# ifdef HYBRID_MALLOC
+  bool disable_aslr = dumping;
+# elif defined SYSTEM_MALLOC || defined DOUG_LEA_MALLOC
+  bool disable_aslr = false;
+# else
+  bool disable_aslr = true;
+# endif
+
+  if (disable_aslr)
+    {
+      int pers = personality (0xffffffff);
+      if (! (pers & ADDR_NO_RANDOMIZE)
+	  && 0 <= personality (pers | ADDR_NO_RANDOMIZE))
+	{
+	  /* Address randomization was enabled, but is now disabled.
+	     Re-execute Emacs to get a clean slate.  */
+	  execvp (argv[0], argv);
+
+	  /* If the exec fails, warn the user and then try without a
+	     clean slate.  */
+	  fprintf (stderr, "%s: %s\n", argv[0], strerror (errno));
+	}
+    }
+#endif
+
 #ifndef CANNOT_DUMP
   might_dump = !initialized;
 #endif
@@ -781,28 +812,6 @@  main (int argc, char **argv)
         }
     }
 
-  dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
-			     || strcmp (argv[argc - 1], "bootstrap") == 0);
-
-#ifdef HAVE_PERSONALITY_LINUX32
-  if (dumping && ! getenv ("EMACS_HEAP_EXEC"))
-    {
-      /* Set this so we only do this once.  */
-      xputenv ("EMACS_HEAP_EXEC=true");
-
-      /* A flag to turn off address randomization which is introduced
-         in linux kernel shipped with fedora core 4 */
-#define ADD_NO_RANDOMIZE 0x0040000
-      personality (PER_LINUX32 | ADD_NO_RANDOMIZE);
-#undef  ADD_NO_RANDOMIZE
-
-      execvp (argv[0], argv);
-
-      /* If the exec fails, try to dump anyway.  */
-      emacs_perror (argv[0]);
-    }
-#endif /* HAVE_PERSONALITY_LINUX32 */
-
 #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN)
   /* Extend the stack space available.  Don't do that if dumping,
      since some systems (e.g. DJGPP) might define a smaller stack
-- 
2.5.5