diff mbox

glibc 2.24 -- Release blockers

Message ID 1663a6c7-7c30-c42b-8f2a-aaa98d35388c@cs.ucla.edu
State New, archived
Headers show

Commit Message

Paul Eggert July 19, 2016, 3:02 p.m. UTC
On 07/18/2016 08:49 AM, Sinny Kumari wrote:
> Sadly, after applying this patch, build fails for Fedora rawhide on
> both ppc64 and
> ppc64le due to Memory exhausted problem.
>
> koji scratch build link -
> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3540001
> Direct link to build log of ppc64 -
> http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/2/3540002/build.log

Thanks for checking. I thought about it for a bit and came up with the 
attached Emacs patch instead, which works for me on x86-64 and x86 
Fedora 23. Can you please try it on ppc64? I assume __PPC64__ is 
defined, if not please feel to substitute the proper symbol and please 
let me know what that symbol is.

This patch applies to Emacs's emacs-25 branch; if it works I plan to 
port it to Emacs's master branch. The patch does not implement Florian's 
idea of re-enabling ASLR before Emacs execs a child process, as I want 
to keep the emacs-25 patch as simple as possible. However, I do plan to 
try Florian's idea in the master branch.

Comments

Sinny Kumari July 20, 2016, 7:03 a.m. UTC | #1
On Tue, Jul 19, 2016 at 8:32 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 07/18/2016 08:49 AM, Sinny Kumari wrote:
>>
>> Sadly, after applying this patch, build fails for Fedora rawhide on
>> both ppc64 and
>> ppc64le due to Memory exhausted problem.
>>
>> koji scratch build link -
>> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3540001
>> Direct link to build log of ppc64 -
>> http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/2/3540002/build.log
>
>
> Thanks for checking. I thought about it for a bit and came up with the
> attached Emacs patch instead, which works for me on x86-64 and x86 Fedora
> 23. Can you please try it on ppc64? I assume __PPC64__ is defined, if not
> please feel to substitute the proper symbol and please let me know what that
> symbol is.

Yes, __PPC64__ macro is defined.
Applied this patch and emacs builds fine for Fedora rawhide [1] and
Fedora 24 [2] for
ppc64(le) arches.
Ran updated emacs on Fedora rawhide, ppc64 box and it uses few megabytes memory.
Checked content of  /proc/self/personality inside emacs shell and it has value
00040000.

[1] http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3544685
[2] http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3544697
Paul Eggert July 20, 2016, 8:07 a.m. UTC | #2
Thanks for checking it. I have pushed a patch into Emacs master which 
incorporates this idea plus Florian's suggestion about restoring 
randomization in child processes; please give it a try.

Also, I have filed a bug report and proposed patch for the emacs-25 
branch here:

http://bugs.gnu.org/24033

and will try to convince the Emacs maintainers that the patch is worth 
installing into emacs-25 even though it's pretty late in emacs-25's 
release schedule.
Sinny Kumari July 21, 2016, 10:44 a.m. UTC | #3
On Wed, Jul 20, 2016 at 1:37 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> Thanks for checking it. I have pushed a patch into Emacs master which
> incorporates this idea plus Florian's suggestion about restoring
> randomization in child processes; please give it a try.

So far, didn't succeeded to test patch pushed in master branch.
At first trying to reproduce memory issue by building source tar obtained
from commit bf5ddded70c11edaf3514b25da27fc71cfb8e965 and
098d29af10a869e65a36804579f16edc49aabcfa . But didn't see memory
issue in those commits.
I think, this is because ./configure gives following output:

 Should Emacs use the GNU version of malloc?             no (only
before dumping)
 Should Emacs use a relocating allocator for buffers?    no

> Also, I have filed a bug report and proposed patch for the emacs-25 branch
> here:
>
> http://bugs.gnu.org/24033

Thanks Paul for proposing it for emacs-25. I tested final patch proposed at
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11 and it works on
ppc64(le) architectures.

> and will try to convince the Emacs maintainers that the patch is worth
> installing into emacs-25 even though it's pretty late in emacs-25's release
> schedule.

Glad to see that fix will be included in emacs-25.
Paul Eggert July 21, 2016, 11:25 a.m. UTC | #4
On 07/21/2016 12:44 PM, Sinny Kumari wrote:
> Thanks Paul for proposing it for emacs-25. I tested final patch proposed at
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11  and it works on
> ppc64(le) architectures.

Ah, sorry, our emails crossed paths. Thanks for checking, and I'll 
install the patch into emacs-25.
Aurelien Jarno Aug. 7, 2016, 11:19 a.m. UTC | #5
On 2016-07-21 13:25, Paul Eggert wrote:
> On 07/21/2016 12:44 PM, Sinny Kumari wrote:
> > Thanks Paul for proposing it for emacs-25. I tested final patch proposed at
> > http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11  and it works on
> > ppc64(le) architectures.
> 
> Ah, sorry, our emails crossed paths. Thanks for checking, and I'll install
> the patch into emacs-25.

Thanks for doing so. I got a look at the emacs git, and found your patch
to backport to emacs-24. However as said on the wiki [1], it is also
necessary to compile src/gmalloc.c with -ffreestanding. I have done that
and it indeed works fine. I have found it is necessary to do so starting
with GCC 5, however I haven't found any corresponding in the emacs git.
Do you have any pointer to an upstream patch?

Aurelien

[1] https://sourceware.org/glibc/wiki/Release/2.24#Known_Issues
Andreas Schwab Aug. 7, 2016, 12:19 p.m. UTC | #6
On So, Aug 07 2016, Aurelien Jarno <aurelien@aurel32.net> wrote:

> Thanks for doing so. I got a look at the emacs git, and found your patch
> to backport to emacs-24. However as said on the wiki [1], it is also
> necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> and it indeed works fine. I have found it is necessary to do so starting
> with GCC 5, however I haven't found any corresponding in the emacs git.
> Do you have any pointer to an upstream patch?

Do you mean commit 4b1436b?

Andreas.
Paul Eggert Aug. 7, 2016, 7:35 p.m. UTC | #7
Aurelien Jarno wrote:
> I got a look at the emacs git, and found your patch
> to backport to emacs-24. However as said on the wiki [1], it is also
> necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> and it indeed works fine.

Is that still really needed with the emacs-25 branch? As Andreas notes, it has a 
different way of avoiding the calloc issue. What happens if you build the 
emacs-25 branch without using -ffreestanding?
Aurelien Jarno Aug. 7, 2016, 9:47 p.m. UTC | #8
On 2016-08-07 14:19, Andreas Schwab wrote:
> On So, Aug 07 2016, Aurelien Jarno <aurelien@aurel32.net> wrote:
> 
> > Thanks for doing so. I got a look at the emacs git, and found your patch
> > to backport to emacs-24. However as said on the wiki [1], it is also
> > necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> > and it indeed works fine. I have found it is necessary to do so starting
> > with GCC 5, however I haven't found any corresponding in the emacs git.
> > Do you have any pointer to an upstream patch?
> 
> Do you mean commit 4b1436b?

Thanks for the hint, the wiki talked about -ffreestanding, so I was
looking for a commit containing it.

Aurelien
Aurelien Jarno Aug. 7, 2016, 9:48 p.m. UTC | #9
On 2016-08-07 12:35, Paul Eggert wrote:
> Aurelien Jarno wrote:
> > I got a look at the emacs git, and found your patch
> > to backport to emacs-24. However as said on the wiki [1], it is also
> > necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> > and it indeed works fine.
> 
> Is that still really needed with the emacs-25 branch? As Andreas notes, it
> has a different way of avoiding the calloc issue. What happens if you build
> the emacs-25 branch without using -ffreestanding?

The emacs-25 branch is probably fine, I was searching the commit to
backport to emacs-24 and was looking for a commit with -ffreestanding.
The emacs-25 branch is indeed fine.

Aurelien
diff mbox

Patch

From da7966b5870fc65897bda5adaa403bd5e5740238 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 19 Jul 2016 15:23:14 +0200
Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64

Inspired by a suggestion by Florian Weimer in:
https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html
* configure.ac (HAVE_PERSONALITY_ADDR_NO_RANDOMIZE):
Rename from HAVE_PERSONALITY_LINUX32, and check for
ADDR_NO_RANDOMIZE (the crucial thing) instead of for LINUX32.
All uses changed.
* src/emacs.c (main) [HAVE_PERSONALITY_ADDR_NO_RANDOMIZE]:
Use ADDR_NO_RANDOMIZE from personality.h rather than inventing the
flag ourselves.  Just set that flag, rather than also setting the
persona.  Do all this earlier, so as to avoid problems with calls
to brk in the interim.  When doing it, avoid functions like
putenv that may allocate memory.  Special case for __PPC64__,
which needs ASLR disabled in dumped Emacs too.
---
 admin/CPP-DEFINES |  2 +-
 configure.ac      | 20 +++++++++++---------
 src/emacs.c       | 53 ++++++++++++++++++++++++++++++-----------------------
 3 files changed, 42 insertions(+), 33 deletions(-)

diff --git a/admin/CPP-DEFINES b/admin/CPP-DEFINES
index 796b57d..d404dee 100644
--- a/admin/CPP-DEFINES
+++ b/admin/CPP-DEFINES
@@ -237,7 +237,7 @@  HAVE_NET_IF_DL_H
 HAVE_NET_IF_H
 HAVE_NLIST_H
 HAVE_OTF_GET_VARIATION_GLYPHS
-HAVE_PERSONALITY_LINUX32
+HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 HAVE_PNG
 HAVE_PNG_H
 HAVE_POSIX_MEMALIGN
diff --git a/configure.ac b/configure.ac
index 678e98e..9da23d1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1615,15 +1615,17 @@  AC_CHECK_HEADERS_ONCE(
   sys/resource.h
   sys/utsname.h pwd.h utmp.h util.h)
 
-AC_MSG_CHECKING(if personality LINUX32 can be set)
-AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/personality.h>]], [[personality (PER_LINUX32)]])],
-               emacs_cv_personality_linux32=yes,
-	       emacs_cv_personality_linux32=no)
-AC_MSG_RESULT($emacs_cv_personality_linux32)
-
-if test $emacs_cv_personality_linux32 = yes; then
-  AC_DEFINE(HAVE_PERSONALITY_LINUX32, 1,
-            [Define to 1 if personality LINUX32 can be set.])
+AC_CACHE_CHECK([if personality ADDR_NO_RANDOMIZE flag exists],
+  [emacs_cv_personality_addr_no_randomize],
+  [AC_COMPILE_IFELSE(
+     [AC_LANG_PROGRAM([[#include <sys/personality.h>]],
+		      [[personality (personality (0xffffffff)
+				     | ADDR_NO_RANDOMIZE)]])],
+     [emacs_cv_personality_addr_no_randomize=yes],
+     [emacs_cv_personality_addr_no_randomize=no])])
+if test $emacs_cv_personality_addr_no_randomize = yes; then
+  AC_DEFINE([HAVE_PERSONALITY_ADDR_NO_RANDOMIZE], [1],
+            [Define to 1 if personality flag ADDR_NO_RANDOMIZE exists.])
 fi
 
 # Note that Solaris has sys/sysinfo.h which defines struct
diff --git a/src/emacs.c b/src/emacs.c
index 5c187e7..b0e5a05 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -106,7 +106,7 @@  extern void moncontrol (int mode);
 #include <sys/resource.h>
 #endif
 
-#ifdef HAVE_PERSONALITY_LINUX32
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
 #include <sys/personality.h>
 #endif
 
@@ -674,6 +674,35 @@  main (int argc, char **argv)
 
   stack_base = &dummy;
 
+  dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
+			     || strcmp (argv[argc - 1], "bootstrap") == 0);
+
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
+
+  /* True if address randomization interferes with memory allocaiton.  */
+# ifdef __PPC64__
+  bool disable_aslr = true;
+# else
+  bool disable_aslr = dumping;
+# endif
+
+  if (disable_aslr)
+    {
+      int pers = personality (0xffffffff);
+      if (! (pers & ADDR_NO_RANDOMIZE)
+	  && 0 <= personality (pers | ADDR_NO_RANDOMIZE))
+	{
+	  /* Address randomization was enabled, but is now disabled.
+	     Re-execute Emacs to get a clean slate.  */
+	  execvp (argv[0], argv);
+
+	  /* If the exec fails, warn the user and then try without a
+	     clean slate.  */
+	  fprintf (stderr, "%s: %s\n", argv[0], strerror (errno));
+	}
+    }
+#endif
+
 #ifndef CANNOT_DUMP
   might_dump = !initialized;
 #endif
@@ -781,28 +810,6 @@  main (int argc, char **argv)
         }
     }
 
-  dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
-			     || strcmp (argv[argc - 1], "bootstrap") == 0);
-
-#ifdef HAVE_PERSONALITY_LINUX32
-  if (dumping && ! getenv ("EMACS_HEAP_EXEC"))
-    {
-      /* Set this so we only do this once.  */
-      xputenv ("EMACS_HEAP_EXEC=true");
-
-      /* A flag to turn off address randomization which is introduced
-         in linux kernel shipped with fedora core 4 */
-#define ADD_NO_RANDOMIZE 0x0040000
-      personality (PER_LINUX32 | ADD_NO_RANDOMIZE);
-#undef  ADD_NO_RANDOMIZE
-
-      execvp (argv[0], argv);
-
-      /* If the exec fails, try to dump anyway.  */
-      emacs_perror (argv[0]);
-    }
-#endif /* HAVE_PERSONALITY_LINUX32 */
-
 #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN)
   /* Extend the stack space available.  Don't do that if dumping,
      since some systems (e.g. DJGPP) might define a smaller stack
-- 
2.5.5