| Message ID | 1663a6c7-7c30-c42b-8f2a-aaa98d35388c@cs.ucla.edu |
|---|---|
| State | New, archived |
| Headers | show |
On Tue, Jul 19, 2016 at 8:32 PM, Paul Eggert <eggert@cs.ucla.edu> wrote: > On 07/18/2016 08:49 AM, Sinny Kumari wrote: >> >> Sadly, after applying this patch, build fails for Fedora rawhide on >> both ppc64 and >> ppc64le due to Memory exhausted problem. >> >> koji scratch build link - >> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3540001 >> Direct link to build log of ppc64 - >> http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/2/3540002/build.log > > > Thanks for checking. I thought about it for a bit and came up with the > attached Emacs patch instead, which works for me on x86-64 and x86 Fedora > 23. Can you please try it on ppc64? I assume __PPC64__ is defined, if not > please feel to substitute the proper symbol and please let me know what that > symbol is. Yes, __PPC64__ macro is defined. Applied this patch and emacs builds fine for Fedora rawhide [1] and Fedora 24 [2] for ppc64(le) arches. Ran updated emacs on Fedora rawhide, ppc64 box and it uses few megabytes memory. Checked content of /proc/self/personality inside emacs shell and it has value 00040000. [1] http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3544685 [2] http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3544697
Thanks for checking it. I have pushed a patch into Emacs master which incorporates this idea plus Florian's suggestion about restoring randomization in child processes; please give it a try. Also, I have filed a bug report and proposed patch for the emacs-25 branch here: http://bugs.gnu.org/24033 and will try to convince the Emacs maintainers that the patch is worth installing into emacs-25 even though it's pretty late in emacs-25's release schedule.
On Wed, Jul 20, 2016 at 1:37 PM, Paul Eggert <eggert@cs.ucla.edu> wrote: > Thanks for checking it. I have pushed a patch into Emacs master which > incorporates this idea plus Florian's suggestion about restoring > randomization in child processes; please give it a try. So far, didn't succeeded to test patch pushed in master branch. At first trying to reproduce memory issue by building source tar obtained from commit bf5ddded70c11edaf3514b25da27fc71cfb8e965 and 098d29af10a869e65a36804579f16edc49aabcfa . But didn't see memory issue in those commits. I think, this is because ./configure gives following output: Should Emacs use the GNU version of malloc? no (only before dumping) Should Emacs use a relocating allocator for buffers? no > Also, I have filed a bug report and proposed patch for the emacs-25 branch > here: > > http://bugs.gnu.org/24033 Thanks Paul for proposing it for emacs-25. I tested final patch proposed at http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11 and it works on ppc64(le) architectures. > and will try to convince the Emacs maintainers that the patch is worth > installing into emacs-25 even though it's pretty late in emacs-25's release > schedule. Glad to see that fix will be included in emacs-25.
On 07/21/2016 12:44 PM, Sinny Kumari wrote: > Thanks Paul for proposing it for emacs-25. I tested final patch proposed at > http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11 and it works on > ppc64(le) architectures. Ah, sorry, our emails crossed paths. Thanks for checking, and I'll install the patch into emacs-25.
On 2016-07-21 13:25, Paul Eggert wrote: > On 07/21/2016 12:44 PM, Sinny Kumari wrote: > > Thanks Paul for proposing it for emacs-25. I tested final patch proposed at > > http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11 and it works on > > ppc64(le) architectures. > > Ah, sorry, our emails crossed paths. Thanks for checking, and I'll install > the patch into emacs-25. Thanks for doing so. I got a look at the emacs git, and found your patch to backport to emacs-24. However as said on the wiki [1], it is also necessary to compile src/gmalloc.c with -ffreestanding. I have done that and it indeed works fine. I have found it is necessary to do so starting with GCC 5, however I haven't found any corresponding in the emacs git. Do you have any pointer to an upstream patch? Aurelien [1] https://sourceware.org/glibc/wiki/Release/2.24#Known_Issues
On So, Aug 07 2016, Aurelien Jarno <aurelien@aurel32.net> wrote: > Thanks for doing so. I got a look at the emacs git, and found your patch > to backport to emacs-24. However as said on the wiki [1], it is also > necessary to compile src/gmalloc.c with -ffreestanding. I have done that > and it indeed works fine. I have found it is necessary to do so starting > with GCC 5, however I haven't found any corresponding in the emacs git. > Do you have any pointer to an upstream patch? Do you mean commit 4b1436b? Andreas.
Aurelien Jarno wrote: > I got a look at the emacs git, and found your patch > to backport to emacs-24. However as said on the wiki [1], it is also > necessary to compile src/gmalloc.c with -ffreestanding. I have done that > and it indeed works fine. Is that still really needed with the emacs-25 branch? As Andreas notes, it has a different way of avoiding the calloc issue. What happens if you build the emacs-25 branch without using -ffreestanding?
On 2016-08-07 14:19, Andreas Schwab wrote: > On So, Aug 07 2016, Aurelien Jarno <aurelien@aurel32.net> wrote: > > > Thanks for doing so. I got a look at the emacs git, and found your patch > > to backport to emacs-24. However as said on the wiki [1], it is also > > necessary to compile src/gmalloc.c with -ffreestanding. I have done that > > and it indeed works fine. I have found it is necessary to do so starting > > with GCC 5, however I haven't found any corresponding in the emacs git. > > Do you have any pointer to an upstream patch? > > Do you mean commit 4b1436b? Thanks for the hint, the wiki talked about -ffreestanding, so I was looking for a commit containing it. Aurelien
On 2016-08-07 12:35, Paul Eggert wrote: > Aurelien Jarno wrote: > > I got a look at the emacs git, and found your patch > > to backport to emacs-24. However as said on the wiki [1], it is also > > necessary to compile src/gmalloc.c with -ffreestanding. I have done that > > and it indeed works fine. > > Is that still really needed with the emacs-25 branch? As Andreas notes, it > has a different way of avoiding the calloc issue. What happens if you build > the emacs-25 branch without using -ffreestanding? The emacs-25 branch is probably fine, I was searching the commit to backport to emacs-24 and was looking for a commit with -ffreestanding. The emacs-25 branch is indeed fine. Aurelien
From da7966b5870fc65897bda5adaa403bd5e5740238 Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Tue, 19 Jul 2016 15:23:14 +0200 Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64 Inspired by a suggestion by Florian Weimer in: https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html * configure.ac (HAVE_PERSONALITY_ADDR_NO_RANDOMIZE): Rename from HAVE_PERSONALITY_LINUX32, and check for ADDR_NO_RANDOMIZE (the crucial thing) instead of for LINUX32. All uses changed. * src/emacs.c (main) [HAVE_PERSONALITY_ADDR_NO_RANDOMIZE]: Use ADDR_NO_RANDOMIZE from personality.h rather than inventing the flag ourselves. Just set that flag, rather than also setting the persona. Do all this earlier, so as to avoid problems with calls to brk in the interim. When doing it, avoid functions like putenv that may allocate memory. Special case for __PPC64__, which needs ASLR disabled in dumped Emacs too. --- admin/CPP-DEFINES | 2 +- configure.ac | 20 +++++++++++--------- src/emacs.c | 53 ++++++++++++++++++++++++++++++----------------------- 3 files changed, 42 insertions(+), 33 deletions(-) diff --git a/admin/CPP-DEFINES b/admin/CPP-DEFINES index 796b57d..d404dee 100644 --- a/admin/CPP-DEFINES +++ b/admin/CPP-DEFINES @@ -237,7 +237,7 @@ HAVE_NET_IF_DL_H HAVE_NET_IF_H HAVE_NLIST_H HAVE_OTF_GET_VARIATION_GLYPHS -HAVE_PERSONALITY_LINUX32 +HAVE_PERSONALITY_ADDR_NO_RANDOMIZE HAVE_PNG HAVE_PNG_H HAVE_POSIX_MEMALIGN diff --git a/configure.ac b/configure.ac index 678e98e..9da23d1 100644 --- a/configure.ac +++ b/configure.ac @@ -1615,15 +1615,17 @@ AC_CHECK_HEADERS_ONCE( sys/resource.h sys/utsname.h pwd.h utmp.h util.h) -AC_MSG_CHECKING(if personality LINUX32 can be set) -AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/personality.h>]], [[personality (PER_LINUX32)]])], - emacs_cv_personality_linux32=yes, - emacs_cv_personality_linux32=no) -AC_MSG_RESULT($emacs_cv_personality_linux32) - -if test $emacs_cv_personality_linux32 = yes; then - AC_DEFINE(HAVE_PERSONALITY_LINUX32, 1, - [Define to 1 if personality LINUX32 can be set.]) +AC_CACHE_CHECK([if personality ADDR_NO_RANDOMIZE flag exists], + [emacs_cv_personality_addr_no_randomize], + [AC_COMPILE_IFELSE( + [AC_LANG_PROGRAM([[#include <sys/personality.h>]], + [[personality (personality (0xffffffff) + | ADDR_NO_RANDOMIZE)]])], + [emacs_cv_personality_addr_no_randomize=yes], + [emacs_cv_personality_addr_no_randomize=no])]) +if test $emacs_cv_personality_addr_no_randomize = yes; then + AC_DEFINE([HAVE_PERSONALITY_ADDR_NO_RANDOMIZE], [1], + [Define to 1 if personality flag ADDR_NO_RANDOMIZE exists.]) fi # Note that Solaris has sys/sysinfo.h which defines struct diff --git a/src/emacs.c b/src/emacs.c index 5c187e7..b0e5a05 100644 --- a/src/emacs.c +++ b/src/emacs.c @@ -106,7 +106,7 @@ extern void moncontrol (int mode); #include <sys/resource.h> #endif -#ifdef HAVE_PERSONALITY_LINUX32 +#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE #include <sys/personality.h> #endif @@ -674,6 +674,35 @@ main (int argc, char **argv) stack_base = &dummy; + dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0 + || strcmp (argv[argc - 1], "bootstrap") == 0); + +#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE + + /* True if address randomization interferes with memory allocaiton. */ +# ifdef __PPC64__ + bool disable_aslr = true; +# else + bool disable_aslr = dumping; +# endif + + if (disable_aslr) + { + int pers = personality (0xffffffff); + if (! (pers & ADDR_NO_RANDOMIZE) + && 0 <= personality (pers | ADDR_NO_RANDOMIZE)) + { + /* Address randomization was enabled, but is now disabled. + Re-execute Emacs to get a clean slate. */ + execvp (argv[0], argv); + + /* If the exec fails, warn the user and then try without a + clean slate. */ + fprintf (stderr, "%s: %s\n", argv[0], strerror (errno)); + } + } +#endif + #ifndef CANNOT_DUMP might_dump = !initialized; #endif @@ -781,28 +810,6 @@ main (int argc, char **argv) } } - dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0 - || strcmp (argv[argc - 1], "bootstrap") == 0); - -#ifdef HAVE_PERSONALITY_LINUX32 - if (dumping && ! getenv ("EMACS_HEAP_EXEC")) - { - /* Set this so we only do this once. */ - xputenv ("EMACS_HEAP_EXEC=true"); - - /* A flag to turn off address randomization which is introduced - in linux kernel shipped with fedora core 4 */ -#define ADD_NO_RANDOMIZE 0x0040000 - personality (PER_LINUX32 | ADD_NO_RANDOMIZE); -#undef ADD_NO_RANDOMIZE - - execvp (argv[0], argv); - - /* If the exec fails, try to dump anyway. */ - emacs_perror (argv[0]); - } -#endif /* HAVE_PERSONALITY_LINUX32 */ - #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN) /* Extend the stack space available. Don't do that if dumping, since some systems (e.g. DJGPP) might define a smaller stack -- 2.5.5