glibc 2.24 -- Release blockers
Commit Message
On 07/18/2016 08:49 AM, Sinny Kumari wrote:
> Sadly, after applying this patch, build fails for Fedora rawhide on
> both ppc64 and
> ppc64le due to Memory exhausted problem.
>
> koji scratch build link -
> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3540001
> Direct link to build log of ppc64 -
> http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/2/3540002/build.log
Thanks for checking. I thought about it for a bit and came up with the
attached Emacs patch instead, which works for me on x86-64 and x86
Fedora 23. Can you please try it on ppc64? I assume __PPC64__ is
defined, if not please feel to substitute the proper symbol and please
let me know what that symbol is.
This patch applies to Emacs's emacs-25 branch; if it works I plan to
port it to Emacs's master branch. The patch does not implement Florian's
idea of re-enabling ASLR before Emacs execs a child process, as I want
to keep the emacs-25 patch as simple as possible. However, I do plan to
try Florian's idea in the master branch.
Comments
On Tue, Jul 19, 2016 at 8:32 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 07/18/2016 08:49 AM, Sinny Kumari wrote:
>>
>> Sadly, after applying this patch, build fails for Fedora rawhide on
>> both ppc64 and
>> ppc64le due to Memory exhausted problem.
>>
>> koji scratch build link -
>> http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3540001
>> Direct link to build log of ppc64 -
>> http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/2/3540002/build.log
>
>
> Thanks for checking. I thought about it for a bit and came up with the
> attached Emacs patch instead, which works for me on x86-64 and x86 Fedora
> 23. Can you please try it on ppc64? I assume __PPC64__ is defined, if not
> please feel to substitute the proper symbol and please let me know what that
> symbol is.
Yes, __PPC64__ macro is defined.
Applied this patch and emacs builds fine for Fedora rawhide [1] and
Fedora 24 [2] for
ppc64(le) arches.
Ran updated emacs on Fedora rawhide, ppc64 box and it uses few megabytes memory.
Checked content of /proc/self/personality inside emacs shell and it has value
00040000.
[1] http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3544685
[2] http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=3544697
Thanks for checking it. I have pushed a patch into Emacs master which
incorporates this idea plus Florian's suggestion about restoring
randomization in child processes; please give it a try.
Also, I have filed a bug report and proposed patch for the emacs-25
branch here:
http://bugs.gnu.org/24033
and will try to convince the Emacs maintainers that the patch is worth
installing into emacs-25 even though it's pretty late in emacs-25's
release schedule.
On Wed, Jul 20, 2016 at 1:37 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> Thanks for checking it. I have pushed a patch into Emacs master which
> incorporates this idea plus Florian's suggestion about restoring
> randomization in child processes; please give it a try.
So far, didn't succeeded to test patch pushed in master branch.
At first trying to reproduce memory issue by building source tar obtained
from commit bf5ddded70c11edaf3514b25da27fc71cfb8e965 and
098d29af10a869e65a36804579f16edc49aabcfa . But didn't see memory
issue in those commits.
I think, this is because ./configure gives following output:
Should Emacs use the GNU version of malloc? no (only
before dumping)
Should Emacs use a relocating allocator for buffers? no
> Also, I have filed a bug report and proposed patch for the emacs-25 branch
> here:
>
> http://bugs.gnu.org/24033
Thanks Paul for proposing it for emacs-25. I tested final patch proposed at
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11 and it works on
ppc64(le) architectures.
> and will try to convince the Emacs maintainers that the patch is worth
> installing into emacs-25 even though it's pretty late in emacs-25's release
> schedule.
Glad to see that fix will be included in emacs-25.
On 07/21/2016 12:44 PM, Sinny Kumari wrote:
> Thanks Paul for proposing it for emacs-25. I tested final patch proposed at
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11 and it works on
> ppc64(le) architectures.
Ah, sorry, our emails crossed paths. Thanks for checking, and I'll
install the patch into emacs-25.
On 2016-07-21 13:25, Paul Eggert wrote:
> On 07/21/2016 12:44 PM, Sinny Kumari wrote:
> > Thanks Paul for proposing it for emacs-25. I tested final patch proposed at
> > http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24033#11 and it works on
> > ppc64(le) architectures.
>
> Ah, sorry, our emails crossed paths. Thanks for checking, and I'll install
> the patch into emacs-25.
Thanks for doing so. I got a look at the emacs git, and found your patch
to backport to emacs-24. However as said on the wiki [1], it is also
necessary to compile src/gmalloc.c with -ffreestanding. I have done that
and it indeed works fine. I have found it is necessary to do so starting
with GCC 5, however I haven't found any corresponding in the emacs git.
Do you have any pointer to an upstream patch?
Aurelien
[1] https://sourceware.org/glibc/wiki/Release/2.24#Known_Issues
On So, Aug 07 2016, Aurelien Jarno <aurelien@aurel32.net> wrote:
> Thanks for doing so. I got a look at the emacs git, and found your patch
> to backport to emacs-24. However as said on the wiki [1], it is also
> necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> and it indeed works fine. I have found it is necessary to do so starting
> with GCC 5, however I haven't found any corresponding in the emacs git.
> Do you have any pointer to an upstream patch?
Do you mean commit 4b1436b?
Andreas.
Aurelien Jarno wrote:
> I got a look at the emacs git, and found your patch
> to backport to emacs-24. However as said on the wiki [1], it is also
> necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> and it indeed works fine.
Is that still really needed with the emacs-25 branch? As Andreas notes, it has a
different way of avoiding the calloc issue. What happens if you build the
emacs-25 branch without using -ffreestanding?
On 2016-08-07 14:19, Andreas Schwab wrote:
> On So, Aug 07 2016, Aurelien Jarno <aurelien@aurel32.net> wrote:
>
> > Thanks for doing so. I got a look at the emacs git, and found your patch
> > to backport to emacs-24. However as said on the wiki [1], it is also
> > necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> > and it indeed works fine. I have found it is necessary to do so starting
> > with GCC 5, however I haven't found any corresponding in the emacs git.
> > Do you have any pointer to an upstream patch?
>
> Do you mean commit 4b1436b?
Thanks for the hint, the wiki talked about -ffreestanding, so I was
looking for a commit containing it.
Aurelien
On 2016-08-07 12:35, Paul Eggert wrote:
> Aurelien Jarno wrote:
> > I got a look at the emacs git, and found your patch
> > to backport to emacs-24. However as said on the wiki [1], it is also
> > necessary to compile src/gmalloc.c with -ffreestanding. I have done that
> > and it indeed works fine.
>
> Is that still really needed with the emacs-25 branch? As Andreas notes, it
> has a different way of avoiding the calloc issue. What happens if you build
> the emacs-25 branch without using -ffreestanding?
The emacs-25 branch is probably fine, I was searching the commit to
backport to emacs-24 and was looking for a commit with -ffreestanding.
The emacs-25 branch is indeed fine.
Aurelien
From da7966b5870fc65897bda5adaa403bd5e5740238 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 19 Jul 2016 15:23:14 +0200
Subject: [PATCH] Port to glibc 2.24 (pre-release) + ppc64
Inspired by a suggestion by Florian Weimer in:
https://sourceware.org/ml/libc-alpha/2016-07/msg00425.html
* configure.ac (HAVE_PERSONALITY_ADDR_NO_RANDOMIZE):
Rename from HAVE_PERSONALITY_LINUX32, and check for
ADDR_NO_RANDOMIZE (the crucial thing) instead of for LINUX32.
All uses changed.
* src/emacs.c (main) [HAVE_PERSONALITY_ADDR_NO_RANDOMIZE]:
Use ADDR_NO_RANDOMIZE from personality.h rather than inventing the
flag ourselves. Just set that flag, rather than also setting the
persona. Do all this earlier, so as to avoid problems with calls
to brk in the interim. When doing it, avoid functions like
putenv that may allocate memory. Special case for __PPC64__,
which needs ASLR disabled in dumped Emacs too.
---
admin/CPP-DEFINES | 2 +-
configure.ac | 20 +++++++++++---------
src/emacs.c | 53 ++++++++++++++++++++++++++++++-----------------------
3 files changed, 42 insertions(+), 33 deletions(-)
@@ -237,7 +237,7 @@ HAVE_NET_IF_DL_H
HAVE_NET_IF_H
HAVE_NLIST_H
HAVE_OTF_GET_VARIATION_GLYPHS
-HAVE_PERSONALITY_LINUX32
+HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
HAVE_PNG
HAVE_PNG_H
HAVE_POSIX_MEMALIGN
@@ -1615,15 +1615,17 @@ AC_CHECK_HEADERS_ONCE(
sys/resource.h
sys/utsname.h pwd.h utmp.h util.h)
-AC_MSG_CHECKING(if personality LINUX32 can be set)
-AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/personality.h>]], [[personality (PER_LINUX32)]])],
- emacs_cv_personality_linux32=yes,
- emacs_cv_personality_linux32=no)
-AC_MSG_RESULT($emacs_cv_personality_linux32)
-
-if test $emacs_cv_personality_linux32 = yes; then
- AC_DEFINE(HAVE_PERSONALITY_LINUX32, 1,
- [Define to 1 if personality LINUX32 can be set.])
+AC_CACHE_CHECK([if personality ADDR_NO_RANDOMIZE flag exists],
+ [emacs_cv_personality_addr_no_randomize],
+ [AC_COMPILE_IFELSE(
+ [AC_LANG_PROGRAM([[#include <sys/personality.h>]],
+ [[personality (personality (0xffffffff)
+ | ADDR_NO_RANDOMIZE)]])],
+ [emacs_cv_personality_addr_no_randomize=yes],
+ [emacs_cv_personality_addr_no_randomize=no])])
+if test $emacs_cv_personality_addr_no_randomize = yes; then
+ AC_DEFINE([HAVE_PERSONALITY_ADDR_NO_RANDOMIZE], [1],
+ [Define to 1 if personality flag ADDR_NO_RANDOMIZE exists.])
fi
# Note that Solaris has sys/sysinfo.h which defines struct
@@ -106,7 +106,7 @@ extern void moncontrol (int mode);
#include <sys/resource.h>
#endif
-#ifdef HAVE_PERSONALITY_LINUX32
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
#include <sys/personality.h>
#endif
@@ -674,6 +674,35 @@ main (int argc, char **argv)
stack_base = &dummy;
+ dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
+ || strcmp (argv[argc - 1], "bootstrap") == 0);
+
+#ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE
+
+ /* True if address randomization interferes with memory allocaiton. */
+# ifdef __PPC64__
+ bool disable_aslr = true;
+# else
+ bool disable_aslr = dumping;
+# endif
+
+ if (disable_aslr)
+ {
+ int pers = personality (0xffffffff);
+ if (! (pers & ADDR_NO_RANDOMIZE)
+ && 0 <= personality (pers | ADDR_NO_RANDOMIZE))
+ {
+ /* Address randomization was enabled, but is now disabled.
+ Re-execute Emacs to get a clean slate. */
+ execvp (argv[0], argv);
+
+ /* If the exec fails, warn the user and then try without a
+ clean slate. */
+ fprintf (stderr, "%s: %s\n", argv[0], strerror (errno));
+ }
+ }
+#endif
+
#ifndef CANNOT_DUMP
might_dump = !initialized;
#endif
@@ -781,28 +810,6 @@ main (int argc, char **argv)
}
}
- dumping = !initialized && (strcmp (argv[argc - 1], "dump") == 0
- || strcmp (argv[argc - 1], "bootstrap") == 0);
-
-#ifdef HAVE_PERSONALITY_LINUX32
- if (dumping && ! getenv ("EMACS_HEAP_EXEC"))
- {
- /* Set this so we only do this once. */
- xputenv ("EMACS_HEAP_EXEC=true");
-
- /* A flag to turn off address randomization which is introduced
- in linux kernel shipped with fedora core 4 */
-#define ADD_NO_RANDOMIZE 0x0040000
- personality (PER_LINUX32 | ADD_NO_RANDOMIZE);
-#undef ADD_NO_RANDOMIZE
-
- execvp (argv[0], argv);
-
- /* If the exec fails, try to dump anyway. */
- emacs_perror (argv[0]);
- }
-#endif /* HAVE_PERSONALITY_LINUX32 */
-
#if defined (HAVE_SETRLIMIT) && defined (RLIMIT_STACK) && !defined (CYGWIN)
/* Extend the stack space available. Don't do that if dumping,
since some systems (e.g. DJGPP) might define a smaller stack
--
2.5.5