From patchwork Tue Sep 11 12:09:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joseph Myers X-Patchwork-Id: 29302 Received: (qmail 55017 invoked by alias); 11 Sep 2018 12:09:25 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 53195 invoked by uid 89); 11 Sep 2018 12:09:25 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-24.5 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy=H*Ad:U*ian X-HELO: relay1.mentorg.com Date: Tue, 11 Sep 2018 12:09:15 +0000 From: Joseph Myers To: CC: , , , , Subject: Add build-many-glibcs.py support for building more GCC libraries [committed] Message-ID: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Every so often we get libsanitizer or libgo builds breaking with new glibc because of some change in the glibc headers. glibc's build-many-glibcs.py deliberately disables libsanitizer and GCC languages other than C and C++ because the point is to test glibc and find glibc problems (including problems shown up by new compiler warnings in new GCC), not to test libsanitizer or libgo; if the compiler build fails because of libsanitizer or libgo failing to build, that could hide the existence of new problems in glibc. However, it seems reasonable to have a non-default mode where build-many-glibcs.py does build those additional pieces, which this patch adds. Note that I do not intend to run a build-many-glibcs.py bot with this new option. If people concerned with libsanitizer, libgo or other potentially affected GCC libraries wish to find out about such problems more quickly, they may wish to run such a bot or bots (and to monitor the results and fix issues found - obviously there will be some overlap with issues found by my bots not using that option). I can provide instructions on setting up a bot if desired. Note also that building a non-native Ada compiler requires a sufficiently recent native (or build-x-host, in general) Ada compiler to be used, possibly more or less the same version as being built. That needs to be in the PATH when build-many-glibcs.py --full-gcc is run; the script does not deal with setting up such a compiler (or any of the other host tools needed for building GCC and glibc, beyond the GMP / MPFR / MPC libraries), but perhaps it should, to avoid the need to keep updating such a compiler manually when running a bot. Tested by running build-many-glibcs.py with the new option, with mainline GCC. There are build failures for various configurations, which may be of interest to Go / Ada people even if you're not interested in running such a bot: * mips64 / mips64el (all configuration): ICE building libstdc++, as seen without using the new option . * aarch64_be: error building libgo (little-endian aarch64 works fine): version.go:67:13: error: expected ';' or ')' or newline 67 | BigEndian = | ^ version.go:67:3: error: reference to undefined name 'BigEndian' 67 | BigEndian = | ^ * arm (all configurations): error building libgo: /scratch/jmyers/glibc/many9/src/gcc/libgo/go/internal/syscall/unix/getrandom_linux.go:29:5: error: reference to undefined name 'randomTrap' 29 | if randomTrap == 0 { | ^ /scratch/jmyers/glibc/many9/src/gcc/libgo/go/internal/syscall/unix/getrandom_linux.go:38:34: error: reference to undefined name 'randomTrap' 38 | r1, _, errno := syscall.Syscall(randomTrap, | ^ What's happening there is, I think, that the arm*b*-*-* case in libgo/configure.ac is wrongly matching arm-glibc-linux-gnueabi with the 'b' in the vendor part, and then something else is failing to handle GOARCH=armbe. Given that you can have configurations with multilibs of both endiannesses, endianness should always be detected by configure.ac, for all architectures, using a compile test of whether __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__, not based on textual matches to the host (= target at top-level) triplet. * armeb (all configurations): error building libada (for some reason the Arm libada configuration seems to do different things for EH for big-endian, which makes no sense to me and doesn't actually work): a-exexpr.adb:87:06: "System.Exceptions.Machine" is not a predefined library unit a-exexpr.adb:87:06: "Ada.Exceptions (body)" depends on "Ada.Exceptions.Exception_Propagation (body)" a-exexpr.adb:87:06: "Ada.Exceptions.Exception_Propagation (body)" depends on "System.Exceptions.Machine (spec)" * hppa: error building libgo (same error as for aarch64_be). * ia64: ICE building libgo. I've filed for this. * m68k: ICE in the Go front end building libgo . * microblaze, microblazeel, nios2, sh3, sh3eb: build failure in libada for lack of a libada port to those systems (I'm not sure sh3 would actually need anything different from sh4): a-cbdlli.ads:38:14: violation of restriction "No_Finalization" at system.ads:47 * i686-gnu: build failure in libada, might be fixed by the patch attached to (not tested): terminals.c:1115:13: fatal error: termio.h: No such file or directory Committed. 2018-09-11 Joseph Myers * scripts/build-many-glibcs.py (Context.__init__): Add full_gcc argument. (Config.build_gcc): Use --disable-libsanitizer for first GCC build, but not for second build if --full-gcc. Use --enable-languages=all for second build if --full-gcc. (get_parser): Add --full-gcc option. (main): Update call to Context. diff --git a/scripts/build-many-glibcs.py b/scripts/build-many-glibcs.py index a7c2b67bce..61f98651c6 100755 --- a/scripts/build-many-glibcs.py +++ b/scripts/build-many-glibcs.py @@ -91,13 +91,14 @@ class Context(object): """The global state associated with builds in a given directory.""" def __init__(self, topdir, parallelism, keep, replace_sources, strip, - action): + full_gcc, action): """Initialize the context.""" self.topdir = topdir self.parallelism = parallelism self.keep = keep self.replace_sources = replace_sources self.strip = strip + self.full_gcc = full_gcc self.srcdir = os.path.join(topdir, 'src') self.versions_json = os.path.join(self.srcdir, 'versions.json') self.build_state_json = os.path.join(topdir, 'build-state.json') @@ -1122,6 +1123,8 @@ class Context(object): """Run a copy of this script with given options.""" cmd = [sys.executable, sys.argv[0], '--keep=none', '-j%d' % self.parallelism] + if self.full_gcc: + cmd.append('--full-gcc') cmd.extend(opts) cmd.extend([self.topdir, action]) sys.stdout.flush() @@ -1328,15 +1331,13 @@ class Config(object): def build_gcc(self, cmdlist, bootstrap): """Build GCC.""" - # libsanitizer commonly breaks because of glibc header - # changes, or on unusual targets. libssp is of little - # relevance with glibc's own stack checking support. - # libcilkrts does not support GNU/Hurd (and has been removed - # in GCC 8, so --disable-libcilkrts can be removed once glibc - # no longer supports building with older GCC versions). + # libssp is of little relevance with glibc's own stack + # checking support. libcilkrts does not support GNU/Hurd (and + # has been removed in GCC 8, so --disable-libcilkrts can be + # removed once glibc no longer supports building with older + # GCC versions). cfg_opts = list(self.gcc_cfg) - cfg_opts += ['--disable-libsanitizer', '--disable-libssp', - '--disable-libcilkrts'] + cfg_opts += ['--disable-libssp', '--disable-libcilkrts'] host_libs = self.ctx.host_libraries_installdir cfg_opts += ['--with-gmp=%s' % host_libs, '--with-mpfr=%s' % host_libs, @@ -1359,14 +1360,20 @@ class Config(object): '--disable-libitm', '--disable-libmpx', '--disable-libquadmath', + '--disable-libsanitizer', '--without-headers', '--with-newlib', '--with-glibc-version=%s' % self.ctx.glibc_version ] cfg_opts += self.first_gcc_cfg else: tool_build = 'gcc' - cfg_opts += ['--enable-languages=c,c++', '--enable-shared', - '--enable-threads'] + # libsanitizer commonly breaks because of glibc header + # changes, or on unusual targets. + if not self.ctx.full_gcc: + cfg_opts += ['--disable-libsanitizer'] + langs = 'all' if self.ctx.full_gcc else 'c,c++' + cfg_opts += ['--enable-languages=%s' % langs, + '--enable-shared', '--enable-threads'] self.build_cross_tool(cmdlist, 'gcc', tool_build, cfg_opts) @@ -1658,6 +1665,8 @@ def get_parser(): 'with the wrong version of a component') parser.add_argument('--strip', action='store_true', help='Strip installed glibc libraries') + parser.add_argument('--full-gcc', action='store_true', + help='Build GCC with all languages and libsanitizer') parser.add_argument('topdir', help='Toplevel working directory') parser.add_argument('action', @@ -1676,7 +1685,7 @@ def main(argv): opts = parser.parse_args(argv) topdir = os.path.abspath(opts.topdir) ctx = Context(topdir, opts.parallelism, opts.keep, opts.replace_sources, - opts.strip, opts.action) + opts.strip, opts.full_gcc, opts.action) ctx.run_builds(opts.action, opts.configs)