From patchwork Thu Aug 9 20:26:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joseph Myers X-Patchwork-Id: 28819 Received: (qmail 48449 invoked by alias); 9 Aug 2018 20:26:41 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 48431 invoked by uid 89); 9 Aug 2018 20:26:41 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-24.5 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_PASS, URIBL_RED autolearn=ham version=3.3.2 spammy=moveifchange, move-if-change, pwd, cvs X-HELO: relay1.mentorg.com Date: Thu, 9 Aug 2018 20:26:31 +0000 From: Joseph Myers To: Subject: Use gen-libm-test.py to generate ulps table for manual Message-ID: User-Agent: Alpine 2.20 (DEB 67 2015-01-07) MIME-Version: 1.0 This patch, relative to a tree with (pending review) applied, extends gen-libm-test.py to generate the ulps table for the manual, so meaning there is only a single ulps file parser needed and another Perl script is eliminated. As with the introduction of gen-libm-test.py, this is designed to generate exactly the same libm-err.texi as libm-err-tab.pl did. (gen-libm-test.py is still shorter in lines than the old gen-libm-test.pl even after this patch.) Note that this introduces a Python dependency for building the manual, which is thus noted in install.texi and NEWS. Tested building html / info / pdf versions of the manual. 2018-08-09 Joseph Myers * math/gen-libm-test.py: Import os. (ALL_FLOATS_MANUAL): New constant. (ALL_FLOATS_SUFFIX): Likewise. (Ulps.all_functions): New function. (real_all_ulps): Likewise. (generate_err_table_sub): Likewise. (generate_err_table): Likewise. (main): Handle -s and -m options. * manual/libm-err-tab.pl: Remove. * manual/Makefile ($(objpfx)stamp-libm-err): Use gen-libm-test.py instead of libm-err-tab.pl. [$(PERL) != no]: Change condition to [$(if $(PYTHON),$(PERL),no) != no]. * manual/install.texi (Tools for Compilation): Document requirement for Python to build manual. * INSTALL: Regenerated. diff --git a/INSTALL b/INSTALL index 603b23b..518ed58 100644 --- a/INSTALL +++ b/INSTALL @@ -514,8 +514,9 @@ build the GNU C Library: * Python 2.7/3.4 or later - Python is required to run some tests. As of release time, Python - 3.6 is the newest verified to work for testing the GNU C Library. + Python is required to build the GNU C Library manual and to run + some tests. As of release time, Python 3.6 is the newest verified + to work for testing the GNU C Library. * PExpect 4.0 diff --git a/NEWS b/NEWS index 7868c84..b208404 100644 --- a/NEWS +++ b/NEWS @@ -23,7 +23,7 @@ Deprecated and removed features, and other changes affecting compatibility: Changes to build and runtime requirements: - [Add changes to build and runtime requirements here] +* Python is required to build the GNU C Library manual. Security related changes: diff --git a/manual/Makefile b/manual/Makefile index c275664..9c35c9d 100644 --- a/manual/Makefile +++ b/manual/Makefile @@ -99,10 +99,11 @@ $(objpfx)dir-add.texi: xtract-typefun.awk $(texis-path) # The table with the math errors is generated. $(objpfx)libm-err.texi: $(objpfx)stamp-libm-err -$(objpfx)stamp-libm-err: libm-err-tab.pl $(wildcard $(foreach dir,$(sysdirs),\ - $(dir)/libm-test-ulps)) +$(objpfx)stamp-libm-err: $(..)math/gen-libm-test.py \ + $(wildcard $(foreach dir,$(sysdirs),\ + $(dir)/libm-test-ulps)) pwd=`pwd`; \ - $(PERL) $< $$pwd/.. > $(objpfx)libm-err-tmp + $(PYTHON) $< -s $$pwd/.. -m $(objpfx)libm-err-tmp $(move-if-change) $(objpfx)libm-err-tmp $(objpfx)libm-err.texi touch $@ @@ -169,8 +170,9 @@ include ../Rules .PHONY: install subdir_install install-data install-data subdir_install: install -# Generated files requiring perl: libm-err.texi, summary.texi -ifneq ($(PERL),no) +# Generated files requiring python: libm-err.texi +# Generated files requiring perl: summary.texi +ifneq ($(if $(PYTHON),$(PERL),no),no) ifneq ($(strip $(MAKEINFO)),:) install: $(inst_infodir)/libc.info @if $(SHELL) -c '$(INSTALL_INFO) --version' >/dev/null 2>&1; then \ diff --git a/manual/install.texi b/manual/install.texi index df32b39..5372bc4 100644 --- a/manual/install.texi +++ b/manual/install.texi @@ -560,8 +560,9 @@ with any version of @code{sed}. As of release time, @code{sed} version @item Python 2.7/3.4 or later -Python is required to run some tests. As of release time, Python 3.6 -is the newest verified to work for testing @theglibc{}. +Python is required to build the @glibcadj{} manual and to run some +tests. As of release time, Python 3.6 is the newest verified to work +for testing @theglibc{}. @item PExpect 4.0 diff --git a/manual/libm-err-tab.pl b/manual/libm-err-tab.pl deleted file mode 100755 index 768d2da..0000000 --- a/manual/libm-err-tab.pl +++ /dev/null @@ -1,206 +0,0 @@ -#!/usr/bin/perl -w -# Copyright (C) 1999-2018 Free Software Foundation, Inc. -# This file is part of the GNU C Library. -# Contributed by Andreas Jaeger , 1999. - -# The GNU C Library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. - -# The GNU C Library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. - -# You should have received a copy of the GNU Lesser General Public -# License along with the GNU C Library; if not, see -# . - -# Information about tests are stored in: %results -# $results{$test}{"type"} is the result type, e.g. normal or complex. -# In the following description $platform, $type and $float are: -# - $platform is the used platform -# - $type is either "normal", "real" (for the real part of a complex number) -# or "imag" (for the imaginary part # of a complex number). -# - $float is either of float, ifloat, double, idouble, ldouble, ildouble; -# It represents the underlying floating point type (float, double or long -# double) and if inline functions (the leading i stands for inline) -# are used. -# $results{$test}{$platform}{$type}{$float} is defined and has a delta -# or 'fail' as value. - -use File::Find; - -use strict; - -use vars qw ($sources @platforms %pplatforms); -use vars qw (%results @all_floats %suffices %all_functions); - - -# all_floats is in output order and contains all recognised float types that -# we're going to output -@all_floats = ('float', 'double', 'ldouble', 'float128'); -%suffices = - ( 'float' => 'f', - 'double' => '', - 'ldouble' => 'l', - 'float128' => 'f128' - ); - -# Pretty description of platform -%pplatforms = (); - -%all_functions = (); - -if ($#ARGV == 0) { - $sources = $ARGV[0]; -} else { - $sources = '/usr/src/cvs/libc'; -} - -find (\&find_files, $sources); - -@platforms = sort by_platforms @platforms; - -&print_all; - -sub find_files { - if ($_ eq 'libm-test-ulps') { - # print "Parsing $File::Find::name\n"; - push @platforms, $File::Find::dir; - my ($file, $name); - $file = "${File::Find::name}-name"; - open NAME, $file or die ("Can't open $file: $!"); - $name = ; - chomp $name; - close NAME; - $pplatforms{$File::Find::dir} = $name; - &parse_ulps ($File::Find::name, $File::Find::dir); - } -} - -# Parse ulps file -sub parse_ulps { - my ($file, $platform) = @_; - my ($test, $type, $float, $eps, $ignore_fn); - - # $type has the following values: - # "normal": No complex variable - # "real": Real part of complex result - # "imag": Imaginary part of complex result - open ULP, $file or die ("Can't open $file: $!"); - while () { - chop; - # ignore comments and empty lines - next if /^#/; - next if /^\s*$/; - if (/^Function: /) { - if (/Real part of/) { - s/Real part of //; - $type = 'real'; - } elsif (/Imaginary part of/) { - s/Imaginary part of //; - $type = 'imag'; - } else { - $type = 'normal'; - } - ($test) = ($_ =~ /^Function:\s*\"([a-zA-Z0-9_]+)\"/); - next; - } - if ($test =~ /_(downward|towardzero|upward|vlen)/) { - $ignore_fn = 1; - } else { - $ignore_fn = 0; - $all_functions{$test} = 1; - } - if (/^i?(float|double|ldouble|float128):/) { - ($float, $eps) = split /\s*:\s*/,$_,2; - if ($ignore_fn) { - next; - } elsif ($eps eq 'fail') { - $results{$test}{$platform}{$type}{$float} = 'fail'; - } elsif ($eps eq "0") { - # ignore - next; - } elsif (!exists $results{$test}{$platform}{$type}{$float} - || $results{$test}{$platform}{$type}{$float} ne 'fail') { - $results{$test}{$platform}{$type}{$float} = $eps; - } - if ($type =~ /^real|imag$/) { - $results{$test}{'type'} = 'complex'; - } elsif ($type eq 'normal') { - $results{$test}{'type'} = 'normal'; - } - next; - } - print "Skipping unknown entry: `$_'\n"; - } - close ULP; -} - -sub get_value { - my ($fct, $platform, $type, $float) = @_; - - return (exists $results{$fct}{$platform}{$type}{$float} - ? $results{$fct}{$platform}{$type}{$float} : "0"); -} - -sub print_platforms { - my (@p) = @_; - my ($fct, $platform, $float, $first, $i, $platform_no, $platform_total); - - print '@multitable {nexttowardf} '; - foreach (@p) { - print ' {1000 + i 1000}'; - } - print "\n"; - - print '@item Function '; - foreach (@p) { - print ' @tab '; - print $pplatforms{$_}; - } - print "\n"; - - - foreach $fct (sort keys %all_functions) { - foreach $float (@all_floats) { - print "\@item $fct$suffices{$float} "; - foreach $platform (@p) { - print ' @tab '; - if (exists $results{$fct}{$platform}{'normal'}{$float} - || exists $results{$fct}{$platform}{'real'}{$float} - || exists $results{$fct}{$platform}{'imag'}{$float}) { - if ($results{$fct}{'type'} eq 'complex') { - print &get_value ($fct, $platform, 'real', $float), - ' + i ', &get_value ($fct, $platform, 'imag', $float); - } else { - print $results{$fct}{$platform}{'normal'}{$float}; - } - } else { - print '-'; - } - } - print "\n"; - } - } - - print "\@end multitable\n"; -} - -sub print_all { - my ($i, $max); - - my ($columns) = 5; - - # Print only 5 platforms at a time. - for ($i=0; $i < $#platforms; $i+=$columns) { - $max = $i+$columns-1 > $#platforms ? $#platforms : $i+$columns-1; - print_platforms (@platforms[$i .. $max]); - } -} - -sub by_platforms { - return $pplatforms{$a} cmp $pplatforms{$b}; -} diff --git a/math/gen-libm-test.py b/math/gen-libm-test.py index 8d156ea..0b25a90 100755 --- a/math/gen-libm-test.py +++ b/math/gen-libm-test.py @@ -19,6 +19,7 @@ import argparse from collections import defaultdict +import os import re @@ -32,6 +33,16 @@ ALL_FLOATS_PFX = {'double': 'DBL', 'float': 'FLT', 'float128': 'FLT128'} +# Float types in the order used in the generated ulps tables in the +# manual. +ALL_FLOATS_MANUAL = ('float', 'double', 'ldouble', 'float128') + +# Map float types in ulps files to C function suffix. +ALL_FLOATS_SUFFIX = {'double': '', + 'ldouble': 'l', + 'float': 'f', + 'float128': 'f128'} + # Number of arguments in structure (as opposed to arguments that are # pointers to return values) for an argument descriptor. DESCR_NUM_ARGS = {'f': 1, 'a': 1, 'j': 1, 'i': 1, 'u': 1, 'l': 1, 'L': 1, @@ -161,6 +172,17 @@ class Ulps(object): ulps_dict[ulps_fn][line_first], ulps_val) + def all_functions(self): + """Return the set of functions with ulps and whether they are + complex.""" + funcs = set() + complex = {} + for k_prefix, k_dict in self.ulps_kinds: + for f in k_dict: + funcs.add(f) + complex[f] = True if k_prefix else False + return funcs, complex + def write(self, ulps_file): """Write ulps back out as a sorted ulps file.""" # Output is sorted first by function name, then by (real, @@ -231,6 +253,18 @@ class Ulps(object): f.write(header_text) +def read_all_ulps(srcdir): + """Read all platforms' libm-test-ulps files.""" + all_ulps = {} + for dirpath, dirnames, filenames in os.walk(srcdir): + if 'libm-test-ulps' in filenames: + with open(os.path.join(dirpath, 'libm-test-ulps-name')) as f: + name = f.read().rstrip() + all_ulps[name] = Ulps() + all_ulps[name].read(os.path.join(dirpath, 'libm-test-ulps')) + return all_ulps + + def read_auto_tests(test_file): """Read tests from auto-libm-test-out- (possibly None).""" auto_tests = defaultdict(lambda: defaultdict(dict)) @@ -571,6 +605,60 @@ def generate_testfile(inc_input, auto_tests, c_output): f.write(''.join(test_list)) +def generate_err_table_sub(all_ulps, all_functions, fns_complex, platforms): + """Generate a single table within the overall ulps table section.""" + plat_width = [' {1000 + i 1000}' for p in platforms] + plat_header = [' @tab %s' % p for p in platforms] + table_list = ['@multitable {nexttowardf} %s\n' % ''.join(plat_width), + '@item Function %s\n' % ''.join(plat_header)] + for func in all_functions: + for flt in ALL_FLOATS_MANUAL: + func_ulps = [] + for p in platforms: + p_ulps = all_ulps[p] + if fns_complex[func]: + ulp_real = p_ulps.real[func][flt] + ulp_imag = p_ulps.imag[func][flt] + ulp_str = '%d + i %d' % (ulp_real, ulp_imag) + ulp_str = ulp_str if ulp_real or ulp_imag else '-' + else: + ulp = p_ulps.normal[func][flt] + ulp_str = str(ulp) if ulp else '-' + func_ulps.append(ulp_str) + table_list.append('@item %s%s @tab %s\n' + % (func, ALL_FLOATS_SUFFIX[flt], + ' @tab '.join(func_ulps))) + table_list.append('@end multitable\n') + return ''.join(table_list) + + +def generate_err_table(all_ulps, err_table): + """Generate ulps table for manual.""" + all_platforms = sorted(all_ulps.keys()) + functions_set = set() + functions_complex = {} + for p in all_platforms: + p_functions, p_complex = all_ulps[p].all_functions() + functions_set.update(p_functions) + functions_complex.update(p_complex) + all_functions = sorted([f for f in functions_set + if ('_downward' not in f + and '_towardzero' not in f + and '_upward' not in f + and '_vlen' not in f)]) + err_table_list = [] + # Print five platforms at a time. + num_platforms = len(all_platforms) + for i in range((num_platforms + 4) // 5): + start = i * 5 + end = i * 5 + 5 if num_platforms >= i * 5 + 5 else num_platforms + err_table_list.append(generate_err_table_sub(all_ulps, all_functions, + functions_complex, + all_platforms[start:end])) + with open(err_table, 'w') as f: + f.write(''.join(err_table_list)) + + def main(): """The main entry point.""" parser = argparse.ArgumentParser(description='Generate libm tests.') @@ -580,23 +668,31 @@ def main(): help='input file .inc file with tests') parser.add_argument('-u', dest='ulps_file', metavar='FILE', help='input file with ulps') + parser.add_argument('-s', dest='srcdir', metavar='DIR', + help='input source directory with all ulps') parser.add_argument('-n', dest='ulps_output', metavar='FILE', help='generate sorted ulps file FILE') parser.add_argument('-C', dest='c_output', metavar='FILE', help='generate output C file FILE from .inc file') parser.add_argument('-H', dest='ulps_header', metavar='FILE', help='generate output ulps header FILE') + parser.add_argument('-m', dest='err_table', metavar='FILE', + help='generate output ulps table for manual FILE') args = parser.parse_args() ulps = Ulps() if args.ulps_file is not None: ulps.read(args.ulps_file) auto_tests = read_auto_tests(args.auto_input) + if args.srcdir is not None: + all_ulps = read_all_ulps(args.srcdir) if args.ulps_output is not None: ulps.write(args.ulps_output) if args.ulps_header is not None: ulps.write_header(args.ulps_header) if args.c_output is not None: generate_testfile(args.inc_input, auto_tests, args.c_output) + if args.err_table is not None: + generate_err_table(all_ulps, args.err_table) if __name__ == '__main__':