From patchwork Fri Nov 30 03:09:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joseph Myers X-Patchwork-Id: 30412 Received: (qmail 62947 invoked by alias); 30 Nov 2018 03:09:36 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 62933 invoked by uid 89); 30 Nov 2018 03:09:36 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy=h, _Static_assert, _static_assert X-HELO: relay1.mentorg.com Date: Fri, 30 Nov 2018 03:09:25 +0000 From: Joseph Myers To: Subject: Replace gen-as-const.awk by gen-as-const.py Message-ID: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 This patch replaces gen-as-const.awk, and some fragments of the Makefile code that used it, by a Python script. The point is not such much that awk is problematic for this particular script, as that I'd like to build up a general Python infrastructure for extracting information from C headers, for use in writing tests of such headers. Thus, although this patch does not set up such infrastructure, the compute_c_consts function in gen-as-const.py might be moved to a separate Python module in a subsequent patch as a starting point for such infrastructure. The general idea of the code is the same as in the awk version, but no attempt is made to make the output files textually identical. When generating a header, a dict of constant names and values is generated internally then defines are printed in sorted order (rather than the order in the .sym file, which would have been used before). When generating a test that the values computed match those from a normal header inclusion, the test code is made into a compilation test using _Static_assert, where previously the comparisons were done only when the test was executed. One fragment of test generation (converting the previously generated header to use asconst_* prefixes on its macro names) is still in awk code in the makefiles; only the .sym processing and subsequent execution of the compiler to extract constants have moved to the Python script. Tested for x86_64, and with build-many-glibcs.py. 2018-11-30 Joseph Myers * scripts/gen-as-const.py: New file. * scripts/gen-as-const.awk: Remove. * Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use gen-as-const.py. ($(objpfx)test-as-const-%.c): Likewise. diff --git a/Makerules b/Makerules index 07bfe8abcb..8e49a73342 100644 --- a/Makerules +++ b/Makerules @@ -282,15 +282,12 @@ ifdef gen-as-const-headers # may include . Target header files can check if # GEN_AS_CONST_HEADERS is defined to avoid circular dependency which # may lead to build hang on a many-core machine. -$(common-objpfx)%.h $(common-objpfx)%.h.d: $(..)scripts/gen-as-const.awk \ +$(common-objpfx)%.h $(common-objpfx)%.h.d: $(..)scripts/gen-as-const.py \ %.sym $(common-before-compile) - $(AWK) -f $< $(filter %.sym,$^) \ - | $(CC) -S -o $(@:.h.d=.h)T3 $(CFLAGS) $(CPPFLAGS) \ - -DGEN_AS_CONST_HEADERS -x c - \ - -MD -MP -MF $(@:.h=.h.d)T -MT '$(@:.h=.h.d) $(@:.h.d=.h)' - sed -n 's/^.*@@@name@@@\([^@]*\)@@@value@@@[^0-9Xxa-fA-F-]*\([0-9Xxa-fA-F-][0-9Xxa-fA-F-]*\).*@@@end@@@.*$$/#define \1 \2/p' \ - $(@:.h.d=.h)T3 > $(@:.h.d=.h)T - rm -f $(@:.h.d=.h)T3 + $(PYTHON) $< --cc="$(CC) $(CFLAGS) $(CPPFLAGS) -DGEN_AS_CONST_HEADERS \ + -MD -MP -MF $(@:.h=.h.d)T \ + -MT '$(@:.h=.h.d) $(@:.h.d=.h)'" \ + $(filter %.sym,$^) > $(@:.h.d=.h)T sed $(sed-remove-objpfx) $(sed-remove-dotdot) \ $(@:.h=.h.d)T > $(@:.h=.h.d)T2 rm -f $(@:.h=.h.d)T @@ -301,11 +298,10 @@ before-compile += $(gen-as-const-headers:%.sym=$(common-objpfx)%.h) tests-internal += $(gen-as-const-headers:%.sym=test-as-const-%) generated += $(gen-as-const-headers:%.sym=test-as-const-%.c) -$(objpfx)test-as-const-%.c: $(..)scripts/gen-as-const.awk $(..)Makerules \ +$(objpfx)test-as-const-%.c: $(..)scripts/gen-as-const.py $(..)Makerules \ %.sym $(common-objpfx)%.h ($(AWK) '{ sub(/^/, "asconst_", $$2); print; }' $(filter %.h,$^); \ - $(AWK) -v test=1 -f $< $(filter %.sym,$^); \ - echo '#include "$(..)test-skeleton.c"') > $@T + $(PYTHON) $< --test $(filter %.sym,$^)) > $@T mv -f $@T $@ endif diff --git a/scripts/gen-as-const.awk b/scripts/gen-as-const.awk deleted file mode 100644 index 1ffd5f2c1c..0000000000 --- a/scripts/gen-as-const.awk +++ /dev/null @@ -1,63 +0,0 @@ -# Script used in producing headers of assembly constants from C expressions. -# The input to this script looks like: -# #cpp-directive ... -# NAME1 -# NAME2 expression ... -# The output of this script is C code to be run through gcc -S and then -# massaged to extract the integer constant values of the given C expressions. -# A line giving just a name implies an expression consisting of just that name. - -BEGIN { started = 0 } - -# cpp directives go straight through. -/^#/ { print; next } - -NF >= 1 && !started { - if (test) { - print "\n#include "; - print "\n#include "; - print "\n#include "; - print "\n#if __WORDSIZE == 64"; - print "\ntypedef uint64_t c_t;"; - print "\n#define U(n) UINT64_C (n)"; - print "\n#define PRI PRId64"; - print "\n#else"; - print "\ntypedef uint32_t c_t;"; - print "\n#define U(n) UINT32_C (n)"; - print "\n#define PRI PRId32"; - print "\n#endif"; - print "\nstatic int do_test (void)\n{\n int bad = 0, good = 0;\n"; - print "#define TEST(name, source, expr) \\\n" \ - " if (U (asconst_##name) != (c_t) (expr)) { ++bad;" \ - " fprintf (stderr, \"%s: %s is %\" PRI \" but %s is %\"PRI \"\\n\"," \ - " source, #name, U (asconst_##name), #expr, (c_t) (expr));" \ - " } else ++good;\n"; - } - else - print "void dummy(void) {"; - started = 1; -} - -# Separator. -$1 == "--" { next } - -NF == 1 { sub(/^.*$/, "& &"); } - -NF > 1 { - name = $1; - sub(/^[^ ]+[ ]+/, ""); - if (test) - print " TEST (" name ", \"" FILENAME ":" FNR "\", " $0 ")"; - else - printf "asm (\"@@@name@@@%s@@@value@@@%%0@@@end@@@\" : : \"i\" ((long) %s));\n", - name, $0; -} - -END { - if (test) { - print " printf (\"%d errors in %d tests\\n\", bad, good + bad);" - print " return bad != 0 || good == 0;\n}\n"; - print "#define TEST_FUNCTION do_test ()"; - } - else if (started) print "}"; -} diff --git a/scripts/gen-as-const.py b/scripts/gen-as-const.py new file mode 100644 index 0000000000..b7a5744bb1 --- /dev/null +++ b/scripts/gen-as-const.py @@ -0,0 +1,159 @@ +#!/usr/bin/python3 +# Produce headers of assembly constants from C expressions. +# Copyright (C) 2018 Free Software Foundation, Inc. +# This file is part of the GNU C Library. +# +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +# The input to this script looks like: +# #cpp-directive ... +# NAME1 +# NAME2 expression ... +# A line giving just a name implies an expression consisting of just that name. + +import argparse +import os.path +import re +import subprocess +import tempfile + + +def compute_c_consts(sym_data, cc): + """Compute the values of some C constants. + + The first argument is a list whose elements are either strings + (preprocessor directives) or pairs of strings (a name and a C + expression for the corresponding value). Preprocessor directives + in the middle of the list may be used to select which constants + end up being evaluated using which expressions. + + """ + out_lines = [] + started = False + for arg in sym_data: + if isinstance(arg, str): + out_lines.append(arg) + continue + name = arg[0] + value = arg[1] + if not started: + out_lines.append('void\ndummy (void)\n{') + started = True + out_lines.append('asm ("@@@name@@@%s@@@value@@@%%0@@@end@@@" ' + ': : \"i\" ((long int) (%s)));' + % (name, value)) + if started: + out_lines.append('}') + out_lines.append('') + out_text = '\n'.join(out_lines) + with tempfile.TemporaryDirectory() as temp_dir: + c_file_name = os.path.join(temp_dir, 'test.c') + s_file_name = os.path.join(temp_dir, 'test.s') + with open(c_file_name, 'w') as c_file: + c_file.write(out_text) + # Compilation has to be from stdin to avoid the temporary file + # name being written into the generated dependencies. + cmd = ('%s -S -o %s -x c - < %s' % (cc, s_file_name, c_file_name)) + subprocess.check_call(cmd, shell=True) + consts = {} + with open(s_file_name, 'r') as s_file: + for line in s_file: + match = re.search('@@@name@@@([^@]*)' + '@@@value@@@[^0-9Xxa-fA-F-]*' + '([0-9Xxa-fA-F-]+).*@@@end@@@', line) + if match: + if (match.group(1) in consts + and match.group(2) != consts[match.group(1)]): + raise ValueError('duplicate constant %s' + % match.group(1)) + consts[match.group(1)] = match.group(2) + return consts + + +def gen_test(sym_data): + """Generate a test for the values of some C constants. + + The first argument is as for compute_c_consts. + + """ + out_lines = [] + started = False + for arg in sym_data: + if isinstance(arg, str): + out_lines.append(arg) + continue + name = arg[0] + value = arg[1] + if not started: + out_lines.append('#include \n' + '#include \n' + '#include \n' + '#if __WORDSIZE == 64\n' + 'typedef uint64_t c_t;\n' + '# define U(n) UINT64_C (n)\n' + '#else\n' + 'typedef uint32_t c_t;\n' + '# define U(n) UINT32_C (n)\n' + '#endif\n' + 'static int\n' + 'do_test (void)\n' + '{\n' + # Compilation test only, using static assertions. + ' return 0;\n' + '}\n' + '#include ') + started = True + out_lines.append('_Static_assert (U (asconst_%s) == (c_t) (%s), ' + '"value of %s");' + % (name, value, name)) + return '\n'.join(out_lines) + + +def main(): + """The main entry point.""" + parser = argparse.ArgumentParser( + description='Produce headers of assembly constants.') + parser.add_argument('--cc', metavar='CC', + help='C compiler (including options) to use') + parser.add_argument('--test', action='store_true', + help='Generate test case instead of header') + parser.add_argument('sym_file', + help='.sym file to process') + args = parser.parse_args() + sym_data = [] + with open(args.sym_file, 'r') as sym_file: + for line in sym_file: + line = line.strip() + if line == '': + continue + # Pass preprocessor directives through. + if line.startswith('#'): + sym_data.append(line) + continue + words = line.split(maxsplit=1) + # Separator. + if words[0] == '--': + continue + name = words[0] + value = words[1] if len(words) > 1 else words[0] + sym_data.append((name, value)) + if args.test: + print(gen_test(sym_data)) + else: + consts = compute_c_consts(sym_data, args.cc) + print('\n'.join('#define %s %s' % c for c in sorted(consts.items()))) + +if __name__ == '__main__': + main()