From: Kyrylo Tkachov <ktkachov@nvidia.com>
The generated emit, recog, and attrtab sources have different shapes. One
partition count therefore does not give each generator the same compile
critical path. Tamar Christina suggested a colon-separated configuration
value so that each generator can have an independent count.
The existing --with-insnemit-partitions name refers only to genemit, but the
option now controls three machine-description generators. Add
--with-md-generator-partitions as the canonical name. Given
--with-insnemit-partitions was in previously-released GCC, do we need to
keep it? This patch keeps it as an alias, but I can remove it if needed.
Accept either one count for all three generators or E:R:A for separate
genemit, genrecog, and genattrtab counts. Let config.gcc set the target
default in either form. Keep 10 as the generic default and let an explicit
configure option take precedence. Reject malformed values and counts outside
the 1 to 9999 range supported by one_to_9999.
Make each generator stamp depend on Makefile. This regenerates the source
set after reconfigure changes a count, including when the new set has files
that did not exist before.
Stop when any emit or recog output move fails. The previous foreach command
reported only the status of the last move. Remove all numbered generator
sources during mostlyclean, including files left after a partition-count
decrease.
I think we want each target to have the option of setting the defaults
as the optimal split is ultimately a function of the complexity and
"shape" of the target MD files. Set the AArch64 default to 16:16:8, for example.
Bootstrapped and tested on aarch64-none-linux-gnu and x86_64-linux.
Ok for trunk?
Thanks,
Kyrill
gcc/ChangeLog:
* Makefile.in (NUM_INSNRECOG_SPLITS): New variable.
(INSNRECOG_SPLITS_SEQ): Use it.
(NUM_INSNATTRTAB_SPLITS): New variable.
(INSNATTRTAB_SPLITS_SEQ): Use it.
(MOSTLYCLEANFILES): Remove all numbered generator sources.
(s-tmp-emit, s-tmp-recog): Stop when an output move fails.
(s-tmp-emit, s-tmp-recog, s-attrtab): Depend on Makefile.
* config.gcc (default_md_generator_partitions): New variable.
Document its scalar and tuple forms.
(aarch64*-*-*): Set the target default.
* configure.ac (--with-md-generator-partitions): New option.
(--with-insnemit-partitions): Keep as a deprecated alias.
Use the target default when no option is specified. Validate and
substitute all three counts.
* configure: Regenerate.
* doc/install.texi (--with-md-generator-partitions): Document.
Suggested-by: Tamar Christina <tamar.christina@arm.com>
Signed-off-by: Kyrylo Tkachov <ktkachov@nvidia.com>
---
gcc/Makefile.in | 37 ++++++++++--------
gcc/config.gcc | 8 ++++
gcc/configure | 92 ++++++++++++++++++++++++++++++++++++++------
gcc/configure.ac | 82 +++++++++++++++++++++++++++++++++++----
gcc/doc/install.texi | 11 ++++++
5 files changed, 194 insertions(+), 36 deletions(-)
@@ -235,14 +235,16 @@ INSNEMIT_SEQ_SRC = $(patsubst %, insn-emit-%.cc, $(INSNEMIT_SPLITS_SEQ))
INSNEMIT_SEQ_TMP = $(patsubst %, tmp-emit-%.cc, $(INSNEMIT_SPLITS_SEQ))
INSNEMIT_SEQ_O = $(patsubst %, insn-emit-%.o, $(INSNEMIT_SPLITS_SEQ))
-# Re-use the split number for insn-recog as well.
-INSNRECOG_SPLITS_SEQ = $(wordlist 1,$(NUM_INSNEMIT_SPLITS),$(one_to_9999))
+# The number of splits to be made for the insn-recog files.
+NUM_INSNRECOG_SPLITS = @DEFAULT_INSNRECOG_PARTITIONS@
+INSNRECOG_SPLITS_SEQ = $(wordlist 1,$(NUM_INSNRECOG_SPLITS),$(one_to_9999))
INSNRECOG_SEQ_SRC = $(patsubst %, insn-recog-%.cc, $(INSNRECOG_SPLITS_SEQ))
INSNRECOG_SEQ_TMP = $(patsubst %, tmp-recog-%.cc, $(INSNRECOG_SPLITS_SEQ))
INSNRECOG_SEQ_O = $(patsubst %, insn-recog-%.o, $(INSNRECOG_SPLITS_SEQ))
-# Re-use the split number for insn-attrtab as well.
-INSNATTRTAB_SPLITS_SEQ = $(wordlist 1,$(NUM_INSNEMIT_SPLITS),$(one_to_9999))
+# The number of splits to be made for the insn-attrtab files.
+NUM_INSNATTRTAB_SPLITS = @DEFAULT_INSNATTRTAB_PARTITIONS@
+INSNATTRTAB_SPLITS_SEQ = $(wordlist 1,$(NUM_INSNATTRTAB_SPLITS),$(one_to_9999))
INSNATTRTAB_SEQ_SRC = $(patsubst %, insn-attrtab-%.cc, \
$(INSNATTRTAB_SPLITS_SEQ))
INSNATTRTAB_SEQ_TMP = $(patsubst %, tmp-attrtab-%.cc, \
@@ -1992,9 +1994,9 @@ TREECHECKING = @TREECHECKING@
FULL_DRIVER_NAME=$(target_noncanonical)-gcc-$(version)$(exeext)
MOSTLYCLEANFILES = insn-flags.h insn-config.h insn-codes.h \
- insn-output.cc $(INSNRECOG_SEQ_SRC) insn-recog.h \
- $(INSNEMIT_SEQ_SRC) insn-extract.cc insn-peep.cc \
- insn-attr.h insn-attr-common.h $(INSNATTRTAB_SEQ_SRC) insn-attrtab.cc \
+ insn-output.cc insn-recog-*.cc insn-recog.h \
+ insn-emit-*.cc insn-extract.cc insn-peep.cc \
+ insn-attr.h insn-attr-common.h insn-attrtab-*.cc insn-attrtab.cc \
insn-dfatab.cc insn-latencytab.cc insn-opinit.cc \
insn-opinit.h insn-preds.cc \
insn-constants.h \
@@ -2831,24 +2833,27 @@ $(simple_generated_c:insn-%.cc=s-%): s-%: build/gen%$(build_exeext)
# genemit splits its output into different files and doesn't write to
# stdout. (but rather to tmp-emit-01.cc..tmp-emit-10.cc)
$(INSNEMIT_SEQ_SRC): s-tmp-emit; @true
-s-tmp-emit: build/genemit$(build_exeext) $(MD_DEPS) insn-conditions.md
+s-tmp-emit: build/genemit$(build_exeext) $(MD_DEPS) insn-conditions.md Makefile
$(RUN_GEN) build/genemit$(build_exeext) $(md_file) insn-conditions.md \
$(addprefix -O,${INSNEMIT_SEQ_TMP})
- $(foreach id, $(INSNEMIT_SPLITS_SEQ), \
- $(SHELL) $(srcdir)/../move-if-change tmp-emit-$(id).cc \
- insn-emit-$(id).cc;)
+ for id in $(INSNEMIT_SPLITS_SEQ); do \
+ $(SHELL) $(srcdir)/../move-if-change tmp-emit-$$id.cc \
+ insn-emit-$$id.cc || exit 1; \
+ done
$(STAMP) s-tmp-emit
# Same for genrecog.
$(INSNRECOG_SEQ_SRC): s-tmp-recog; @true
insn-recog.h: s-tmp-recog; @true
-s-tmp-recog: build/genrecog$(build_exeext) $(MD_DEPS) insn-conditions.md
+s-tmp-recog: build/genrecog$(build_exeext) $(MD_DEPS) \
+ insn-conditions.md Makefile
$(RUN_GEN) build/genrecog$(build_exeext) $(md_file) insn-conditions.md \
-Hinsn-recog.h \
$(addprefix -O,${INSNRECOG_SEQ_TMP})
- $(foreach id, $(INSNRECOG_SPLITS_SEQ), \
- $(SHELL) $(srcdir)/../move-if-change tmp-recog-$(id).cc \
- insn-recog-$(id).cc;)
+ for id in $(INSNRECOG_SPLITS_SEQ); do \
+ $(SHELL) $(srcdir)/../move-if-change tmp-recog-$$id.cc \
+ insn-recog-$$id.cc || exit 1; \
+ done
$(STAMP) s-tmp-recog
# gencheck doesn't read the machine description, and the file produced
@@ -2865,7 +2870,7 @@ s-check : build/gencheck$(build_exeext)
$(INSNATTRTAB_SEQ_SRC): s-attrtab ; @true
insn-dfatab.cc insn-latencytab.cc: s-attrtab ; @true
s-attrtab : $(MD_DEPS) build/genattrtab$(build_exeext) \
- insn-conditions.md
+ insn-conditions.md Makefile
$(RUN_GEN) build/genattrtab$(build_exeext) $(md_file) insn-conditions.md \
$(addprefix -A,${INSNATTRTAB_SEQ_TMP}) \
-Dtmp-dfatab.cc -Ltmp-latencytab.cc
@@ -54,6 +54,12 @@
# support, the default value of
# $default_gnu_indirect_function is set to
# "no" except for targets which are known to be OK.
+#
+# default_md_generator_partitions
+# The default partition counts for genemit, genrecog,
+# and genattrtab. Set this to N to use the same count
+# for all three generators, or to E:R:A to set each
+# count separately.
# This file sets the following shell variables for use by the
# autoconf-generated configure script:
@@ -240,6 +246,7 @@ gdb_needs_out_file_path=
thread_file=
default_use_cxa_atexit=no
default_gnu_indirect_function=no
+default_md_generator_partitions=10
target_gtfiles=
need_64bit_isa=
native_system_header_dir=/usr/include
@@ -335,6 +342,7 @@ cpu_is_64bit=
case ${target} in
aarch64*-*-*)
cpu_type=aarch64
+ default_md_generator_partitions=16:16:8
extra_headers="${extra_headers} arm_fp16.h"
extra_headers="${extra_headers} arm_neon.h"
extra_headers="${extra_headers} arm_bf16.h"
@@ -859,6 +859,8 @@ enable_gcov
enable_shared
enable_fixed_point
enable_decimal_float
+DEFAULT_INSNATTRTAB_PARTITIONS
+DEFAULT_INSNRECOG_PARTITIONS
DEFAULT_INSNEMIT_PARTITIONS
DEFAULT_MATCHPD_PARTITIONS
with_multi_buildlist
@@ -987,6 +989,7 @@ enable_multilib
enable_multiarch
with_stack_clash_protection_guard_size
with_matchpd_partitions
+with_md_generator_partitions
with_insnemit_partitions
enable___cxa_atexit
enable_decimal_float
@@ -1881,9 +1884,12 @@ Optional Packages:
--with-matchpd-partitions=num
Set the number of partitions to make for gimple and
generic when splitting match.pd. [default=10]
- --with-insnemit-partitions=num
- Set the number of partitions used by genemit,
- genrecog and genattrtab. [default=10]
+ --with-md-generator-partitions=num|emit:recog:attrtab
+ Set the partition counts used by genemit, genrecog
+ and genattrtab. A single number sets all three
+ counts. [default=target dependent]
+ --with-insnemit-partitions=num|emit:recog:attrtab
+ Deprecated alias for --with-md-generator-partitions.
--with-dwarf2 force the default debug format to be DWARF 2 (or
later)
--with-specs=SPECS add SPECS to driver command-line processing
@@ -7908,21 +7914,33 @@ fi
# Specify the number of output partitions used by the machine-description
-# generators.
+# generators. Select and validate the effective value after config.gcc has
+# supplied the target default.
+md_generator_partitions_set=no
+md_generator_partitions_legacy=no
+
+# Check whether --with-md-generator-partitions was given.
+if test "${with_md_generator_partitions+set}" = set; then :
+ withval=$with_md_generator_partitions; md_generator_partitions=$with_md_generator_partitions
+md_generator_partitions_set=yes
+fi
+
# Check whether --with-insnemit-partitions was given.
if test "${with_insnemit_partitions+set}" = set; then :
- withval=$with_insnemit_partitions; DEFAULT_INSNEMIT_PARTITIONS="$with_insnemit_partitions"
-else
- DEFAULT_INSNEMIT_PARTITIONS=10
+ withval=$with_insnemit_partitions; if test "$md_generator_partitions_set" = yes; then
+ as_fn_error $? "--with-md-generator-partitions and --with-insnemit-partitions cannot be used together." "$LINENO" 5
fi
-
-if (test $DEFAULT_INSNEMIT_PARTITIONS -lt 1); then
- as_fn_error $? "Invalid value $DEFAULT_INSNEMIT_PARTITIONS for --with-insnemit-partitions. Cannot be negative." "$LINENO" 5
+md_generator_partitions=$with_insnemit_partitions
+md_generator_partitions_set=yes
+md_generator_partitions_legacy=yes
fi
+
+
+
# Enable __cxa_atexit for C++.
# Check whether --enable-__cxa_atexit was given.
if test "${enable___cxa_atexit+set}" = set; then :
@@ -12995,6 +13013,56 @@ target_gtfiles=
# Collect target-machine-specific information.
. ${srcdir}/config.gcc || exit 1
+# Use the target default unless the user specified an option.
+if test "$md_generator_partitions_set" = no; then
+ md_generator_partitions=$default_md_generator_partitions
+fi
+
+case "$md_generator_partitions" in
+ '' | *[!0-9:]* | :* | *: | *:*:*:*)
+ DEFAULT_INSNATTRTAB_PARTITIONS=
+ ;;
+ *:*:*)
+ IFS=: read -r DEFAULT_INSNEMIT_PARTITIONS DEFAULT_INSNRECOG_PARTITIONS \
+ DEFAULT_INSNATTRTAB_PARTITIONS <<EOF
+$md_generator_partitions
+EOF
+ ;;
+ *:*)
+ DEFAULT_INSNATTRTAB_PARTITIONS=
+ ;;
+ *)
+ DEFAULT_INSNEMIT_PARTITIONS=$md_generator_partitions
+ DEFAULT_INSNRECOG_PARTITIONS=$md_generator_partitions
+ DEFAULT_INSNATTRTAB_PARTITIONS=$md_generator_partitions
+ ;;
+esac
+
+for generator_partitions in \
+ "$DEFAULT_INSNEMIT_PARTITIONS" \
+ "$DEFAULT_INSNRECOG_PARTITIONS" \
+ "$DEFAULT_INSNATTRTAB_PARTITIONS"; do
+ case "$generator_partitions" in
+ '' | *[!0-9]*) generator_partitions= ;;
+ *)
+ while test "${generator_partitions#0}" != "$generator_partitions"; do
+ generator_partitions=${generator_partitions#0}
+ done
+ case "$generator_partitions" in
+ '' | ?????*) generator_partitions= ;;
+ esac
+ ;;
+ esac
+ if test -z "$generator_partitions"; then
+ as_fn_error $? "Invalid machine-description generator partition value $md_generator_partitions. Use N or E:R:A, with each number between 1 and 9999." "$LINENO" 5
+ fi
+done
+
+if test "$md_generator_partitions_legacy" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: --with-insnemit-partitions is deprecated. Use --with-md-generator-partitions instead." >&5
+$as_echo "$as_me: WARNING: --with-insnemit-partitions is deprecated. Use --with-md-generator-partitions instead." >&2;}
+fi
+
extra_objs="${host_extra_objs} ${extra_objs}"
extra_gcc_objs="${host_extra_gcc_objs} ${extra_gcc_objs}"
@@ -21922,7 +21990,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
-#line 21925 "configure"
+#line 21993 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -22028,7 +22096,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
-#line 22031 "configure"
+#line 22099 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -903,19 +903,31 @@ fi
AC_SUBST(DEFAULT_MATCHPD_PARTITIONS)
# Specify the number of output partitions used by the machine-description
-# generators.
+# generators. Select and validate the effective value after config.gcc has
+# supplied the target default.
+md_generator_partitions_set=no
+md_generator_partitions_legacy=no
+AC_ARG_WITH(md-generator-partitions,
+[AS_HELP_STRING([--with-md-generator-partitions=num|emit:recog:attrtab],
+[Set the partition counts used by genemit, genrecog and genattrtab.
+A single number sets all three counts. [default=target dependent]])],
+[md_generator_partitions=$with_md_generator_partitions
+md_generator_partitions_set=yes])
AC_ARG_WITH(insnemit-partitions,
-[AS_HELP_STRING([--with-insnemit-partitions=num],
-[Set the number of partitions used by genemit, genrecog and genattrtab.
-[default=10]])],
-[DEFAULT_INSNEMIT_PARTITIONS="$with_insnemit_partitions"], [DEFAULT_INSNEMIT_PARTITIONS=10])
-if (test $DEFAULT_INSNEMIT_PARTITIONS -lt 1); then
+[AS_HELP_STRING([--with-insnemit-partitions=num|emit:recog:attrtab],
+[Deprecated alias for --with-md-generator-partitions.])],
+[if test "$md_generator_partitions_set" = yes; then
AC_MSG_ERROR(m4_normalize([
- Invalid value $DEFAULT_INSNEMIT_PARTITIONS for --with-insnemit-partitions. \
- Cannot be negative.]))
+ --with-md-generator-partitions and --with-insnemit-partitions \
+ cannot be used together.]))
fi
+md_generator_partitions=$with_insnemit_partitions
+md_generator_partitions_set=yes
+md_generator_partitions_legacy=yes])
AC_SUBST(DEFAULT_INSNEMIT_PARTITIONS)
+AC_SUBST(DEFAULT_INSNRECOG_PARTITIONS)
+AC_SUBST(DEFAULT_INSNATTRTAB_PARTITIONS)
# Enable __cxa_atexit for C++.
AC_ARG_ENABLE(__cxa_atexit,
@@ -1928,6 +1940,60 @@ target_gtfiles=
# Collect target-machine-specific information.
. ${srcdir}/config.gcc || exit 1
+# Use the target default unless the user specified an option.
+if test "$md_generator_partitions_set" = no; then
+ md_generator_partitions=$default_md_generator_partitions
+fi
+
+case "$md_generator_partitions" in
+ '' | *[[!0-9:]]* | :* | *: | *:*:*:*)
+ DEFAULT_INSNATTRTAB_PARTITIONS=
+ ;;
+ *:*:*)
+ IFS=: read -r DEFAULT_INSNEMIT_PARTITIONS DEFAULT_INSNRECOG_PARTITIONS \
+ DEFAULT_INSNATTRTAB_PARTITIONS <<EOF
+$md_generator_partitions
+EOF
+ ;;
+ *:*)
+ DEFAULT_INSNATTRTAB_PARTITIONS=
+ ;;
+ *)
+ DEFAULT_INSNEMIT_PARTITIONS=$md_generator_partitions
+ DEFAULT_INSNRECOG_PARTITIONS=$md_generator_partitions
+ DEFAULT_INSNATTRTAB_PARTITIONS=$md_generator_partitions
+ ;;
+esac
+
+for generator_partitions in \
+ "$DEFAULT_INSNEMIT_PARTITIONS" \
+ "$DEFAULT_INSNRECOG_PARTITIONS" \
+ "$DEFAULT_INSNATTRTAB_PARTITIONS"; do
+ case "$generator_partitions" in
+ '' | *[[!0-9]]*) generator_partitions= ;;
+ *)
+ while test "${generator_partitions#0}" != "$generator_partitions"; do
+ generator_partitions=${generator_partitions#0}
+ done
+ case "$generator_partitions" in
+ '' | ?????*) generator_partitions= ;;
+ esac
+ ;;
+ esac
+ if test -z "$generator_partitions"; then
+ AC_MSG_ERROR(m4_normalize([
+ Invalid machine-description generator partition value \
+ $md_generator_partitions. \
+ Use N or E:R:A, with each number between 1 and 9999.]))
+ fi
+done
+
+if test "$md_generator_partitions_legacy" = yes; then
+ AC_MSG_WARN(m4_normalize([
+ --with-insnemit-partitions is deprecated. \
+ Use --with-md-generator-partitions instead.]))
+fi
+
extra_objs="${host_extra_objs} ${extra_objs}"
extra_gcc_objs="${host_extra_gcc_objs} ${extra_gcc_objs}"
@@ -1967,6 +1967,17 @@ dependencies and thus with parallel make potentially link different
compilers concurrently. If the argument is a positive integer, allow
that number of concurrent link processes for the large binaries.
+@item --with-md-generator-partitions=@var{num}
+@itemx --with-md-generator-partitions=@var{counts}
+Set the number of C++ source partitions produced by genemit, genrecog,
+and genattrtab. A single @var{num} sets all three counts. The
+@var{counts} form is
+@code{@var{emit}:@var{recog}:@var{attrtab}}. Each count must be between
+1 and 9999. The default is @code{16:16:8} for AArch64 and 10 for each
+generator on other targets. A target can set a different default in
+@file{config.gcc}. @option{--with-insnemit-partitions} is a deprecated
+alias.
+
@item --enable-maintainer-mode
The build rules that regenerate the Autoconf and Automake output files as
well as the GCC master message catalog @file{gcc.pot} are normally