benchtests: Build benchmarks in parallel
Checks
| Context |
Check |
Description |
| redhat-pt-bot/TryBot-apply_patch |
success
|
Patch applied to master at the time it was sent
|
| redhat-pt-bot/TryBot-32bit |
success
|
Build for i686
|
| linaro-tcwg-bot/tcwg_glibc_build--master-arm |
success
|
Build passed
|
| linaro-tcwg-bot/tcwg_glibc_build--master-aarch64 |
success
|
Build passed
|
| linaro-tcwg-bot/tcwg_glibc_check--master-aarch64 |
success
|
Test passed
|
| linaro-tcwg-bot/tcwg_glibc_check--master-arm |
success
|
Test passed
|
Commit Message
The unconditional '.NOTPARALLEL' in benchtests/Makefile forced the whole
subdirectory to build serially, even though its only purpose is to keep
the benchmark *runs* from perturbing each other's timing.
Replace it with ordering that serializes only the benchmark runs, and
only when more than one benchmark group will actually run. The combined
'bench' goal builds every benchmark program in parallel (through
bench-build) and then runs the bench-set, bench-func and bench-malloc
groups strictly one after another.
---
benchtests/Makefile | 27 ++++++++++++++++++++++++---
1 file changed, 24 insertions(+), 3 deletions(-)
Comments
Adhemerval Zanella <adhemerval.zanella@linaro.org> writes:
> The unconditional '.NOTPARALLEL' in benchtests/Makefile forced the whole
> subdirectory to build serially, even though its only purpose is to keep
> the benchmark *runs* from perturbing each other's timing.
>
> Replace it with ordering that serializes only the benchmark runs, and
> only when more than one benchmark group will actually run. The combined
> 'bench' goal builds every benchmark program in parallel (through
> bench-build) and then runs the bench-set, bench-func and bench-malloc
> groups strictly one after another.
> ---
> benchtests/Makefile | 27 ++++++++++++++++++++++++---
> 1 file changed, 24 insertions(+), 3 deletions(-)
>
> diff --git a/benchtests/Makefile b/benchtests/Makefile
> index 53f78232ffe..f407e492cb7 100644
> --- a/benchtests/Makefile
> +++ b/benchtests/Makefile
> @@ -409,9 +409,9 @@ $(addprefix $(objpfx)bench-,calloc-thread): $(libm-benchtests)
> # Rules to build and execute the benchmarks. Do not put any benchmark
> # parameters beyond this point.
>
> -# We don't want the benchmark programs to run in parallel since that could
> -# affect their performance.
> -.NOTPARALLEL:
> +# Benchmark programs must not run concurrently, however building them is
> +# safe in parallel. So '.NOTPARALLEL:' is used only when benchmarks run
> +# or when more than one group is going to run.
>
> bench-extra-objs = json-lib.o
>
> @@ -627,3 +627,24 @@ $(objpfx)bench-%.c: %-inputs $(bench-deps)
> fi; \
> $(PYTHON) scripts/bench.py $(patsubst %-inputs,%,$<); } > $@-tmp
> mv -f $@-tmp $@
> +
> +# Serialize the benchmark runs so their timing is not perturbed by a
> +# concurrent workload, while still building every benchmark program in parallel.
> +# Ordering is only needed when more than one benchmark group run: the combined
> +# 'bench' goal runs all three groups, and the bench* goals can also be combined
> +# on a single command line. A lone group needs no ordering -- make builds its
> +# programs in parallel and then runs its single recipe, which cannot overlap a
> +# compile.
> +ifneq (,$(filter bench,$(MAKECMDGOALS)))
> +bench-run-active := bench-set bench-func bench-malloc
> +else
> +bench-run-active := $(filter bench-set bench-func bench-malloc,$(MAKECMDGOALS))
> +endif
> +
> +ifneq (,$(word 2,$(bench-run-active)))
> +# Hold every active run until all benchmark programs have been built.
> +$(bench-run-active): | bench-build
> +# And then run the active groups strictly one at a time.
> +bench-func: | $(filter bench-set,$(bench-run-active))
> +bench-malloc: | $(filter bench-set bench-func,$(bench-run-active))
> +endif
Reviewed-by: Sam James <sam@gentoo.org>
@@ -409,9 +409,9 @@ $(addprefix $(objpfx)bench-,calloc-thread): $(libm-benchtests)
# Rules to build and execute the benchmarks. Do not put any benchmark
# parameters beyond this point.
-# We don't want the benchmark programs to run in parallel since that could
-# affect their performance.
-.NOTPARALLEL:
+# Benchmark programs must not run concurrently, however building them is
+# safe in parallel. So '.NOTPARALLEL:' is used only when benchmarks run
+# or when more than one group is going to run.
bench-extra-objs = json-lib.o
@@ -627,3 +627,24 @@ $(objpfx)bench-%.c: %-inputs $(bench-deps)
fi; \
$(PYTHON) scripts/bench.py $(patsubst %-inputs,%,$<); } > $@-tmp
mv -f $@-tmp $@
+
+# Serialize the benchmark runs so their timing is not perturbed by a
+# concurrent workload, while still building every benchmark program in parallel.
+# Ordering is only needed when more than one benchmark group run: the combined
+# 'bench' goal runs all three groups, and the bench* goals can also be combined
+# on a single command line. A lone group needs no ordering -- make builds its
+# programs in parallel and then runs its single recipe, which cannot overlap a
+# compile.
+ifneq (,$(filter bench,$(MAKECMDGOALS)))
+bench-run-active := bench-set bench-func bench-malloc
+else
+bench-run-active := $(filter bench-set bench-func bench-malloc,$(MAKECMDGOALS))
+endif
+
+ifneq (,$(word 2,$(bench-run-active)))
+# Hold every active run until all benchmark programs have been built.
+$(bench-run-active): | bench-build
+# And then run the active groups strictly one at a time.
+bench-func: | $(filter bench-set,$(bench-run-active))
+bench-malloc: | $(filter bench-set bench-func,$(bench-run-active))
+endif