"make check" times

  While digging for some low-hanging fruit in "make" times, I did this:


That one instance of "head" is called over 6000 times per "make
check", and as it's the only non-builtin in that script, it adds about
11 seconds of overhead compared to just reading that one line with
builtins.

My question here is: how much of a time savings is worth the
complexity of said savings?
  

On 7/19/19 6:11 PM, DJ Delorie wrote:
> While digging for some low-hanging fruit in "make" times, I did this:
> 
> diff --git a/scripts/merge-test-results.sh b/scripts/merge-test-results.sh
> index 919bbae253..7088ef6996 100755
> --- a/scripts/merge-test-results.sh
> +++ b/scripts/merge-test-results.sh
> @@ -35,7 +35,11 @@ case $type in
>       subdir=${subdir:+$subdir/}
>       for t in "$@"; do
>         if [ -s "$objpfx$t.test-result" ]; then
> -	head -n1 "$objpfx$t.test-result"
> +	  #head -n1 "$objpfx$t.test-result"
> +	  exec 6<"$objpfx$t.test-result"
> +	  read line <&6
> +	  echo $line
> +	  exec 6<&-
>         else
>   	echo "UNRESOLVED: $subdir$t"
>         fi
> 
> That one instance of "head" is called over 6000 times per "make
> check", and as it's the only non-builtin in that script, it adds about
> 11 seconds of overhead compared to just reading that one line with
> builtins.
> 
> My question here is: how much of a time savings is worth the
> complexity of said savings?

One needs to tackle the problem the other way around.

* First determine those things we cannot change and remove them
   form the accounting.
   - We cannot avoid compiling each file.

* Whatever is left over after removing the things we can't
   avoid doing is our "overhead".

* Sort the "overhead" by the amount of time spent, and find ways
   to reduce it.

If the "overhead" is small, then there isn't much value in adding
complexity.

If the "overhead" is large, we may want to rewrite the whole build
system to get that time back.

Does that make sense?

I can't answer your question without knowing how much is 11 seconds
out of the total time taken.

If the build took 20 seconds to complete, then yes this is a valuable
change ;-)
  

On Mär 11 2020, DJ Delorie via Libc-alpha wrote:

>> diff --git a/scripts/merge-test-results.sh b/scripts/merge-test-results.sh
>> index 919bbae253..7088ef6996 100755
>> --- a/scripts/merge-test-results.sh
>> +++ b/scripts/merge-test-results.sh
>> @@ -35,7 +35,11 @@ case $type in
>>      subdir=${subdir:+$subdir/}
>>      for t in "$@"; do
>>        if [ -s "$objpfx$t.test-result" ]; then
>> -	head -n1 "$objpfx$t.test-result"
>> +	  #head -n1 "$objpfx$t.test-result"
>> +	  exec 6<"$objpfx$t.test-result"
>> +	  read line <&6
>> +	  echo $line
>> +	  exec 6<&-

Why do you need to use the exec dance?

Andreas.