[V2] benchtests: Expand range of tests names in schema.json

  When executing bench-math the benchmark output is invalid with this
error msg:

Invalid benchmark output: 'workload-spec2006.wrf' does not match any of
the regexes: '^[_a-zA-Z0-9]*$¹ or Invalid benchmark output: Additional
properties are not allowed ('workload-spec2006.wrf' was unexpected)

This patch changes regex in benchout.schema.json to accept symbols in
benchmark tests names. This patch came after error running
workload-spec2006.wrf, that detects an error in validation script, which
does not expected the hyphen and the dot. Other commond symbols like: "," and
"=" are added to prevent future errors.

	(VERSION): Set to 2.27
        * benchtests/scripts/benchout.schema.json: Fix regex to accept a
        wider range of tests names.

Signed-off-by: Victor Rodriguez <victor.rodriguez.bahena@intel.com>
---
 ChangeLog                               | 6 ++++++
 benchtests/scripts/benchout.schema.json | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)
  

On Tuesday 21 November 2017 11:34 PM, Victor Rodriguez wrote:
> When executing bench-math the benchmark output is invalid with this
> error msg:
> 
> Invalid benchmark output: 'workload-spec2006.wrf' does not match any of
> the regexes: '^[_a-zA-Z0-9]*$¹ or Invalid benchmark output: Additional
> properties are not allowed ('workload-spec2006.wrf' was unexpected)
> 
> This patch changes regex in benchout.schema.json to accept symbols in
> benchmark tests names. This patch came after error running
> workload-spec2006.wrf, that detects an error in validation script, which
> does not expected the hyphen and the dot. Other commond symbols like: "," and
> "=" are added to prevent future errors.

Lets not add , and = since they're typically not part of names.  OK with
the regex only adding a hyphen and dot.

Thanks,
Siddhesh

> 
> 	(VERSION): Set to 2.27
>         * benchtests/scripts/benchout.schema.json: Fix regex to accept a
>         wider range of tests names.
> 
> Signed-off-by: Victor Rodriguez <victor.rodriguez.bahena@intel.com>
> ---
>  ChangeLog                               | 6 ++++++
>  benchtests/scripts/benchout.schema.json | 2 +-
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/ChangeLog b/ChangeLog
> index 8dbfc7e..f29d180 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -1,3 +1,9 @@
> +2017-11-15  Victor Rodriguez  <victor.rodriguez.bahena@intel.com>
> +
> +	(VERSION): Set to 2.27
> +	* benchtests/scripts/benchout.schema.json: Fix regex to accept a wider range
> +	of tests names.
> +
>  2017-08-02  Siddhesh Poyarekar  <siddhesh@sourceware.org>
>  
>  	* version.h (RELEASE): Set to "stable"
> diff --git a/benchtests/scripts/benchout.schema.json b/benchtests/scripts/benchout.schema.json
> index affb7c1..815ad97 100644
> --- a/benchtests/scripts/benchout.schema.json
> +++ b/benchtests/scripts/benchout.schema.json
> @@ -13,7 +13,7 @@
>            "title": "Function names",
>            "type": "object",
>            "patternProperties": {
> -            "^[_a-zA-Z0-9]*$": {
> +            "^[_a-zA-Z0-9,=.-]*$": {
>                "title": "Function variants",
>                "type": "object",
>                "properties": {
>
  

-----Original Message-----
From: Siddhesh Poyarekar <siddhesh@gotplt.org>

Date: Wednesday, November 22, 2017 at 4:18 AM
To: Victor Rodriguez Bahena <victor.rodriguez.bahena@intel.com>,
"libc-alpha@sourceware.org" <libc-alpha@sourceware.org>
Subject: Re: [PATCH V2] benchtests: Expand range of tests names in
schema.json

>On Tuesday 21 November 2017 11:34 PM, Victor Rodriguez wrote:

>> When executing bench-math the benchmark output is invalid with this

>> error msg:

>> 

>> Invalid benchmark output: 'workload-spec2006.wrf' does not match any of

>> the regexes: '^[_a-zA-Z0-9]*$¹ or Invalid benchmark output: Additional

>> properties are not allowed ('workload-spec2006.wrf' was unexpected)

>> 

>> This patch changes regex in benchout.schema.json to accept symbols in

>> benchmark tests names. This patch came after error running

>> workload-spec2006.wrf, that detects an error in validation script, which

>> does not expected the hyphen and the dot. Other commond symbols like:

>>"," and

>> "=" are added to prevent future errors.

>

>Lets not add , and = since they're typically not part of names.  OK with

>the regex only adding a hyphen and dot.


Now I remember why I added “=" and “,: There is another test with these
symbolism its parameters:

Invalid benchmark output: 'stack=1024,guard=1', 'stack=1024,guard=2',
'stack=2048,guard=1', 'stack=2048,guard=2' do not match any of the
regexes: '^[_a-zA-Z0-9.-]*$'
make[1]: *** [Makefile:215: bench-func] Error 65


Full log: https://hastebin.com/menofisuki.pl

From : BENCHSET="bench-pthread bench-math malloc-thread”

Should I add , and = to the regex or fix the name of the test?

Thanks 

Victor 

>

>Thanks,

>Siddhesh

>

>> 

>> 	(VERSION): Set to 2.27

>>         * benchtests/scripts/benchout.schema.json: Fix regex to accept a

>>         wider range of tests names.

>> 

>> Signed-off-by: Victor Rodriguez <victor.rodriguez.bahena@intel.com>

>> ---

>>  ChangeLog                               | 6 ++++++

>>  benchtests/scripts/benchout.schema.json | 2 +-

>>  2 files changed, 7 insertions(+), 1 deletion(-)

>> 

>> diff --git a/ChangeLog b/ChangeLog

>> index 8dbfc7e..f29d180 100644

>> --- a/ChangeLog

>> +++ b/ChangeLog

>> @@ -1,3 +1,9 @@

>> +2017-11-15  Victor Rodriguez  <victor.rodriguez.bahena@intel.com>

>> +

>> +	(VERSION): Set to 2.27

>> +	* benchtests/scripts/benchout.schema.json: Fix regex to accept a

>>wider range

>> +	of tests names.

>> +

>>  2017-08-02  Siddhesh Poyarekar  <siddhesh@sourceware.org>

>>  

>>  	* version.h (RELEASE): Set to "stable"

>> diff --git a/benchtests/scripts/benchout.schema.json

>>b/benchtests/scripts/benchout.schema.json

>> index affb7c1..815ad97 100644

>> --- a/benchtests/scripts/benchout.schema.json

>> +++ b/benchtests/scripts/benchout.schema.json

>> @@ -13,7 +13,7 @@

>>            "title": "Function names",

>>            "type": "object",

>>            "patternProperties": {

>> -            "^[_a-zA-Z0-9]*$": {

>> +            "^[_a-zA-Z0-9,=.-]*$": {

>>                "title": "Function variants",

>>                "type": "object",

>>                "properties": {

>>
  

On Thursday 23 November 2017 04:29 AM, Rodriguez Bahena, Victor wrote:
> Now I remember why I added “=" and “,: There is another test with these
> symbolism its parameters:
> 
> Invalid benchmark output: 'stack=1024,guard=1', 'stack=1024,guard=2',
> 'stack=2048,guard=1', 'stack=2048,guard=2' do not match any of the
> regexes: '^[_a-zA-Z0-9.-]*$'
> make[1]: *** [Makefile:215: bench-func] Error 65
> 
> 
> Full log: https://hastebin.com/menofisuki.pl
> 
> From : BENCHSET="bench-pthread bench-math malloc-thread”
> 
> Should I add , and = to the regex or fix the name of the test?

I didn't see that go in.  I was originally going to suggest replacing it
with - and _ but the current one is far more readable.  The original
patch is OK.

Thanks,
Siddhesh