(Re: Splitting up 27_io/basic_istream/ignore/wchar_t/94749.cc (takes too long))
Checks
Commit Message
Thank you for your consideration. (Or is that phrase only used negatively?)
> From: Jonathan Wakely <jwakely@redhat.com>
> Date: Fri, 9 Jun 2023 21:40:15 +0100
> test01, test02, test03 and test04 should run almost instantly. On my system
> they take about 5 microseconds each. So I don't think splitting those up
> will help.
Right.
> I thought it would help to avoid re-allocating the buffer and zeroing it
> again. If we reuse the same buffer, then we just have to loop until we
> overflow the 32-bit counter. That would make the whole test run much
> faster, which would reduce the total time for a testsuite run. Splitting
> the file up into smaller files would not decrease the total time, only
> decrease the time for that single test so it doesn't time out.
>
> I've attached a patch that does that. I makes very little difference for
> me, probably because allocating zero-filled pages isn't actually expensive
> on linux. Maybe it will make a differene for your simulator though?
Nope, just some five seconds down (from about 10min 21s).
> You could also try reducing the size of the buffer:
> +#ifdef SIMULATOR_TEST
> + static const streamsize bufsz = 16 << limits::digits10;
> +#else
> static const streamsize bufsz = 2048 << limits::digits10;
> +#endif
Was that supposed to be with or without the patch? Anyway;
both: 606s. Only smaller bufsz: 614s. (All numbers subject
to usual system jitter.)
> test06 is the really slow part, that takes 10+ seconds for me. But that
> entire function should already be skipped for simulators.
Yep, we may have been here before... I certainly get a
deja-vu feeling here, but visiting old email conversations
of ours, it seems I easily conflate several similar ones.
I see that here, test06 was always #ifndef SIMULATOR_TEST.
> We can probably skip test05 for simulators too, none of the code it tests
> is platform-specific, so as long as it's being tested on x86 we don't
> really need to test it on cris-elf too.
Thanks. Let's do that, then. The similar s/wchar_t/char/
test clocks in at "only" 3m30s, but I suggest treating it
the same, if nothing else than for symmetry.
Ok as below?
-- >8 --
Subject: [PATCH] testsuite: Cut down 27_io/basic_istream/.../94749.cc for
simulators
The test wchar_t/94749.cc can take about 10 minutes on some
simulator/host combinations with char/94749.cc at a third of
that time. The cause is test05 which is quite heavy and
includes wrapping a 32-bit counter. Run it only for native
setups.
* testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc (main)
[! SIMULATOR_TEST]: Also exclude running test05.
* testsuite/27_io/basic_istream/ignore/char/94749.cc: Ditto.
---
libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc | 2 +-
.../testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
Comments
On Sat, 10 Jun 2023, 06:18 Hans-Peter Nilsson via Libstdc++, <
libstdc++@gcc.gnu.org> wrote:
> Thank you for your consideration. (Or is that phrase only used
> negatively?)
>
> > From: Jonathan Wakely <jwakely@redhat.com>
> > Date: Fri, 9 Jun 2023 21:40:15 +0100
>
> > test01, test02, test03 and test04 should run almost instantly. On my
> system
> > they take about 5 microseconds each. So I don't think splitting those up
> > will help.
>
> Right.
>
> > I thought it would help to avoid re-allocating the buffer and zeroing it
> > again. If we reuse the same buffer, then we just have to loop until we
> > overflow the 32-bit counter. That would make the whole test run much
> > faster, which would reduce the total time for a testsuite run. Splitting
> > the file up into smaller files would not decrease the total time, only
> > decrease the time for that single test so it doesn't time out.
> >
> > I've attached a patch that does that. I makes very little difference for
> > me, probably because allocating zero-filled pages isn't actually
> expensive
> > on linux. Maybe it will make a differene for your simulator though?
>
> Nope, just some five seconds down (from about 10min 21s).
>
Bah, worth a try :)
> > You could also try reducing the size of the buffer:
> > +#ifdef SIMULATOR_TEST
> > + static const streamsize bufsz = 16 << limits::digits10;
> > +#else
> > static const streamsize bufsz = 2048 << limits::digits10;
> > +#endif
>
> Was that supposed to be with or without the patch? Anyway;
> both: 606s. Only smaller bufsz: 614s. (All numbers subject
> to usual system jitter.)
>
> > test06 is the really slow part, that takes 10+ seconds for me. But that
> > entire function should already be skipped for simulators.
>
> Yep, we may have been here before... I certainly get a
> deja-vu feeling here, but visiting old email conversations
> of ours, it seems I easily conflate several similar ones.
> I see that here, test06 was always #ifndef SIMULATOR_TEST.
>
> > We can probably skip test05 for simulators too, none of the code it tests
> > is platform-specific, so as long as it's being tested on x86 we don't
> > really need to test it on cris-elf too.
>
> Thanks. Let's do that, then. The similar s/wchar_t/char/
> test clocks in at "only" 3m30s, but I suggest treating it
> the same, if nothing else than for symmetry.
>
> Ok as below?
>
OK for trunk, and all branches you care about.
> -- >8 --
> Subject: [PATCH] testsuite: Cut down 27_io/basic_istream/.../94749.cc for
> simulators
>
> The test wchar_t/94749.cc can take about 10 minutes on some
> simulator/host combinations with char/94749.cc at a third of
> that time. The cause is test05 which is quite heavy and
> includes wrapping a 32-bit counter. Run it only for native
> setups.
>
> * testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc (main)
> [! SIMULATOR_TEST]: Also exclude running test05.
> * testsuite/27_io/basic_istream/ignore/char/94749.cc: Ditto.
> ---
> libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc | 2 +-
> .../testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git
> a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> index 6416863983b7..9160995c05ec 100644
> --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> @@ -221,8 +221,8 @@ main()
> test02();
> test03();
> test04();
> - test05();
> #ifndef SIMULATOR_TEST
> + test05();
> test06();
> #endif
> }
> diff --git
> a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> index 65e0a326c109..a5b9eb71a389 100644
> --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> @@ -221,8 +221,8 @@ main()
> test02();
> test03();
> test04();
> - test05();
> #ifndef SIMULATOR_TEST
> + test05();
> test06();
> #endif
> }
> --
> 2.30.2
>
>
Hi,
I have noticed that in gcc-13, test05 (in the 94749.cc testcase) is still enabled for simulators, and I have noticed that because of test05, the 27_io/basic_istream/ignore/char/94749.cc execution test is not terminating on our simulator for armv8.1-m.main+mve, even after 3 hours.
The execution test was passing before this commit :
https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=e30211cb0b3a2b88959e9bc40626a17461de52de
Could you please provide some hints or ideas as to what might be causing this regression?
I imagine that the issue could be with the simulator or with code-gen. However, could this also highlight a different issue in test05? Is this testing a commonly used feature or area of the compiler? And would it be worth re-including it for simulators?
Kind regards,
Vasee
> From: Vaseeharan Vinayagamoorthy <Vaseeharan.Vinayagamoorthy@arm.com>
> Date: Sat, 22 Jun 2024 01:38:09 +0000
Sorry for the late reply. I sort of hoped somebody else
would chime in. Maybe the issue has resolved itself in the
meantime?
> Hi,
>
> I have noticed that in gcc-13, test05 (in the 94749.cc
> testcase) is still enabled for simulators, and I have
> noticed that because of test05, the
> 27_io/basic_istream/ignore/char/94749.cc execution test is
> not terminating on our simulator for armv8.1-m.main+mve,
> even after 3 hours.
>
> The execution test was passing before this commit :
> https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=e30211cb0b3a2b88959e9bc40626a17461de52de
>
> Could you please provide some hints or ideas as to what
> might be causing this regression?
No, sorry. If I were you and in this situation: IIUC with a
noticeable codegen regression around a specific commit (not
"just" say a percent pushing it over a timeout), I'd analyze
it with regards to the actual code regression around that
commit. Though, that's just the usual leg-work when this
kind of regression happens: I have no insight specific to
this test. I see no easy way around that hard work here.
> I imagine that the issue could be with the simulator or
> with code-gen. However, could this also highlight a
> different issue in test05?
It could, but I'm guessing that commit just caused a codegen
regression, perhaps even generating incorrect code to the
effect of an infinite loop. Or is somehow it that usual ARM
caveat: default unsigned char?
> Is this testing a commonly used
> feature or area of the compiler?
All I know is that the intent is specific to functionality
in libstdc++-v3.
> And would it be worth
> re-including it for simulators?
IMHO: only if you somehow make it ARM-specific. It'd be bad
practice to "re-enable" it for all simulator targets only
because it exposes an uninvestigated issue for one specific
configuration, and a timeout at that.
Alternatively (after analysis), the SOP is to put a derived
minimal testcase in the *generic* parts of the test-suite (C
or C++, as a runtime test) unless the compiled code really
only runs on an ARM, in which case it goes in gcc.target/arm
or g++.target/arm.
HTH.
brgds, H-P
>
> Kind regards,
> Vasee
>
> ________________________________________
> From: Libstdc++ <libstdc++-bounces+vvinayag=arm.com@gcc.gnu.org> on behalf of Jonathan Wakely via Libstdc++ <libstdc++@gcc.gnu.org>
> Sent: 10 June 2023 08:12
> To: Hans-Peter Nilsson
> Cc: Jonathan Wakely; libstdc++; gcc-patches
> Subject: Re: [PATCH] (Re: Splitting up 27_io/basic_istream/ignore/wchar_t/94749.cc (takes too long))
>
> On Sat, 10 Jun 2023, 06:18 Hans-Peter Nilsson via Libstdc++, <
> libstdc++@gcc.gnu.org> wrote:
>
> > Thank you for your consideration. (Or is that phrase only used
> > negatively?)
> >
> > > From: Jonathan Wakely <jwakely@redhat.com>
> > > Date: Fri, 9 Jun 2023 21:40:15 +0100
> >
> > > test01, test02, test03 and test04 should run almost instantly. On my
> > system
> > > they take about 5 microseconds each. So I don't think splitting those up
> > > will help.
> >
> > Right.
> >
> > > I thought it would help to avoid re-allocating the buffer and zeroing it
> > > again. If we reuse the same buffer, then we just have to loop until we
> > > overflow the 32-bit counter. That would make the whole test run much
> > > faster, which would reduce the total time for a testsuite run. Splitting
> > > the file up into smaller files would not decrease the total time, only
> > > decrease the time for that single test so it doesn't time out.
> > >
> > > I've attached a patch that does that. I makes very little difference for
> > > me, probably because allocating zero-filled pages isn't actually
> > expensive
> > > on linux. Maybe it will make a differene for your simulator though?
> >
> > Nope, just some five seconds down (from about 10min 21s).
> >
>
> Bah, worth a try :)
>
>
> > > You could also try reducing the size of the buffer:
> > > +#ifdef SIMULATOR_TEST
> > > + static const streamsize bufsz = 16 << limits::digits10;
> > > +#else
> > > static const streamsize bufsz = 2048 << limits::digits10;
> > > +#endif
> >
> > Was that supposed to be with or without the patch? Anyway;
> > both: 606s. Only smaller bufsz: 614s. (All numbers subject
> > to usual system jitter.)
> >
> > > test06 is the really slow part, that takes 10+ seconds for me. But that
> > > entire function should already be skipped for simulators.
> >
> > Yep, we may have been here before... I certainly get a
> > deja-vu feeling here, but visiting old email conversations
> > of ours, it seems I easily conflate several similar ones.
> > I see that here, test06 was always #ifndef SIMULATOR_TEST.
> >
> > > We can probably skip test05 for simulators too, none of the code it tests
> > > is platform-specific, so as long as it's being tested on x86 we don't
> > > really need to test it on cris-elf too.
> >
> > Thanks. Let's do that, then. The similar s/wchar_t/char/
> > test clocks in at "only" 3m30s, but I suggest treating it
> > the same, if nothing else than for symmetry.
> >
> > Ok as below?
> >
>
> OK for trunk, and all branches you care about.
>
>
>
> > -- >8 --
> > Subject: [PATCH] testsuite: Cut down 27_io/basic_istream/.../94749.cc for
> > simulators
> >
> > The test wchar_t/94749.cc can take about 10 minutes on some
> > simulator/host combinations with char/94749.cc at a third of
> > that time. The cause is test05 which is quite heavy and
> > includes wrapping a 32-bit counter. Run it only for native
> > setups.
> >
> > * testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc (main)
> > [! SIMULATOR_TEST]: Also exclude running test05.
> > * testsuite/27_io/basic_istream/ignore/char/94749.cc: Ditto.
> > ---
> > libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc | 2 +-
> > .../testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc | 2 +-
> > 2 files changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git
> > a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> > b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> > index 6416863983b7..9160995c05ec 100644
> > --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> > +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> > @@ -221,8 +221,8 @@ main()
> > test02();
> > test03();
> > test04();
> > - test05();
> > #ifndef SIMULATOR_TEST
> > + test05();
> > test06();
> > #endif
> > }
> > diff --git
> > a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> > b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> > index 65e0a326c109..a5b9eb71a389 100644
> > --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> > +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> > @@ -221,8 +221,8 @@ main()
> > test02();
> > test03();
> > test04();
> > - test05();
> > #ifndef SIMULATOR_TEST
> > + test05();
> > test06();
> > #endif
> > }
> > --
> > 2.30.2
> >
> >
>
On Wed, 14 Aug 2024 at 15:47, Hans-Peter Nilsson <hp@axis.com> wrote:
>
> > From: Vaseeharan Vinayagamoorthy <Vaseeharan.Vinayagamoorthy@arm.com>
> > Date: Sat, 22 Jun 2024 01:38:09 +0000
>
> Sorry for the late reply. I sort of hoped somebody else
> would chime in. Maybe the issue has resolved itself in the
> meantime?
>
> > Hi,
> >
> > I have noticed that in gcc-13, test05 (in the 94749.cc
> > testcase) is still enabled for simulators, and I have
> > noticed that because of test05, the
> > 27_io/basic_istream/ignore/char/94749.cc execution test is
> > not terminating on our simulator for armv8.1-m.main+mve,
> > even after 3 hours.
> >
> > The execution test was passing before this commit :
> > https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=e30211cb0b3a2b88959e9bc40626a17461de52de
> >
> > Could you please provide some hints or ideas as to what
> > might be causing this regression?
>
> No, sorry. If I were you and in this situation: IIUC with a
> noticeable codegen regression around a specific commit (not
> "just" say a percent pushing it over a timeout), I'd analyze
> it with regards to the actual code regression around that
> commit. Though, that's just the usual leg-work when this
> kind of regression happens: I have no insight specific to
> this test. I see no easy way around that hard work here.
>
> > I imagine that the issue could be with the simulator or
> > with code-gen. However, could this also highlight a
> > different issue in test05?
>
> It could, but I'm guessing that commit just caused a codegen
> regression, perhaps even generating incorrect code to the
> effect of an infinite loop. Or is somehow it that usual ARM
> caveat: default unsigned char?
>
> > Is this testing a commonly used
> > feature or area of the compiler?
>
> All I know is that the intent is specific to functionality
> in libstdc++-v3.
Right, it should not be testing *anything* in the compiler, beyond
relying on "the compiler can compile C++ code correctly".
>
> > And would it be worth
> > re-including it for simulators?
>
> IMHO: only if you somehow make it ARM-specific. It'd be bad
> practice to "re-enable" it for all simulator targets only
> because it exposes an uninvestigated issue for one specific
> configuration, and a timeout at that.
>
> Alternatively (after analysis), the SOP is to put a derived
> minimal testcase in the *generic* parts of the test-suite (C
> or C++, as a runtime test) unless the compiled code really
> only runs on an ARM, in which case it goes in gcc.target/arm
> or g++.target/arm.
>
> HTH.
>
> brgds, H-P
>
>
> >
> > Kind regards,
> > Vasee
> >
> > ________________________________________
> > From: Libstdc++ <libstdc++-bounces+vvinayag=arm.com@gcc.gnu.org> on behalf of Jonathan Wakely via Libstdc++ <libstdc++@gcc.gnu.org>
> > Sent: 10 June 2023 08:12
> > To: Hans-Peter Nilsson
> > Cc: Jonathan Wakely; libstdc++; gcc-patches
> > Subject: Re: [PATCH] (Re: Splitting up 27_io/basic_istream/ignore/wchar_t/94749.cc (takes too long))
> >
> > On Sat, 10 Jun 2023, 06:18 Hans-Peter Nilsson via Libstdc++, <
> > libstdc++@gcc.gnu.org> wrote:
> >
> > > Thank you for your consideration. (Or is that phrase only used
> > > negatively?)
> > >
> > > > From: Jonathan Wakely <jwakely@redhat.com>
> > > > Date: Fri, 9 Jun 2023 21:40:15 +0100
> > >
> > > > test01, test02, test03 and test04 should run almost instantly. On my
> > > system
> > > > they take about 5 microseconds each. So I don't think splitting those up
> > > > will help.
> > >
> > > Right.
> > >
> > > > I thought it would help to avoid re-allocating the buffer and zeroing it
> > > > again. If we reuse the same buffer, then we just have to loop until we
> > > > overflow the 32-bit counter. That would make the whole test run much
> > > > faster, which would reduce the total time for a testsuite run. Splitting
> > > > the file up into smaller files would not decrease the total time, only
> > > > decrease the time for that single test so it doesn't time out.
> > > >
> > > > I've attached a patch that does that. I makes very little difference for
> > > > me, probably because allocating zero-filled pages isn't actually
> > > expensive
> > > > on linux. Maybe it will make a differene for your simulator though?
> > >
> > > Nope, just some five seconds down (from about 10min 21s).
> > >
> >
> > Bah, worth a try :)
> >
> >
> > > > You could also try reducing the size of the buffer:
> > > > +#ifdef SIMULATOR_TEST
> > > > + static const streamsize bufsz = 16 << limits::digits10;
> > > > +#else
> > > > static const streamsize bufsz = 2048 << limits::digits10;
> > > > +#endif
> > >
> > > Was that supposed to be with or without the patch? Anyway;
> > > both: 606s. Only smaller bufsz: 614s. (All numbers subject
> > > to usual system jitter.)
> > >
> > > > test06 is the really slow part, that takes 10+ seconds for me. But that
> > > > entire function should already be skipped for simulators.
> > >
> > > Yep, we may have been here before... I certainly get a
> > > deja-vu feeling here, but visiting old email conversations
> > > of ours, it seems I easily conflate several similar ones.
> > > I see that here, test06 was always #ifndef SIMULATOR_TEST.
> > >
> > > > We can probably skip test05 for simulators too, none of the code it tests
> > > > is platform-specific, so as long as it's being tested on x86 we don't
> > > > really need to test it on cris-elf too.
> > >
> > > Thanks. Let's do that, then. The similar s/wchar_t/char/
> > > test clocks in at "only" 3m30s, but I suggest treating it
> > > the same, if nothing else than for symmetry.
> > >
> > > Ok as below?
> > >
> >
> > OK for trunk, and all branches you care about.
> >
> >
> >
> > > -- >8 --
> > > Subject: [PATCH] testsuite: Cut down 27_io/basic_istream/.../94749.cc for
> > > simulators
> > >
> > > The test wchar_t/94749.cc can take about 10 minutes on some
> > > simulator/host combinations with char/94749.cc at a third of
> > > that time. The cause is test05 which is quite heavy and
> > > includes wrapping a 32-bit counter. Run it only for native
> > > setups.
> > >
> > > * testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc (main)
> > > [! SIMULATOR_TEST]: Also exclude running test05.
> > > * testsuite/27_io/basic_istream/ignore/char/94749.cc: Ditto.
> > > ---
> > > libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc | 2 +-
> > > .../testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc | 2 +-
> > > 2 files changed, 2 insertions(+), 2 deletions(-)
> > >
> > > diff --git
> > > a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> > > b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> > > index 6416863983b7..9160995c05ec 100644
> > > --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> > > +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/char/94749.cc
> > > @@ -221,8 +221,8 @@ main()
> > > test02();
> > > test03();
> > > test04();
> > > - test05();
> > > #ifndef SIMULATOR_TEST
> > > + test05();
> > > test06();
> > > #endif
> > > }
> > > diff --git
> > > a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> > > b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> > > index 65e0a326c109..a5b9eb71a389 100644
> > > --- a/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> > > +++ b/libstdc++-v3/testsuite/27_io/basic_istream/ignore/wchar_t/94749.cc
> > > @@ -221,8 +221,8 @@ main()
> > > test02();
> > > test03();
> > > test04();
> > > - test05();
> > > #ifndef SIMULATOR_TEST
> > > + test05();
> > > test06();
> > > #endif
> > > }
> > > --
> > > 2.30.2
> > >
> > >
> >
>
@@ -221,8 +221,8 @@ main()
test02();
test03();
test04();
- test05();
#ifndef SIMULATOR_TEST
+ test05();
test06();
#endif
}
@@ -221,8 +221,8 @@ main()
test02();
test03();
test04();
- test05();
#ifndef SIMULATOR_TEST
+ test05();
test06();
#endif
}