Tweak test-*exit-race tests to honor the maximum number of processes
Commit Message
When the number of threads created by test-*exit-race tests is close
to the maximum number of simultaneous processes per uid, these tests
fail with the following diagnostics:
error: xpthread_check_return.c:32: pthread_create: Resource temporarily unavailable
Workaround this issue by limiting the number of threads created
by these tests to sysconf (_SC_CHILD_MAX) / 2.
Tested that these tests still fail reliably on x86_64 with glibc-2.26
and sysconf (_SC_CHILD_MAX) == 512.
* stdlib/test-atexit-race-common.c: Include <unistd.h>.
(kNumThreads): Remove const qualifier.
(do_test): Limit kNumThreads to sysconf (_SC_CHILD_MAX) / 2.
---
ChangeLog | 6 ++++++
stdlib/test-atexit-race-common.c | 10 ++++++++--
2 files changed, 14 insertions(+), 2 deletions(-)
Comments
On 01/30/2018 08:54 AM, Dmitry V. Levin wrote:
> When the number of threads created by test-*exit-race tests is close
> to the maximum number of simultaneous processes per uid, these tests
> fail with the following diagnostics:
>
> error: xpthread_check_return.c:32: pthread_create: Resource temporarily unavailable
>
> Workaround this issue by limiting the number of threads created
> by these tests to sysconf (_SC_CHILD_MAX) / 2.
>
> Tested that these tests still fail reliably on x86_64 with glibc-2.26
> and sysconf (_SC_CHILD_MAX) == 512.
>
> * stdlib/test-atexit-race-common.c: Include <unistd.h>.
> (kNumThreads): Remove const qualifier.
> (do_test): Limit kNumThreads to sysconf (_SC_CHILD_MAX) / 2.
The test should return UNSUPPORTED if an expected minimum number of threads
required to reliably trigger the test cannot be created e.g. 512.
> ---
> ChangeLog | 6 ++++++
> stdlib/test-atexit-race-common.c | 10 ++++++++--
> 2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/stdlib/test-atexit-race-common.c b/stdlib/test-atexit-race-common.c
> index 4d7f911..7a65264 100644
> --- a/stdlib/test-atexit-race-common.c
> +++ b/stdlib/test-atexit-race-common.c
> @@ -33,10 +33,11 @@
>
> #include <stdio.h>
> #include <stdlib.h>
> +#include <unistd.h>
> #include <support/xthread.h>
>
> -const size_t kNumThreads = 1024;
> -const size_t kNumHandlers = 1024;
> +static size_t kNumThreads = 1024;
> +static const size_t kNumHandlers = 1024;
>
> static void *
> threadfunc (void *unused)
> @@ -62,6 +63,11 @@ do_test (void)
> 128KiB for a maximum required VM size of 128MiB. */
> xpthread_attr_setstacksize (&attr, 128 * 1024);
>
> + long nproc = sysconf (_SC_CHILD_MAX);
> +
> + if (nproc > 0 && nproc / 2 < kNumThreads)
> + kNumThreads = nproc / 2;
> +
> for (i = 0; i < kNumThreads; ++i) {
> xpthread_create (&attr, threadfunc, NULL);
> }
>
On Tue, Jan 30, 2018 at 12:04:06PM -0800, Carlos O'Donell wrote:
> On 01/30/2018 08:54 AM, Dmitry V. Levin wrote:
> > When the number of threads created by test-*exit-race tests is close
> > to the maximum number of simultaneous processes per uid, these tests
> > fail with the following diagnostics:
> >
> > error: xpthread_check_return.c:32: pthread_create: Resource temporarily unavailable
> >
> > Workaround this issue by limiting the number of threads created
> > by these tests to sysconf (_SC_CHILD_MAX) / 2.
> >
> > Tested that these tests still fail reliably on x86_64 with glibc-2.26
> > and sysconf (_SC_CHILD_MAX) == 512.
> >
> > * stdlib/test-atexit-race-common.c: Include <unistd.h>.
> > (kNumThreads): Remove const qualifier.
> > (do_test): Limit kNumThreads to sysconf (_SC_CHILD_MAX) / 2.
>
> The test should return UNSUPPORTED if an expected minimum number of threads
> required to reliably trigger the test cannot be created e.g. 512.
It depends on what do you call reliably. For example, on an x86_64 box
with `getconf _NPROCESSORS_ONLN` == 32 the chance of test-atexit-race
*not* to trigger a segfault with glibc-2.26 was about 0.08% with current
value of kNumThreads == 1024, and this chance increased roughly 4 times
to about 0.34% with kNumThreads == 256.
On 01/30/2018 03:36 PM, Dmitry V. Levin wrote:
> On Tue, Jan 30, 2018 at 12:04:06PM -0800, Carlos O'Donell wrote:
>> On 01/30/2018 08:54 AM, Dmitry V. Levin wrote:
>>> When the number of threads created by test-*exit-race tests is close
>>> to the maximum number of simultaneous processes per uid, these tests
>>> fail with the following diagnostics:
>>>
>>> error: xpthread_check_return.c:32: pthread_create: Resource temporarily unavailable
>>>
>>> Workaround this issue by limiting the number of threads created
>>> by these tests to sysconf (_SC_CHILD_MAX) / 2.
>>>
>>> Tested that these tests still fail reliably on x86_64 with glibc-2.26
>>> and sysconf (_SC_CHILD_MAX) == 512.
>>>
>>> * stdlib/test-atexit-race-common.c: Include <unistd.h>.
>>> (kNumThreads): Remove const qualifier.
>>> (do_test): Limit kNumThreads to sysconf (_SC_CHILD_MAX) / 2.
>>
>> The test should return UNSUPPORTED if an expected minimum number of threads
>> required to reliably trigger the test cannot be created e.g. 512.
>
> It depends on what do you call reliably. For example, on an x86_64 box
> with `getconf _NPROCESSORS_ONLN` == 32 the chance of test-atexit-race
> *not* to trigger a segfault with glibc-2.26 was about 0.08% with current
> value of kNumThreads == 1024, and this chance increased roughly 4 times
> to about 0.34% with kNumThreads == 256.
Reliably is > 50% of the time in my opinion. This means that if you sample
the test 3-5 times you are almost guaranteed a failure (87.5-97%, multiplicative
statistics).
@@ -33,10 +33,11 @@
#include <stdio.h>
#include <stdlib.h>
+#include <unistd.h>
#include <support/xthread.h>
-const size_t kNumThreads = 1024;
-const size_t kNumHandlers = 1024;
+static size_t kNumThreads = 1024;
+static const size_t kNumHandlers = 1024;
static void *
threadfunc (void *unused)
@@ -62,6 +63,11 @@ do_test (void)
128KiB for a maximum required VM size of 128MiB. */
xpthread_attr_setstacksize (&attr, 128 * 1024);
+ long nproc = sysconf (_SC_CHILD_MAX);
+
+ if (nproc > 0 && nproc / 2 < kNumThreads)
+ kNumThreads = nproc / 2;
+
for (i = 0; i < kNumThreads; ++i) {
xpthread_create (&attr, threadfunc, NULL);
}